<div dir="ltr"><div>Phil,</div><div><br></div><div>Thanks for your response. I'm specifically concerned about the context of passing data between threads using the signal/slot mechanism. As you say, in the direct or blocking case, there's no concern with object lifetime.</div><div><br></div><div>To test this, I modified Ilya's code to make it easier to test the signal/slot mechanism on different types. A QObject fails, but standard Python types (dict, list, object, etc.) pass, showing that their reference count increases after an emit. I can even pass a list containing a QObject safely. This suggests to me that your code increases the reference count when emitting pure Python objects, but not when emitting Qt objects. Based on some digging (see below), I would expect Qt objects to arrive safely to the slot because they're copied, but this doesn't work in practice. <b>What's safe and what's not safe when using the signal/slot mechanism when crossing thread boundaries?</b></div><div><br></div>Looking at the Qt source, from what I understand:<div><br></div><div>1. In qobject.cpp, the <a href="https://code.woboq.org/qt5/qtbase/src/corelib/kernel/qobject.cpp.html#_ZL15queued_activateP7QObjectiPN14QObjectPrivate10ConnectionEPPvR12QMutexLocker" target="_blank">queued_activate</a> function is used to post an signal to another thread's event queue. (See <a href="https://code.woboq.org/qt5/qtbase/src/corelib/kernel/qobject.cpp.html#_ZN11QMetaObject8activateEP7QObjectiiPPv" target="_blank">qobject.cpp::activate</a> for the generic mechanism used to emit a signal, which calls queued_activate for queued connections). Here's a helpful <a href="https://woboq.com/blog/how-qt-signals-slots-work-part3-queuedconnection.html" target="_blank">blog post on the topic</a>.</div><div>2. To do this, queued_activate makes a copy of each argument by <a href="https://code.woboq.org/qt5/qtbase/src/corelib/kernel/qobject.cpp.html#3575" target="_blank">invoking QMetaType::create</a> (which invokes that type's copy constructor) in the signal being emitted.</div><div>3. When all signals have been delivered, <a href="https://code.woboq.org/qt5/qtbase/src/corelib/kernel/qobject.cpp.html#_ZN14QMetaCallEventD1Ev" target="_blank">QMetaCallEvent::destroy</a> frees memory used by this copy.</div><div><br></div><div>Based on this, I conclude that Qt allows the emission of signals with any type registered with the Qt meta-type system by copying the type, delivering it to all slots, then destroying the copy.</div><div><br></div><div>Thanks!<br></div><div><br></div><div>Bryan</div><div class="gmail_extra"><br></div><div class="gmail_extra"><br></div><div class="gmail_extra"><div class="gmail_extra"><font face="monospace, monospace">import time</font></div><div class="gmail_extra"><font face="monospace, monospace">import sys</font></div><div class="gmail_extra"><font face="monospace, monospace">import gc</font></div><div class="gmail_extra"><font face="monospace, monospace">import os</font></div><div class="gmail_extra"><font face="monospace, monospace">import platform</font></div><div class="gmail_extra"><font face="monospace, monospace"><br></font></div><div class="gmail_extra"><font face="monospace, monospace">import sip</font></div><div class="gmail_extra"><font face="monospace, monospace">from PyQt5.QtCore import QT_VERSION_STR</font></div><div class="gmail_extra"><font face="monospace, monospace">from PyQt5.Qt import PYQT_VERSION_STR</font></div><div class="gmail_extra"><font face="monospace, monospace">from PyQt5.QtCore import QObject, QThread, pyqtSignal, pyqtSlot</font></div><div class="gmail_extra"><font face="monospace, monospace">from PyQt5.QtCore import qDebug, Qt</font></div><div class="gmail_extra"><font face="monospace, monospace">from PyQt5.QtWidgets import QApplication, QWidget</font></div><div class="gmail_extra"><font face="monospace, monospace"><br></font></div><div class="gmail_extra"><font face="monospace, monospace"># Change to set the type of object passed between threads.</font></div><div class="gmail_extra"><font face="monospace, monospace">objType = QObject</font></div><div class="gmail_extra"><font face="monospace, monospace"><br></font></div><div class="gmail_extra"><font face="monospace, monospace">def printObject(obj):</font></div><div class="gmail_extra"><font face="monospace, monospace"> qDebug('{} {:#X} {}'.format(type(obj), id(obj), obj))</font></div><div class="gmail_extra"><font face="monospace, monospace"><br></font></div><div class="gmail_extra"><font face="monospace, monospace">class SomeThread(QObject):</font></div><div class="gmail_extra"><font face="monospace, monospace"> signal = pyqtSignal(objType)</font></div><div class="gmail_extra"><font face="monospace, monospace"><br></font></div><div class="gmail_extra"><font face="monospace, monospace"> def __init__(self):</font></div><div class="gmail_extra"><font face="monospace, monospace"> QObject.__init__(self)</font></div><div class="gmail_extra"><font face="monospace, monospace"> self.thread = QThread()</font></div><div class="gmail_extra"><font face="monospace, monospace"> self.wait = self.thread.wait</font></div><div class="gmail_extra"><font face="monospace, monospace"> self.moveToThread(self.thread)</font></div><div class="gmail_extra"><font face="monospace, monospace"> self.signal.connect(self.signalHandler, type = Qt.QueuedConnection)</font></div><div class="gmail_extra"><font face="monospace, monospace"> self.thread.started.connect(self.run)</font></div><div class="gmail_extra"><font face="monospace, monospace"> self.thread.start()</font></div><div class="gmail_extra"><font face="monospace, monospace"><br></font></div><div class="gmail_extra"><font face="monospace, monospace"> @pyqtSlot(objType)</font></div><div class="gmail_extra"><font face="monospace, monospace"> def signalHandler(self, obj):</font></div><div class="gmail_extra"><font face="monospace, monospace"> qDebug('In signalHandler.')</font></div><div class="gmail_extra"><font face="monospace, monospace"> printObject(obj)</font></div><div class="gmail_extra"><font face="monospace, monospace"> qDebug("signalHandler obj ref counts: " + str(sys.getrefcount(obj)))</font></div><div class="gmail_extra"><font face="monospace, monospace"><br></font></div><div class="gmail_extra"><font face="monospace, monospace"> @pyqtSlot()</font></div><div class="gmail_extra"><font face="monospace, monospace"> def run(self):</font></div><div class="gmail_extra"><font face="monospace, monospace"> qDebug("Thread sleeps for 0.1 sec")</font></div><div class="gmail_extra"><font face="monospace, monospace"> time.sleep(0.1)</font></div><div class="gmail_extra"><font face="monospace, monospace"> qDebug("Process events now")</font></div><div class="gmail_extra"><font face="monospace, monospace"> QApplication.processEvents()</font></div><div class="gmail_extra"><font face="monospace, monospace"> self.thread.exit()</font></div><div class="gmail_extra"><font face="monospace, monospace"><br></font></div><div class="gmail_extra"><font face="monospace, monospace">def someFunc(objType, th):</font></div><div class="gmail_extra"><font face="monospace, monospace"> obj = objType()</font></div><div class="gmail_extra"><font face="monospace, monospace"> printObject(obj)</font></div><div class="gmail_extra"><font face="monospace, monospace"> qDebug("obj ref counts before emit: " + str(sys.getrefcount(obj)))</font></div><div class="gmail_extra"><font face="monospace, monospace"> th.signal.emit(obj)</font></div><div class="gmail_extra"><font face="monospace, monospace"> qDebug("obj ref counts after emit: " + str(sys.getrefcount(obj)))</font></div><div class="gmail_extra"><font face="monospace, monospace"><br></font></div><div class="gmail_extra"><font face="monospace, monospace">app = None</font></div><div class="gmail_extra"><font face="monospace, monospace">def main():</font></div><div class="gmail_extra"><font face="monospace, monospace"> print('Qt {}, SIP {}, PtQt {}, Python {}, OS {}, platform {} {}\n'.format(</font></div><div class="gmail_extra"><font face="monospace, monospace"> QT_VERSION_STR, sip.SIP_VERSION_STR, PYQT_VERSION_STR, sys.version,</font></div><div class="gmail_extra"><font face="monospace, monospace"> <a href="http://os.name">os.name</a>, platform.system(), platform.release()))</font></div><div class="gmail_extra"><font face="monospace, monospace"><br></font></div><div class="gmail_extra"><font face="monospace, monospace"> global app</font></div><div class="gmail_extra"><font face="monospace, monospace"> app = QApplication([])</font></div><div class="gmail_extra"><font face="monospace, monospace"> for x in range(100):</font></div><div class="gmail_extra"><font face="monospace, monospace"> print('\n\nTest {}'.format(x + 1))</font></div><div class="gmail_extra"><font face="monospace, monospace"> testType(objType)</font></div><div class="gmail_extra"><font face="monospace, monospace"><br></font></div><div class="gmail_extra"><font face="monospace, monospace">def testType(objType):</font></div><div class="gmail_extra"><font face="monospace, monospace"> th = SomeThread()</font></div><div class="gmail_extra"><font face="monospace, monospace"><br></font></div><div class="gmail_extra"><font face="monospace, monospace"> someFunc(objType, th)</font></div><div class="gmail_extra"><font face="monospace, monospace"> gc.collect()</font></div><div class="gmail_extra"><font face="monospace, monospace"> th.thread.wait()</font></div><div class="gmail_extra"><font face="monospace, monospace"><br></font></div><div class="gmail_extra"><font face="monospace, monospace">if __name__ == '__main__':</font></div><div class="gmail_extra"><font face="monospace, monospace"> main()</font></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Feb 12, 2016 at 1:57 PM, Jones, Bryan <span dir="ltr"><<a href="mailto:bjones@ece.msstate.edu" target="_blank">bjones@ece.msstate.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">I'd like to thank Ilya for bringing this to light. I thought I'd worked around a similar problem, but I now see that I mistakenly assumed Python semantics where C++ semantics apply. To make sure I understand correctly:<div><br></div><div>1. If a Python type can converted to a pass-by-value C++ type (int, long, etc.) then it will be passed as the C++ value through the signal/slot mechanism, meaning I don't need to retain a Python reference to it to keep it valid.</div><div>2. If a Python type cannot be converted to a pass-by-value C++ type (string, big int, dict, object, etc.) then a pointer to it will be passed through the signal/slot mechanism, meaning I must retain a Python reference until the signal has been delivered to all slots.</div><div><br></div><div>However, I do see Qt employing pointers (QFileSystemWatcher's signals emit const QString&, QLabel::setMovie(QMovie*). Does this imply that the emitters of these signals can somehow "know" when it's free to destroy the QString they emitted? Or that developers which use QFileSystemWatcher should not destroy it until all of its signals have been delivered? This seems hard to do. How do C++ developers using Qt do this? Perhaps I can learn from their approach.</div><div><br></div><div>Bryan</div></div><div class="gmail_extra"><div><div><br><div class="gmail_quote">On Fri, Feb 12, 2016 at 3:43 AM, Phil Thompson <span dir="ltr"><<a href="mailto:phil@riverbankcomputing.com" target="_blank">phil@riverbankcomputing.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><span>On 11 Feb 2016, at 7:15 pm, Ilya Kulakov <<a href="mailto:kulakov.ilya@gmail.com" target="_blank">kulakov.ilya@gmail.com</a>> wrote:<br>
><br>
> Phil,<br>
><br>
</span><span>> How does Qt's automatic unsubscribing in QObject's destructor works then?<br>
><br>
> Best Regards<br>
> Ilya Kulakov<br>
<br>
</span>Unsubscribing to what?<br>
<br>
Qt maintains lots of internal data structures about connections, but these aren't exposed. As far as I know Qt does not track pointers to QObjects when they are sitting in a thread's event queue.<br>
<span><font color="#888888"><br>
Phil<br>
</font></span><div><div><br>
>> On 12 февр. 2016 г., at 0:30, Phil Thompson <<a href="mailto:phil@riverbankcomputing.com" target="_blank">phil@riverbankcomputing.com</a>> wrote:<br>
>><br>
>><br>
>>> On 11 Feb 2016, at 6:16 pm, Ilya Kulakov <<a href="mailto:kulakov.ilya@gmail.com" target="_blank">kulakov.ilya@gmail.com</a>> wrote:<br>
>>><br>
>>> Phil,<br>
>>><br>
>>> I said I don't know all the details :)<br>
>>><br>
>>> PyQt "controls" both signals and slots. Doesn't it know how many receivers are there before it sends?<br>
>><br>
>> The short answer to the question is no. Also, with queued connections, a signal may be sitting in an unprocessed event queue for an indeterminate amount of time. In network terms, signals are UDP, not TCP.<br>
>><br>
>> Phil<br>
><br>
<br>
_______________________________________________<br>
PyQt mailing list <a href="mailto:PyQt@riverbankcomputing.com" target="_blank">PyQt@riverbankcomputing.com</a><br>
<a href="https://www.riverbankcomputing.com/mailman/listinfo/pyqt" rel="noreferrer" target="_blank">https://www.riverbankcomputing.com/mailman/listinfo/pyqt</a></div></div></blockquote></div><br><br clear="all"><div><br></div></div></div>-- <br><div>Bryan A. Jones, Ph.D.<br>Associate Professor<br>Department of Electrical and Computer Engineering<br>231 Simrall / PO Box 9571<br>Mississippi State University<br>Mississippi state, MS 39762<br><a href="http://www.ece.msstate.edu/~bjones" target="_blank">http://www.ece.msstate.edu/~bjones</a><br>bjones AT ece DOT msstate DOT edu<br>voice <a href="tel:662-325-3149" value="+16623253149" target="_blank">662-325-3149</a><br>fax <a href="tel:662-325-2298" value="+16623252298" target="_blank">662-325-2298</a><br><br>Our Master, Jesus Christ, is on his way. He'll show up right on<br>time, his arrival guaranteed by the Blessed and Undisputed Ruler,<br>High King, High God.<br>- 1 Tim. 6:14b-15 (The Message)<br></div>
</div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div>Bryan A. Jones, Ph.D.<br>Associate Professor<br>Department of Electrical and Computer Engineering<br>231 Simrall / PO Box 9571<br>Mississippi State University<br>Mississippi state, MS 39762<br><a href="http://www.ece.msstate.edu/~bjones" target="_blank">http://www.ece.msstate.edu/~bjones</a><br>bjones AT ece DOT msstate DOT edu<br>voice <a href="tel:662-325-3149" value="+16623253149" target="_blank">662-325-3149</a><br>fax <a href="tel:662-325-2298" value="+16623252298" target="_blank">662-325-2298</a><br><br>Our Master, Jesus Christ, is on his way. He'll show up right on<br>time, his arrival guaranteed by the Blessed and Undisputed Ruler,<br>High King, High God.<br>- 1 Tim. 6:14b-15 (The Message)<br></div>
</div></div>