[PyKDE] Threading problem with PyQt4 and Macosx

[Kevin Cureton]

This may or may not be related. Alas, it was a few months ago, so the  
information may be irrelevant.

I converted a multi-threaded application from Qt3 to Qt4. The  
application uses QThreads to talk to a database and then tells the  
main GUI thread to populate a table widget (QListView back in Qt3).  
Back in Qt3, I was lazy and locked the GUI thread to let my thread do  
its update of the GUI. Qt4 effectively removed that option.

My first attempt at a rewrite in Qt4 experienced a deadlock  
situation. I've attached a program trace (stack.txt). This version  
used the event loop on the QThreads so they could support receiving  
events and signals (done by calling exec_ in the run method of the  
QThread). The application would run fine for a while, but then I'd  
open a dialog that used threads and things would lock up. It appeared  
to be a deadlock situation between the Qt library lock (which is  
called during QCoreApplication.postEvent) and Python GIL (at least  
that is the best I can come up with given the stack trace).

After a few days of no progress, I reworked the code to avoid the  
QThread event loop and handled passing data to the worker inside the  
thread using a thread-safe queue. While that solved the deadlock  
problem (since I turned off the QThread event loop), I'm now  
experience repeated crashes. I'm in process of trying to scope a  
working test case, but it does seem very sensitive to the number of  
threads. The crashes always end up being in QApplication.notify  
somewhere.

On my Mac (10.4.8 PPC), this is a typical trace from the crash:

Thread 0 Crashed:
0   QtCore              	0x012f987c QObject::thread() const + 4  
(icplusplus.c:28)
1   QtGui               	0x0290cba8 QApplication::notify(QObject*,  
QEvent*) + 116 (qsystemtrayicon_mac.mm:346)
2   QtGui.so            	0x0233c4a8 sipQApplication::notify(QObject*,  
QEvent*) + 112 (bundle1.s:283)
3   QtGui               	0x02947430  
QApplicationPrivate::globalEventProcessor(OpaqueEventHandlerCallRef*,  
OpaqueEventRef*, void*) + 1296 (qsystemtrayicon_mac.mm:346)
4   com.apple.HIToolbox 	0x93207554 DispatchEventToHandlers 
(EventTargetRec*, OpaqueEventRef*, HandlerCallRec*) + 692
5   com.apple.HIToolbox 	0x93206cac SendEventToEventTargetInternal 
(OpaqueEventRef*, OpaqueEventTargetRef*, HandlerCallRec*) + 372
6   com.apple.HIToolbox 	0x93206b28 SendEventToEventTargetWithOptions  
+ 40

I'm using Qt 4.2.1 and PyQt 4.1.1 (with sip 4.5.2). The deadlock  
problem existed in Qt 4.1.5 and PyQt 4.0.1 as well as the current  
config.

I'm combing through my code to make sure there aren't any objects or  
data that is getting shared between threads with using a mutex. So  
while I can't be 100% certain it isn't my code, I'm quickly coming to  
the belief it isn't.

The crashing behavior (and the deadlock behavior) have both been seen  
on Linux here as well (2.6.1x kernel with Qt 4.2.1 and PyQt 4.1.1).  
Likewise for Mac Intel.

As small aside. There was a change note for Qt 4.2.2 that looked  
interesting. In the X11 sections for changes-4.2.2 there is

   * Fixed rare event loop dead-lock when posting many custom events  
to a receiver in another thread.

Alas, I saw this kind of problem on both Linux and Mac. Maybe  
Trolltech's solution wasn't comprehensive enough?

I also tried Qt 4.2.2 to no avail. I even tried various combinations  
of the Qt, PyQt, and sip snapshots. Still a crash. I settled back on  
Qt 4.2.1 and PyQt 4.1.1 while I get a scoped test case working (in  
between other deadlines!).





--kev

--
Kevin Cureton
Co-Founder
Mind The Gap, Inc.
kevin at animationpipeline.com
http://www.animationpipeline.com

On Feb 21, 2007, at 2:53 PM, Matt Newell wrote:

> On Wednesday 21 February 2007 14:35, Matt Newell wrote:
>> On Wednesday 21 February 2007 14:23, Michael Guntsche wrote:
>>
>> It locked for me after I created 70 threads.
>>
>> Linux version 2.6.18-3-686 (Debian 2.6.18-7) (waldi at debian.org) (gcc
>> version 4.1.2 20061115 (prerelease) (Debian 4.1.1-20)) #1 SMP Mon  
>> Dec 4
>> 16:41:14 UTC 2006
>>
>> qt-4.2.2
>> some sip snapshot from a few months ago
>
> 10 threads seem to run fine without lockups.
>
> If i change the write function to
>     def write(self, data):
>         #self.textEdit2.append(data)
>     	print data
>
>
> it still locks up for me once i get somewhere above 50 threads(seems
> inconsistent).  Possibly a race condition in the python signal  
> sending, or
> even in qt itself?
>
> Matt
>
> _______________________________________________
> PyKDE mailing list    PyKDE at mats.imk.fraunhofer.de
> http://mats.imk.fraunhofer.de/mailman/listinfo/pykde

-------------- next part --------------
Skipped content of type multipart/mixed