<div id="__MailbirdStyleContent" style="font-size: 10pt;font-family: Arial;color: #000000">
                                        
                                        
                                            
                                        
                                        
                                        processEvents() will process everything other than DeferredDelete events, which is the type of event that gets created by deleteLater() calls.  That's intended (but confusing) behavior.  See https://doc.qt.io/qt-5/qcoreapplication.html#processEvents.  In your example script, one potential fix is to not give MyDialog a parent, so change "dlg = MyDialog(self)" to "dlg = MyDialog()".  That should let PyQt destroy the dialog as soon as the Python reference count drops to zero, which would happen with "del dlg", "dlg=None", or on the next loop iteration.<div><br></div><div>If you need to parent the dialogs, another option is to use "sip.delete(dlg)" instead of "dlg.deleteLater()", which will trigger immediate deletion of the dialog instead of scheduling it for later.</div><div><br></div><div>- Kevin<br><div class="mb_sig"></div>
                                        
                                        <blockquote class="history_container" type="cite" style="border-left-style: solid;border-width: 1px;margin-top: 20px;margin-left: 0px;padding-left: 10px;min-width: 500px">
                        <p style="color: #AAAAAA; margin-top: 10px;">On 10/4/2019 9:09:48 AM, J Barchan <jnbarchan@gmail.com> wrote:</p><div style="font-family:Arial,Helvetica,sans-serif"><div dir="ltr"><div class="gmail_default" style="font-family:tahoma,sans-serif">Hello Experts,</div><div class="gmail_default" style="font-family:tahoma,sans-serif"><br></div><div class="gmail_default" style="font-family:tahoma,sans-serif">I do not know whether this issue is PyQt-related, but it might be, as I think it's to do with <span style="font-family:monospace">deleteLater()</span> and <i>possibly</i> Python memory management.  I should be so grateful if someone could take the time to look through and advise me what to try next?</div><div class="gmail_default" style="font-family:tahoma,sans-serif"><br></div><div class="gmail_default" style="font-family:tahoma,sans-serif"><div class="gmail-content">
        <p>Qt 5.12.2.  Python/PyQt (though that does not help, it should not be
 the issue).  Tested under Linux, known from user to happen under 
Windows too.  I am in trouble, and I need help from someone who knows 
their QtWebEngine!</p>
<p>Briefly: I have to create and delete QtWebEngines (for non-interactive use, read below) a <em>large</em> number of times from a loop which cannot call the <em>main</em> event loop.  Every instance holds onto its memory allocation --- which is "large" --- until code finally returns to <em>main</em> event loop.  I <em>cannot</em> find any way of getting Qt to release the memory being use by QtWebEngine as it proceeds, <strong>only</strong>
 when return to main event loop.  Result is whole machine runs out of 
memory + swap space, until it dies/freezes machine, requiring reboot!</p>
<ul><li>
<p>In large body of code, <code>QWebEngineView</code> is employed in a <code>QDialog</code>.</p>
</li><li>
<p>Sometimes that dialog is used interactively by user.</p>
</li><li>
<p>But it is also used <em>non</em>-interactively in order to use its ability to print from HTML to PDF file.</p>
</li><li>
<p>Code will do a non-interactive "batch run" of hundreds/thousands of pieces of HTML, exporting to PDF file(s).</p>
</li><li>
<p>During this <em>large</em> amounts of memory will be gobbled by 
QtWebEngine.  Of the order of hundreds of create/delete taking Gigabytes
 of machine memory.  So much so that machine can even run out of all 
memory and die!</p>
</li><li>
<p><em>Only</em> a return to top-level, main Qt event loop allows that memory to be recouped.  I <em>need</em> something better than that!</p>
</li></ul>
<p>I paste below about as minimal an example of code I am using in a test to prove behaviour.</p>
<pre class="gmail-markdown-highlight"><code class="gmail-hljs gmail-ruby">import sys

from PyQt5 import QtCore, QtWidgets
from PyQt5.QtWebEngineWidgets import QWebEngineView


<span class="gmail-hljs-class"><span class="gmail-hljs-keyword">class</span> <span class="gmail-hljs-title">MyDialog</span>(<span class="gmail-hljs-title">QtWidgets</span>.<span class="gmail-hljs-title">QDialog</span>):</span>
    <span class="gmail-hljs-function"><span class="gmail-hljs-keyword">def</span> <span class="gmail-hljs-title">__init__</span><span class="gmail-hljs-params">(<span class="gmail-hljs-keyword">self</span>, parent=None)</span></span>:
        <span class="gmail-hljs-keyword">super</span>(MyDialog, <span class="gmail-hljs-keyword">self</span>).__init_<span class="gmail-hljs-number">_</span>(parent)

        <span class="gmail-hljs-keyword">self</span>.wev = QWebEngineView(<span class="gmail-hljs-keyword">self</span>)

        <span class="gmail-hljs-keyword">self</span>.renderLoop = QtCore.QEventLoop(<span class="gmail-hljs-keyword">self</span>)
        <span class="gmail-hljs-keyword">self</span>.rendered = False
        <span class="gmail-hljs-keyword">self</span>.wev.loadFinished.connect(<span class="gmail-hljs-keyword">self</span>.synchronousWebViewLoaded)

        <span class="gmail-hljs-comment"># set some HTML</span>
        html = <span class="gmail-hljs-string">"<html><body>Hello World</body></html>"</span>
        <span class="gmail-hljs-keyword">self</span>.wev.setHtml(html)

        <span class="gmail-hljs-comment"># wait for the HTML to finish rendering asynchronously</span>
        <span class="gmail-hljs-comment"># see synchronousWebViewLoaded() below</span>
        <span class="gmail-hljs-keyword">if</span> <span class="gmail-hljs-keyword">not</span> <span class="gmail-hljs-keyword">self</span>.<span class="gmail-hljs-symbol">rendered:</span>
            <span class="gmail-hljs-keyword">self</span>.renderLoop.exec()

        <span class="gmail-hljs-comment"># here I would do the printing in real code</span>
        <span class="gmail-hljs-comment"># but it's not necessary to include this to show the memory problem</span>

    <span class="gmail-hljs-function"><span class="gmail-hljs-keyword">def</span> <span class="gmail-hljs-title">synchronousWebViewLoaded</span><span class="gmail-hljs-params">(<span class="gmail-hljs-keyword">self</span>, <span class="gmail-hljs-symbol">ok:</span> bool)</span></span>:
        <span class="gmail-hljs-keyword">self</span>.rendered = True
        <span class="gmail-hljs-comment"># cause the self.renderLoop.exec() above to exit now</span>
        <span class="gmail-hljs-keyword">self</span>.renderLoop.quit()


<span class="gmail-hljs-class"><span class="gmail-hljs-keyword">class</span> <span class="gmail-hljs-title">MyMainWindow</span>(<span class="gmail-hljs-title">QtWidgets</span>.<span class="gmail-hljs-title">QMainWindow</span>):</span>
    <span class="gmail-hljs-function"><span class="gmail-hljs-keyword">def</span> <span class="gmail-hljs-title">__init__</span><span class="gmail-hljs-params">(<span class="gmail-hljs-keyword">self</span>, parent=None)</span></span>:
        <span class="gmail-hljs-keyword">super</span>(MyMainWindow, <span class="gmail-hljs-keyword">self</span>).__init_<span class="gmail-hljs-number">_</span>(parent)

        <span class="gmail-hljs-keyword">self</span>.btn = QtWidgets.QPushButton(<span class="gmail-hljs-string">"Do Test"</span>, <span class="gmail-hljs-keyword">self</span>)
        <span class="gmail-hljs-keyword">self</span>.btn.clicked.connect(<span class="gmail-hljs-keyword">self</span>.doTest)

    <span class="gmail-hljs-function"><span class="gmail-hljs-keyword">def</span> <span class="gmail-hljs-title">doTest</span><span class="gmail-hljs-params">(<span class="gmail-hljs-keyword">self</span>)</span></span>:
        print(<span class="gmail-hljs-string">"Started\n"</span>)
        <span class="gmail-hljs-comment"># create & delete 500 non-interactive dialog instances</span>
        <span class="gmail-hljs-keyword">for</span> i <span class="gmail-hljs-keyword">in</span> range(<span class="gmail-hljs-number">500</span>):
            <span class="gmail-hljs-comment"># create the dialog, it loads the HTML and waits till it has finished rendering</span>
            dlg = MyDialog(<span class="gmail-hljs-keyword">self</span>)
            <span class="gmail-hljs-comment"># try desperately to get to delete the dialog/webengine to reclaim memory...</span>
            dlg.deleteLater()
            <span class="gmail-hljs-comment"># next lines do not help from Python</span>
            <span class="gmail-hljs-comment"># del dlg</span>
            <span class="gmail-hljs-comment"># dlg = None</span>
        QtWidgets.QMessageBox.information(<span class="gmail-hljs-keyword">self</span>, <span class="gmail-hljs-string">"Dismiss to return to main event loop"</span>, <span class="gmail-hljs-string">"At this point memory is still in use :("</span>)


<span class="gmail-hljs-keyword">if</span> __name_<span class="gmail-hljs-number">_</span> == <span class="gmail-hljs-string">'__main__'</span>:
    <span class="gmail-hljs-comment"># -*- coding: utf-8 -*-</span>

    app = QtWidgets.QApplication(sys.argv)

    mainWin = MyMainWindow()
    mainWin.show()

    sys.exit(app.exec())
</code></pre>
<p>I have tried various flavours of <code>processEvents()</code> in the loop after <code>deleteLater()</code> but none releases the memory in use.  <em>Only, only</em> when the code returns to the top-level, main Qt event loop does it get released.  Which is too late.</p>
<p>To monitor what is going on, under Linux I used</p>
<pre class="gmail-markdown-highlight"><code class="gmail-hljs gmail-nginx"><span class="gmail-hljs-attribute">watch</span> -n <span class="gmail-hljs-number">1</span> free mem
watch -n <span class="gmail-hljs-number">1</span> ps -C QtWebEngineProc
top -o %MEM
</code></pre>
<p>There are two areas of memory hogging:</p>
<ul><li>The process itself uses up considerable memory per QtWebEngine</li><li>It will run 26 (yes, precisely 26) <code>QtWebEngineProc</code> processes to service the requests, each also taking memory.</li></ul>
<p>Both of these disappear as & when return to Qt top-level event 
loop, so we know the memory can & should be released.  I do not know
 if this behaviour is QtWebEngine specific.</p>
<p>Anyone kind enough to answer will need to be specific about what to 
put where to resolve or try out, as I say I have tried a lot of 
fiddling!  Unfortunately, advising to do the whole thing "a different 
way" (e.g. "do not use QtWebEngineView", "rewrite code so it does not 
have to do hundreds at a time", etc.) is really not what I am looking 
for, I need to understand why I can't get it to release its memory as it
 is now?  Can anyone make my <code>deleteLater()</code> release its memory without going back to the top-level Qt event loop??</p>

</div></div><br>-- <br><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><span style="font-family:tahoma,sans-serif">Kindest,</span></div><div><span style="font-family:tahoma,sans-serif">Jonathan</span></div></div></div></div></div></div>
_______________________________________________
PyQt mailing list    PyQt@riverbankcomputing.com
https://www.riverbankcomputing.com/mailman/listinfo/pyqt
</div></blockquote></div></div>