[PyQt] performance of custom models and QSortFilterProxyModel
subclasses
Carlos Scheidegger
cscheid at sci.utah.edu
Thu Dec 13 22:01:43 GMT 2007
Hi,
We have a custom model class that gets populated with many items (around a
thousand items) and we need to filter this model on some input string.
However, my experiments seem to indicate a really low performance. In
particular, there seem to be many more calls to the model's index() and
parent() than I have expected (around 400 calls per model item)
We are using a proxy model class to allow sorting and filtering, but need to
to subclass QSortFilterProxyModel to tweak the search criteria. The custom
proxy model looks like this:
class QModuleProxyModel(QtGui.QSortFilterProxyModel):
def filterAcceptsRow(self, sourceRow, sourceParent):
cp = sourceParent.child(sourceRow, 0).internalPointer()
if not cp:
return True
if cp._data[1] != 'Module':
return True
return QtGui.QSortFilterProxyModel.filterAcceptsRow(self,
sourceRow,
sourceParent)
I profiled the code to get a better idea of what's happening, and this is what
I get.
>>> pstats.Stats('p8.txt').sort_stats('cumulative').print_stats()
Thu Dec 13 14:43:44 2007 p8.txt
3810374 function calls in 27.485 CPU seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 27.485 27.485 module_palette.py:82(searchItemName)
1 8.466 8.466 27.485 27.485 {built-in method
setFilterFixedString}
428130 5.282 0.000 8.206 0.000 registry_model.py:196(index)
423257 4.760 0.000 6.878 0.000 registry_model.py:226(parent)
150438 1.597 0.000 3.024 0.000 registry_model.py:241(rowCount)
1008161 1.938 0.000 1.938 0.000 {built-in method internalPointer}
428130 1.341 0.000 1.341 0.000 {built-in method createIndex}
423257 1.301 0.000 1.301 0.000 registry_model.py:70(row)
4484 0.047 0.000 1.258 0.000 module_palette.py:380(sizeHint)
4484 0.452 0.000 1.211 0.000 {sizeHint}
578568 1.016 0.000 1.016 0.000 {len}
150438 0.592 0.000 0.857 0.000 registry_model.py:61(childCount)
23338 0.297 0.000 0.375 0.000 registry_model.py:160(data)
173776 0.338 0.000 0.338 0.000 {built-in method isValid}
934 0.014 0.000 0.098 0.000
module_palette.py:44(filterAcceptsRow)
918 0.014 0.000 0.058 0.000 {filterAcceptsRow}
934 0.008 0.000 0.024 0.000 {built-in method child}
5402 0.013 0.000 0.013 0.000 registry_model.py:49(data)
5402 0.010 0.000 0.010 0.000 {built-in method column}
320 0.001 0.000 0.001 0.000 registry_model.py:152(columnCount)
1 0.000 0.000 0.000 0.000 {method 'disable' of
'_lsprof.Profiler' objects}
There are a surprising number of calls to the model's index() and parent()
methods. Some more details:
- The model currently represents a list of lists. There will be places where
the hierarchy might be more than two levels deep, but the model is always
going broader than deep.
- In the profile, I'm performing a really simple search:
proxy_model.setFilterFixedString('d')
- The model data are all strings of size roughly 15, and the model is
implemented similarly to the custom model in the simpletreeview example. The
most important changes I made were to cache the results of the model's row()
method, so we avoid linear-time calls to index.
- the search seems to get about 10x faster after the first time. Is there any
way I can prebuild whatever the model proxy is doing on the fly?
In any case, here is the model's item code:
class RegistryTreeBaseItem(object):
def __init__(self):
self._parent_item = None
self._children = []
self._my_row = -1
self._parent_children_id = 0
def appendChild(self, item):
# Do not change this to "self._children += [item]":
# we need a new list every time so that id(self._children) changes
# appendChild happens infrequently, so it's ok.
self._children = self._children + [item]
item._my_row = len(self._children)
item._parent_children_id = id(self._children)
def data(self, column):
return self._data[column]
def findChild(self, item):
return self._children.index(item)
def removeChildIndex(self, index):
del self._children[index][:] # we create a shallow copy to change id
def child(self, row):
return self._children[row]
def childCount(self):
return len(self._children)
def columnCount(self):
return 1
def parent(self):
return self._parent_item
def row(self):
if (self._my_row == -1 or
self._parent_children_id != id(self._parent_item._children)):
# ids don't match, so parent changed, so we'll update _my_row
# Updates _my_row,
self._my_row = self._parent_item._children.index(self)
self._parent_children_id = id(self._parent_item._children)
return self._my_row
and the model's index and parent methods look like:
# This has a bunch of inlining, but it's similar to simpletreeview.py
def index(self, row, column, parent):
parent_item = parent.internalPointer()
try:
row_count = len(parent_item._children)
except AttributeError:
parent_item = self._root_item
row_count = len(parent_item._children)
child_item = parent_item._children[row]
return self.createIndex(row, column, child_item)
def parent(self, index):
try:
child_item = index.internalPointer()
parent_item = child_item._parent_item
return self.createIndex(parent_item.row(), 0, parent_item)
except AttributeError:
return self._blank_index
def rowCount(self, parent):
if not parent.isValid():
parent_item = self._root_item
else:
parent_item = parent.internalPointer()
return parent_item.childCount()
The subclasses of this simply populate the data in the right way. I can try
and adapt the simpeltreemodel.py example to produce this kind of behavior, but
I thought I'd ask if any of you have had to deal with this sort of issue
before, and if you have any suggestions.
Thank you very much in advance,
-carlos
More information about the PyQt
mailing list