The "Random Access Enumerator Hack" Results
Seth Spitzer
Seth Spitzer
Summary:
At recent mailnews performance meeting, it was suggested that we should investigate how much performance would improve if we had random access enumerators. To prototype, I hacked nsMsgDBEnumerator to stop returning items at certain point. This threshhold was controlled by an environment variable. This hack made it so we loaded the whole msg db, but had the ability to only get a subset (although incorrect) of the headers.
Next, Suresh (suresh@netscape.com) did some measurements on the target machine with the environment variable set at different values.
Results:
By pretending we have random access enumerators, loading folders used less memory and was faster. An example, loading a 1000 message IMAP folder took 4.22 less seconds (from 12.03 down to 7.81) and took 2.1 MB less memory (from 36.1 MB to 34 MB). This was with the enumerator hacked to return 20 items.
Click here to see all the timings.
Comments:
1) This hack does not indicate the actually overhead required to implement random access enumerators, just the upper limit on the performance to be gained.
2) We've not yet determined where we're saving the memory (in the mail code, rdf, the content model, etc) or where the speed up is coming from. (waterson, hyatt, putterman, mscott and alecf have some educated guesses. They are...).
3) Using random access enumerators will slow down thread pane scrolling. This is because...
To see how it see how the hack was implemented, look at nsMsgDatabase.cpp
Look for ENABLE_TESTING_HACK.
Note, this hack will be removed soon.