qt-interest@trolltech.com
[Top] [All Lists]

Re: [Qt-interest] Faster logfile viewer

Subject: Re: [Qt-interest] Faster logfile viewer
From: Andre Somers
Date: Wed, 05 Aug 2009 09:20:46 +0200
Hi,

Christoph Bartoschek wrote:
Arnold Krille wrote:

You are calling repaint() which executes an immediate repaint. You should
call update() which schedules the paintEvents in the event-loop.
Replacing the repaint() with update() reduces the runtime significantly
(in callgrind) and results naturally in much lower costs and only one
effective paintEvent.

In practice this change will not be a problem because normally the log-
messages are coming distributed over time and there will be enough time to
do a paintEvent every now and then.

Unless you really want a log-_file_-viewer but in that case it should be
far simplier to load the file as whole into the widget. Or at least in
large chunks, not line by line.

Hi,

unfortunately using update() does not work. I have to explain the
application a little bit more:

It's a VLSI-application that executes operations that take several hours of
runtime during which the GUI is effectively blocked. There is only the
TextEdit with the logfile where the operations write status information to.
Normally this works fine but sometimes the operations want to write more
than 100000 lines in a very short time.

The resulting requirements are:

- Because one does not know when the next line has to be shown one cannot
merge several lines together and has to show each one separately.
- Because each line can be the last one for a longer time the widget has to
be updated after each line.

Our old GUI uses Motif and also colored lines. The updates are also after
each line but writing 10000 lines takes less than one second.
Interesting problem, and worth playing around with...

If your calculations are blocking the GUI, would it not be time to separate the calculations to run in a separate thread? This way, you can have your GUI stay responsive (and use the normal update mechanisms). You can communicate your updates through signals & slots. Sending an insane amount of signals this way saturates the event loop, but with 100000 lines on my system this is tolarable, while with 10000 it is barely noticable. I have attached my version of your code.

Three main modifications:
* used QPainTextEdit instead of QTextEdit (although it does not matter *that* much, from 13,8 down to 9,5 seconds) * used a threading approach (made it 5x faster in my case, down to 2,6 seconds) * used a buffer and a timing mechanism to update the log view (almost 30x faster, down to 0,094 seconds)
Overall gain: almost 150x faster.

The display is fast, but like I said, still can get a bit jerky if the eventloop saturates with log messages. Fixing that can be done in two steps:
1) trigger a buffer flush when the buffer count reaches a set limit
2) trigger a repaint in the buffer flush

Both cost time, but especially #2 makes the display much smoother for large sets. Testing with 10,000 (not large!) items, I get 0,26 s runtime (still more than 2x than without), and with 2 enabled to this becomes 0,28; slightly more. Using a larger set (500,000 items), I get 13,8s without smoothness fixes, 21,6 with only fix 1, and 26,6 with fix 1 and 2. The latter is really smooth, and probably worth the time spend. Overall speedup relative to the original version is still 50x faster.

Play around with it using the attached files. Enable the above fixes by commenting or uncommenting them in ama.hpp.

Timing notes: they are really unscientific, based on one or two runs only per combination. All runs on a Intel Core2 Quad 9550 @ 2,83GHz.

Note that you can use the update buffering tricks without threading too. I hope this inspires you to make some further modifications to speed it up even more.

André


_______________________________________________
Qt-interest mailing list
Qt-interest@xxxxxxxxxxxxx
http://lists.trolltech.com/mailman/listinfo/qt-interest
<Prev in Thread] Current Thread [Next in Thread>