OpenOffice.org comes in several editions produced by different groups. Each edition has its own features, performance improvements, bug fixes, and new bugs. Go-oo in particular boasts performance as a feature with its the slogan, "Better, Faster, Freer," but is there truth in advertising? Let's pit four OpenOffice.org editions against each other in a scientific speed smackdown.
The four editions are Sun Microsystem's OpenOffice.org (which I call "Vanilla"), Fedora's, Go-oo, and OxygenOffice Professional. If OpenOffice.org had a family tree, it would be a convoluted redneck family. The last three are derived directly from Sun Microsystem's code, and OxygenOffice is derived from (or part of) Go-oo. Some (but not all) patches in the derivatives go back upstream to Vanilla.
Benchmark test environment
The test system is a modest computer from several years ago.
- Operating system: Fedora 9 i386 with Linux 2.6.26
- CPU: AMD Athlon XP 3000+ (32-bit)
- RAM: 768 MB, DDR 333 (PC 2700)
- HDD: Maxtor 6Y080L0, IDE, 7200 RPM, 80 GB
- Video: Via VT8378 S3 Unichrome IGP at 1024x768
Benchmark test procedure
Like previous benchmarks, this OpenOffice.org benchmark uses automation to precisely measure the duration of a series of common operations: starting the application, opening a document, scrolling through from top to bottom, saving the document, and finally closing both the document and application. Automation is much more precise than a human with a stopwatch, and with the small durations in these tests, automation is necessary. Each of the five tests are repeated for 10 iterations. Before each set of 10 iterations, the system reboots. The purpose of rebooting is to measure the difference in cold start performance where information is not yet cached into fast memory. A reboot marks a pass, and there are 15 passes. That means for each edition of OpenOffice.org, there are 150 iterations. Multiplying by 5 tests and by 4 versions of OpenOffice.org yields 3000 total measurements collected for this article.
Only OxygenOffice enabled a quick starter by default, and as promised in the Microsoft Word benchmark, I disabled the OxygenOffice quick starter to give all editions a level playing field.
The single most important test is cold start performance—how fast the application starts the first time after the computer reboots. Go-oo's home page brags about its low I/O, fast linking, and minimized registry access contribute to better performance. Is it enough for Go-oo?
What a close race! The cold startup performance results range from 4.83 seconds to 5.24 seconds, but for all the smack talk, Go-oo came out fourth out of four. Why? There could be lots of reason. First, Fedora can link to more system libraries, meaning it can share more code with other running programs, so Fedora doesn't need to include its own XML parser, pieces of Mozilla, etc. Second, Fedora is compiled using the latest GCC version 4.3, while OxygenOffice was built with GCC 4.1; Go-Oo, 3.4.6; and Vanilla, apparently 3.4.1 and 2.9.5. Third, Go-oo has more features and is 49% bigger than Vanilla. On the other hand, OxygenOffice is also large. Finally, Go-oo and Vanilla do not use the Hashstyle (also called DT_GNU_CASH or .gnu.hash) technology developed by
Go-oo hacker (employed by Novell) Michael Meeks CORRECTED (twice) Red Hat based on research by Michael to accelerate linking (which happens early in a program's startup): only Fedora and OxygenOffice use Hashstyle.
It may be hard to tell because the benchmark results are so consistent, but most charts are illustrated as box plots. Lower is always better.
What about application startup after a warm start? (A warm start is starting the application a second time after a reboot when the disk cache speeds up loading.) Here OxygenOffice comes in a close second after Fedora while Go-oo and Vanilla practically tie for more distant third place. The difference between all cold and warm starts is about 4 seconds, which is consumed by slow hard disk drive technology, while new solid-state storage devices would see faster performance.
The second most important metric is opening a document, so I tested reference document ODF_text_reference_v3.odt. In colds starts the variation was significant: last-place Go-oo finished 27% slower than first-place Fedora.
The results for warm starts are similar but 5-6 seconds faster.
The scrolling test involves moving down each line from top to bottom, and three editions scored similarly except OxygenOffice which did shockingly bad. The scrolling benchmark is the most artificial and least important.
In exporting a document, Vanilla comes in both first (after warm start) and last (after a cold start). Here Go-oo picks up its only meaningful win, and it wins the race to export the document after a cold start. This chart does not start at zero because it would otherwise be too small to read.
It does take a moment to close OpenOffice.org.
In the end, Fedora OpenOffice.org scored the fastest performance by a significant margin, and OxygenOffice took home a strong second place. Go-oo barely beat Vanilla for third.
Scrolling and closing were excluded from the totals because people generally don't wait for them like they do for starting the application, opening a document, and saving the document.