Comments on OpenOffice.org Ninja: Benchmarking Microsoft Word 95 through Word 2007

Jezza: I think you are making my point without rea...

2008-09-09T08:47:00.000-06:00

Jezza: I think you are making my point without realizing it? :) Basically, my point is generally all software becomes slower over time.

Nice comparison however, compared to your OOO comp...

2008-09-05T19:27:00.000-06:00

Nice comparison however, compared to your OOO comparison, MS Office 2003 is still faster than OOO and 200y is about the same speed as 2.4.

Each office suite is designed with the hardware of the time and running MS 2007 on a three year old machine will of course get you slow results and running MS 2003 on the same machine will get you faster results.

I know you are comparing the speed of each suite on the same platform however, it would be stupid of me to even try and run 2007 on my 5 year old PC (512MB RAM, 80GB IDE) when it runs much faster (obviously) on my newer PC (Dual core 1.8, 320GB SATA, 2GB RAM).

Personally, I'd rather use OOO then Office 2003 but on my older machine, 2003 runs much faster than OOO.

Anonymous: Good question. It took many months of ...

2008-08-13T18:53:00.000-06:00

Anonymous: Good question. It took many months of work for me to create the tool. It is a heavily modified and specialized variation of the available tool called Jungletest QA (which contains no benchmarking features). I hope to publish the Jungletest benchmarking tools for OpenOffice.org and Microsoft Word this year.

This is a nice comparison of Office performance. ...

2008-08-12T10:32:00.000-06:00

This is a nice comparison of Office performance. Can you provide details and the names of the tools you used to script the office benchmarking? I would like to run a similar benchmark against my own hardware and OS image.

Word 2007 is a bunch slower, but still quite a bit...

2008-07-27T17:59:00.000-06:00

Word 2007 is a bunch slower, but still quite a bit faster than OpenOffice 2.4. Then if we consider how fast Word 2003 is compared to both ... OO is even slower.

I'll admit to be slamming OO here, I read the "Is OpenOffice getting slower?" article at this site and the excuses. MS Office has improved dramatically over the years .. hopefully the next version of office will fix these performance regressions (outlook sure took a hit).

Hmmm, what the benchmarks show--besides the slowdo...

2008-07-24T17:21:00.000-06:00

Hmmm, what the benchmarks show--besides the slowdown in 2007--is that Microsoft Word 95-2003 really was simply a series of refinements of the same code base, i.e. minor upgrades, where Microsoft did not add many new features, instead focusing on bug-fixing, performance-tweaking, and fine-tuning.

Word 2007, in contrast, has been a rather marked break with the past--more so than any of the preceding versions listed were with their predecessors. It's as new as Word 6/95 was, and for this reason, slower. But if the tests here are any guide, we may expect future, incrementally improved versions of Word to be faster, at least until the next big release in, say, five to ten years.

Thanks for the nice work. I'm personally OK to spe...

2008-07-24T16:13:00.000-06:00

Thanks for the nice work. I'm personally OK to spend 2 times more on disk activity, if Office 2007 can make my regular work (editing) faster (which it does).

Anyways, as someone else mentioned, a 2007 .doc format comparison would be nice.

This tells nothing. Honestly i even think it's stu...

2008-07-24T09:29:00.000-06:00

This tells nothing. Honestly i even think it's stupid to compare everything this way.

Anonymous: Sorry, I don't plan to look at Outlook ...

2008-07-23T23:03:00.000-06:00

Anonymous: Sorry, I don't plan to look at Outlook or Vista. By the way, I chose XP SP3 because that is what I had. Also, most computers today run XP, and there have been some controversies about Vista performance (though allegedly fixed in Vista SP1). Now, if someone wanted to donate a Mac, I could do a three way comparison of OpenOffice.org on three operating systems on the same hardware. :)

John Mann: Interesting question, but I may not be able to answer it as this blog focuses on OpenOffice.org.

Andy Babiec: I agree that Wirth's Law is not an absolute law, but there is definitely a trend. You can see the same exceptions in OpenOffice.org which significantly improved performance such as from 1.1.5 to 2.0.0. In "Is OpenOffice.org Getting Faster?" I suggested that these exceptions often indicate poor optimization in the older version.

I think the interesting aspect is that performance...

2008-07-23T17:40:00.000-06:00

I think the interesting aspect is that performance improved (even if slightly) between 97 and 2003 - counter to the myth that every office version is slower then the previous. In fact, if you took out scrolling, there would be a bigger jump.

How about a comparison of Word 2007 + .doc file v...

2008-07-23T17:07:00.000-06:00

How about a comparison of Word 2007 + .doc file versus Word 2007 + .docx file?

Is the Word 2003 to Word 2007 slowdown more to do with the application or the file format?

So the bottom line is:Spend $millions per thousand...

2008-07-23T15:22:00.000-06:00

So the bottom line is:

Spend $millions per thousand users to upgrade their hardware/software, back up their data, and get them up to Vista/Office 2007, and my overall labor costs will DOUBLE!!!

Maybe that's why Microsoft had to hire a whole bunch more people and their costs relative to their income is getting out of control.

Such a bargain.

Maybe Open Office is "Good enough".

If Microsoft won't let me buy XP on my next machine, maybe I'll just install Linux, have VMWare Converter create a back-up of my old system, and I'll run that on the VMWare/Linux using VMWare Player or VMWare workstation.

Then I can use Linux/OpenOffice most of the time, and Windows only when I absolutely need it.

If I buy the machine with Vista, I'm legally allowed to downgrade.

But if Microsoft doesn't like that idea, maybe I should just scuttle Windows completely.

I'll bet there are quite a few CIOs thinking this way right now.

"Does anyone remember Windows 3.1, OS/2 Warp, and ...

2008-07-23T15:15:00.000-06:00

"Does anyone remember Windows 3.1, OS/2 Warp, and DESQview?"

I remember all three.

One thing I remember in particular was having a 386 computer (I don't remember how much RAM).

I was given an additional 386 computer to get more work done. Both had Windows 3.1 on it. The new one was slightly faster with slightly more RAM than the "old" one which was about six months old at the most.

I had a spare hard drive and took out the HDD from the "old" computer and put it in and installed a beta for OS/2 2.0 and installed the Apps that I had on my Windows 3.1 box (WordPerfect, Lotus 1-2-3) and was stunned when the DOS and Windows 3.1 programs ran significantly faster on the the OS/2 box than on the new Windows 3.1 box.

One spreadsheet I had took over a minute in Windows 3.1 and took about 12 seconds in a Windows 3.1 box in OS/2.

I found that I could four or fives times the amount of things on OS/2 than I could with W3.1 alone. I installed it on the new box and stuck with OS/2 until it was just impossible due to W95 incompatibilities. Now OS ones but app incompatibilities where I couldn't open up documents anymore.

I didn't get rid of OS/2 at that point. One computer ran OS/2 for years after that and I was much happier using that than '95. It was actually when W2000 came out, which still wasn't as good as OS/2 but things had just come to a head and it just wasn't worth it anymore.

Like sucked until Mac OS X 10.2. Now I use that as much as possible (more so each day) along side my (at work only) XP box which is used less each day. I'm not going to bother getting Vista.

No I don't work for MS or Apple or IBM or any company named or remotely associated with named companies or products.

Nice work! By any chance have you done something...

2008-07-23T14:28:00.000-06:00

Nice work! By any chance have you done something like this for outlook or Vista vs XP?

anonymous, nice advertisement. That wasn't a parti...

2008-07-23T09:08:00.000-06:00

anonymous, nice advertisement. That wasn't a particularly useful comment.

> reactions to the "Is OpenOffice.org> Getting Fas...

2008-07-23T09:06:00.000-06:00

> reactions to the "Is OpenOffice.org
> Getting Faster?" benchmark article. I
> read through dozens of comments of
> people expecting new releases of the
> same software product to be faster. I
> hoped this Word benchmark would help
> those people.

So..because Word gets slower, its okay for OOO to get slower, too? Great reasoning. Wow.

Please take a look at Softmaker Office.
OO may have some more features than this suite. But if you, once in your lifetime, go away from Windows and KDE and GNOME and the fat setups of the large Linux distributions, you'll notice that computers actually got *faster* over the last years.

OO is fucking slow. I only ever use it if I have to.

One idea to look at (especially between 2003 and 2...

2008-07-22T21:32:00.000-06:00

One idea to look at (especially between 2003 and 2007) would be the change to the docx extension. We saw a huge increase in file load time between 2003 and 2007. As I understand it, the 2007 files both compress and use XML to read/write files, which is much different than back in 2003.

I was going to mention, this had to have taken a l...

2008-07-22T12:44:00.000-06:00

I was going to mention, this had to have taken a long, long time. Just setting up all the software would take a couple of weeks. That must be why you haven't posted in a while?

Anyway, kozuch82, if you look back at the article looking at the same stats for OpenOffice, andrew didn't use different version of Linux (or Windows) for that article. He used the same versions. For comparability, it makes more sense to use a single OS version. Additionally, considering the purpose of the blog post is not to compare the speed on different platforms but the speed relative to each other, limiting the variables makes the most sense.

Your comment raises an interesting point: Is there optimization of versions of Word for different iterations of the operating system? Probably (andrew mentions some). But that's not what this article is about.

The only other thing I think would be helpful would be to quantify the changes in iterations of Word in terms of percentages and then overlay those on a graph with OpenOffice to see the two packages simultaneously. It would be a little tricky to do, but would put a nice finishing touch on this issue.

Kozuch82: There is one reason to do that, but on t...

2008-07-22T12:32:00.000-06:00

Kozuch82: There is one reason to do that, but on the other hand that introduces a new variable. My point was to isolate Microsoft Word without skewing the results for differences in performance of Windows.

Anonymous: No, not obvious to all! My main reason for starting this article (which took weeks of hard work, by the way) was the reactions to the "Is OpenOffice.org Getting Faster?" benchmark article. I read through dozens of comments of people expecting new releases of the same software product to be faster. I hoped this Word benchmark would help those people.

You should be commended for all the time and effor...

2008-07-22T11:38:00.000-06:00

You should be commended for all the time and effort that went into that, but isn't the bottom line conclusion, literally, just stating the obvious?

"Upgrading to the latest Microsoft Office or OpenOffice.org on the same hardware year after year, someday something somewhere will have to give: you'll need to stop upgrading your software or start upgrading your hardware."

Heck, if I could use the same software (or OS) forever, some dead horses in the basement would still be in the race. I still do run Paradox 3.5 for DOS on current hardware, and it flies!

The measurings just do not make sense. You should ...

2008-07-22T10:57:00.000-06:00

The measurings just do not make sense. You should have installed Win95 for Word95, Windows XP for XP and Vista for 2007 etc. on a average hardware used those times. This numbers have no statistical value.

Nice benchmark. Thanks for putting this together....

2008-07-22T08:28:00.000-06:00

Nice benchmark. Thanks for putting this together.