(23 March 2011)
Purpose
I've always wondered if there was any practical real-world application where hyperthreading would make a measurable difference.
I have access to a Dell R410 dual-socket, quad-core system (8 cores total) with HT available.
In this specific case, I'm using the Distributed.Net dnetc client as a reference load.
Data
Blocks of work done with hyperthreading:

Blocks of work done without hyperthreading:
Observation
Both of these data runs settle down to a (remarkably) consistent number. The consistancy of the non-hyperthreading run suggests to me that I really need to let the hyperthreading run longer than just the data collected here. (Which might be difficult as this computer is now required for real work.)
But the suggestion from this is that even with this almost zero wait-on-io load you get more work done with hyperthreading enabled, in this case 14% more total work done.
Discussion
Historically, my general tendency is to advise customers to turn hyperthreading off for most deployments. Linux doesn't seem to have a mechanism that can tell the difference between a "processor" (ie hyperthread) that is actually running on the same core as another "processor" is. This has lead to us observing that if you have, say, more than one full-throttle process, you will see that with the processor bouncing around that Linux does you end up occasionally with both active "threads" running on the same core. Which means most likely you'll get less total work done because occasionally one core is busy doing more than one job, even as other cores stand totally idle.
That said, if you will routinely have more active processes than you have cores, hyperthreading will likely yield an increase of total work done (as in our example above) even as individual tasks end up taking longer.
Most of my customers don't end up in this state due to license or memory overhead restrictions, so we end up turning hyperthreading off.