Our research group tested and scrapped CUDA although it was close to perfect for our workload. No doubt could we get massive performance increase, but it was too much work (many lines of code), too unlikely that nvidia would be the performance leader over time, and compatibility issues were seen as very likely. For instance the next big thing (tm) from nvidia might well be compatible with CUDA, but would we have to do the performance tuning all over again? With these highly specialized architectures getting the software to do what you want it to do takes like half the time, and the other half is spent on architecture specific tuning to get the performance potential of the hardware. New architecture most likely means a new round with tuning.

It is actually worse than the VLIW dilemma because new uarch don't mean that your binary is broken and all you need to do is recompile the code. In stead your binary probably runs correctly, but dead slow. Recompiling might do the trick in some cases. In other cases you must rewrite the code because too many details are known to the programmer, which of course is at least half the reason you can get so much performance in the first place.

IMO GPUs are not general purpose enough yet to make the effort worth vile for us. The interesting question is if they will have enough performance advantage over CPUs by the time they get there.

We decided to port from matlab to plain boring C++ rather than CUDA. This way we are guaranteed that the code will have some real value in the future and hopefully the througput/watt focus of CPUs will ensure enough performance and it doesn't matter if the CPU is SPARC, Power, IA64 or x64. Luckily the workload is trivial to scale so it boils down to an economic issue really.

one of my students is working with CUDA
we are doing security stuff with GPUs and we are seeing very nice speedups

Tvar/Malcom,
The reason I say what I say is nicely summed up by Ajensen at the top of the page. I am not doubting the claims of NV directly, you CAN get the required speedups, sometimes by orders of magnitude.
That said, corps are not buying in, if you run the numbers, it isn't worth it. Specialized cases, yes. Armies of free grad students, yes. Real world where you have to pay programmers to write and maintain code? No.
On paper, it was a good idea. Once you take all the external factors into account, the price/performance advantage drops to near zero.
I have talked to a LOT of big companies who were the exact target market for GPGPU. None of them are deploying, they all do what Ajensen did, kick the tires, evaluate it, run the numbers and walk away.
In the end, it just isn't worth it, big picture, for most people.

-Charlie

As long as you don t see SPEC speed up, it is not really ready ...

who?

What Groo said. I'm in one of the key markets targeted by CUDA and GPGPU, and after the initial excitement it has mostly been a disappointment. There has been a lot of tire kicking but deployment is extremely limited, and I don't see that changing any time soon. It's just not worth it at the moment, for all the reasons given above.

GPGPU only make sense in the context of uniform instruction set, other solutions are mole on the architecture. It does not make any sense to make the instruction set splitted for the CPU and the GPGPU, it needs at least a common ground.

who?

Groo -
I now get exactly what the point is you are trying to make, and I find it quite easy to believe. Current hardware is not yet flexible enough, and current APIs are fragmented and difficult to develop for. ie. Value proposition for a business is low.

However, I still believe we are on the cusp of a paradigm shift. The hardware is getting faster and more flexible (can't wait for Larabee!), and I expect that eventually the API landscape will mature.

I also expect that the value proposition will be entirely different in a year or so when (if?) Fusion appears and Larabee matures. It also makes me wonder what nVidia is going to do, although it might also explain why they've been so ornery lately! :)

On a personal note, I believe the most fascinating thing about stream processing tech is the enabling aspect. The order of magnitude (+) speedup can make some applications possible that wouldn't otherwise be.
For example:
http://fastra.ua.ac.be/en/index.html

Malcolm