I'm sure everyone here has heard of Nvidia's CUDA, which will be present in their next generation of graphics card due out in Summer.

Nvidia is making alot of noise about it too. Saying it will be revolutionary, and will make the GPU a much more important part of the system.

Two areas of note in particular interest me.

Physics and Encoding.

CUDA enabled GPUs promise to speed up these two tasks greatly, much more than what can be accomplished on a CPU..

Now my question is, didn't both ATI and Nvidia say that this was going to happen years ago, but it never did?

Didn't ATI have an encoding program for their GPUs, which could "supposedly" run an astonishing 30 times faster than dual core processor at the time?

What the hell happened with that one I wonder?

Also, Nvidia had Havok FX or something, but I guess it was canned?

Why?

But the overriding question is, what will make this time any different than their previous experiment?

Will CUDA truly be revolutionary you think? Or is that just Nvidia overindulging in marketing propaganda?

And, what does this mean for the CPU?

The problem is in the programming.

You have to, to some extent, understand the chip architecture. That's asking a lot for many programmers. Then you have to understand how to exploit that stuff to get the speedup. Then you have to take a current problem you have and remap it in a certain way such that you can actually use a GPU to solve it.

So really, a lot of learning and different methodologies is what's keeping this stuff from bursting the dam with amazing applications.

Remember that ATi and NVidia are promising this stuff because it makes their products look better. Sure, the cards have these capabilities, and the usefulness and applications are limited, and the effort to learn this stuff is too much.

I used to be psyched about it too, but people I've talked to who have gone to presentations have brought up these points, and the reps always point to the "100x speedup" on their powerpoint slides.

CUDA has been around for a while now. I'm not sure how it's doing overall. I went on a training course and did attempt to use it, but it was tough going. It is Nvidia-only, which is limiting unless that is all you care about. It also exposes a lot of internals, eg all the levels of memory hierarchy. Great if you want ultimate control, but a lot to deal with if you don't.

I think the future for most users will lie with higher level wrappers that sit on top of APIs like CUDA.

I am aware of several projects using CUDA at the engineering department in Cambridge; it's the sort of language in which you can get impressive demonstration results given infinite motivated grad-student labour.

I've heard rumours of effective GPU usage in quantitative-finance back rooms, but that's a higher-paid and harder-worked version of the grad student problem. I've seen very few bits of GPGPU usage in software that actually ships, and is intended to be used by people who sit more than three cubicles away from the local GPGPU guru.

I suspect 'wrappers that sit on top of CUDA' will be nightmares to develop or to program with; GPU problems tend to be matched very precisely to the size of the hardware, and to worry a lot about exact data placement. I think it would be very challenging to develop wrappers whose performance profile is comprehensible - I foresee a lot of aha! moments of the form "I modified an entry in the array here, so the wrapper has required that matrix to be in system memory then, so it had to copy it to and from the GPU once per optimisation cycle, and that's why it slowed down by a factor fifty"

I think the problems you describe with wrappers are a little overblown. Of course if you want ultimate performance and control and have vast resources to throw at the problem and only care about Nvidia hardware you will use CUDA. But if you want something cross-platform, more C++-like, and easier to use, a wrapper would do just fine. I'm aware of at least one significant project heading in this direction that is signficantly easier to use than CUDA. I'm not even sure the performance will always be worse, as the higher level allows more widespread optimizations to be implemented that may end up outperforming a handcrafted CUDA implementation in some cases.

Interesting..

Isn't true interactive physics possible on the CPU only though?

I remember reading an article that stated that GPUs cannot "compute" the long term effects of in game physics, due to the matrix heavy computations involved.

http://tw.youtube.com/watch?v=Mvxe04wGmTw

who?
PS: this is my personal opinion ;-)

Nvidia will be claiming an 18x increase in speed for their app called Elemental Badaboom!

http://www.theinquirer.net/gb/inquirer/news/2008/06/12/nvidia-changes-gt200-dates

This is for encoding HD to portable devices, or so they claim. Since they are using custom code, I would be interested in seeing how a Nehalem runs the same benchmark with tweaked Divx code for SSE4.1.

It is a stunt though, any time someone trots out a test app on card release, it was funded directly by the card maker, and that is about all you will ever see that does this. It is almost guaranteed not to run on the next generation part.

With CPUs however, the odds are MUCH greater that it will work on the next gen, and in 3 years there will still be support for SSEx.

How many GPU or custom hardware based apps can you say this about? Ever?

-Charlie

Groo -

CUDA is actually forward compatible, guaranteed. Even better, Nvidia guarantees forward compatibility for PTX as well (the bytecode which CUDA compiles to). So you will be able to run the old CUDA apps on the new cards and get good performance.

CUDA is not a special purpose programming language. It's an answer to the general many-core programming problem. Did you know that CUDA compiles well to Intel multicore systems?
http://www.gigascale.org/pubs/1278.html

Groo-

I think you are missing the point :).

As someone currently developing in this area, I can tell you that it is something of a paradigm shift. All the big players know this (why else do you think AMD almost killed themselves purchasing ATI?), and as such are working very hard to position themselves for the new paradigm. I believe a lot of people have not yet klicked to how big this is going to be.

nVidia has CUDA on their GPUs, Intel has Ct (or will have) on Larabee, AMD has Brook+ on their GPUs.

So while these demos that are being done these days may seem contrived, they are actually a big sign of much more to come. The HPC guys are already embracing GPGPU (in some very specific areas), and before you realise it there will be computer games that silently make use of it as well (or they may not be silent about it...).

Just to give you an idea of the gulf between a GPU and CPU currently. The new generation GPUs can do about 1TFlop peak, my CPU (AthX2) about 18MFlops... :)

Malcolm

Charlie -

GPUs are turning into general purpose manycore processors - which is why Intel has to make Larrabee. Nvidia understands this well, your FUD to the contrary. They're not going to break forward compatibility with CUDA and PTX.

I'm actually a little surprised at your unfounded FUD. Too bad.