Dear list,
I'd like to start a conversation about CUDA and Pd.
For those of you who don't know, CUDA is a minimal instruction set for doing single precision floating point calculations on NVIDIA GPUs. It's a C-based coding paradigm in which blocks of data are copied to GPU device memory and operations are performed on that data with thread blocks in increments of 32 threads. Complete sets of floating point math functions are available for CUDA. The CUDA compiler nvcc works very well alongside gcc.
I've been studying it at work, but have not coded anything for Pd yet. There's a whole lot of performance issues based on tiny details in the documentation--the implementation of cuda based externals could be made fairly simple for developers if a complete set of CUDA<->Pd extensions could be coded from the beginning.
Any project worth doing is worth doing right. So, I want to figure out if: a) it's worth doing and b) how to do it right.
I've got a first draft of top-down design issues, and I'd like to make a list of incremental milestones that would prove the concept is sound.
top-down design issues: 1. The essential CUDA<->Pd functions should be made separate from CUDA based Pd externals, with a separate header file, and compilable to shared and static libraries. 2. The set of CUDA<->Pd extensions needs to be able to manage multiple devices, including device query, initialization and setting global parameter sets per GPU. Most likely, this means a custom data structure and object based method system. 3. Compilation--how to create the build system and handle dependencies for a library of CUDA based externals. Management of CUDA libraries, CUDA-rt and CUDA-BLAS especially. 4. Testing and initialization. At setup time, a CUDA based external should be able to find out if it is legal and ready to run. 5. Abstraction of major device and memory operations. What makes up a sufficient and complete set of operations? This is a list that is most likely to be grown through experimentation, but a good preliminary list of operations will help get things started on the right footing. 6. Performance. How to profile or benchmark and make comparisons between implementations? The single greatest performance issue that I have identified is caching on GPU. host<->device memory transfers can be eliminated in some cases, allowing CUDA based externals to follow one another in the DSP tree with faster scheduling and runtime performance.
(proposed) incremental milestones: 1. Create an external that checks GPUs and hands back error messages to Pd. 2. Create an external that initializes GPUs. 3. Create an external that performs host<->device memory transfer and runs an operation. 4. Create an external that performs an operation and compares the time it takes against the same operation on CPU. At this point, it should be possible to identify and hopefully quantify the potential speedup on GPU, and decide whether or not it is worth it.
That's enough for now. I'd like to know if anyone else has been thinking along similar lines (CUDA has been out for, like, 2 years or so now, so I bet that many people know about it), and if you have any input on the design issues.
Chuck
Hey,
Some brief initial thoughts:
http://en.wikipedia.org/wiki/CUDA pro: better than OpenGL GPGPU hackery pro: API/ABI-stable for foreseeable future con: only one hardware vendor
-vs-
http://en.wikipedia.org/wiki/OpenCL pro: royalty free standard with several vendors pro: works on cpus as well as gpus con: newer, may be API/ABI-unstable
-vs-
http://faust.grame.fr/catalog.php pro: dataflow language optimized for DSP pro: has backend for Pd already con: works by translation via C++
Any project worth doing is worth doing right. So, I want to figure out if: a) it's worth doing
Sufficiently-parallel DSP on GPU is probably worth doing, but only benchmarks will tell. Parallel DSP could either be multichannel or blocky (no data dependencies between samples in the same block) or a mixture of both at different parts of the graph.
b) how to do it right.
I did have an idea that went something like this:
1. write ugens in Faust, compile them into dsp objects, link, load 2. write a transaction-based patcher environment (or dataflow language) wherein transactions (such as multiple object creation deletion connection etc) can be committed atomically (with OSC-style bundle time tags etc) 3. translate the generated networks of dsp objects back to Faust code 4. compile that Faust code into a new dsp object, link, load, run 5. rinse, repeat
So, an OpenCL backend (instead of C++ backend) for Faust is the missing link...
Claude
top-down design issues:
- The essential CUDA<->Pd functions should be made separate from
CUDA based Pd externals, with a separate header file, and compilable to shared and static libraries. 2. The set of CUDA<->Pd extensions needs to be able to manage multiple devices, including device query, initialization and setting global parameter sets per GPU. Most likely, this means a custom data structure and object based method system. 3. Compilation--how to create the build system and handle dependencies for a library of CUDA based externals. Management of CUDA libraries, CUDA-rt and CUDA-BLAS especially. 4. Testing and initialization. At setup time, a CUDA based external should be able to find out if it is legal and ready to run. 5. Abstraction of major device and memory operations. What makes up a sufficient and complete set of operations? This is a list that is most likely to be grown through experimentation, but a good preliminary list of operations will help get things started on the right footing. 6. Performance. How to profile or benchmark and make comparisons between implementations? The single greatest performance issue that I have identified is caching on GPU. host<->device memory transfers can be eliminated in some cases, allowing CUDA based externals to follow one another in the DSP tree with faster scheduling and runtime performance.
add: 7. Namespace. Should be able to duplicate existing objects with unified variations on names.
(proposed) incremental milestones:
- Create an external that checks GPUs and hands back error messages to Pd.
- Create an external that initializes GPUs.
- Create an external that performs host<->device memory transfer and
runs an operation. 4. Create an external that performs an operation and compares the time it takes against the same operation on CPU. At this point, it should be possible to identify and hopefully quantify the potential speedup on GPU, and decide whether or not it is worth it.
add: 5. Create an external that accepts *in as a pointer to host memory, and returns as *out a pointer to gpu memory. 6. Create an external that accepts *in as a pointer to gpu memory, and returns as *out a pointer to host memory. 7. Create an external that performs an operation in gpu memory and returns a pointer to gpu memory without any host<->gpu memory transfers.