Hallo!
Max MSP/Jitter has bridge objects that convert from audio rate to
matrices and back, which would be needed in PD land to readback from
the GPU, and convert to an audio rate signal. Thats how this is done.
Otherwise i have no idea how you would implement souch a patch.
I think the easiest way to do it is simply in the C code of the audio externals (there exist already nice libraries for general computing on the GPU - as pointed to in the last link).
However, of course it would be also interesting to do that at the patching level, but I don't know if that is possible ...
LG Georg