I actually tried a smaller patch on both ubuntu studio 10.10 as well as puredyne 9.10. On both systems the audio was flawless using a 6ms buffer in pd.
So there is nothing wrong with my audio setup.
But still the larger patch that runs fine on windows with 50ms buffer will not run without glitching with jack on either linux system.
There is a difference in gem's default video codec between linux and windows. However I have tried the patch without even loading gem.
What else could be the difference between linux and windows with pd?