Which version of GEM?  There is a problem with the QT DV component looping the audio track that will do exactly what you describe.  I disabled the first audio track in any movie file in the GEM code but your clip might have more than one audio track. 

Also at a certain point you will hit the limit of what the machine will do.  The Mac 'Pro' isn't as fast as Apple's marketing wants it to be.  The Intel chips have serious memory bus speed problems and the OSX kernel likes to destroy any CPU affinity. 

If you do not need any pix_ objects for processing then use pix_movie.  That will improve performance quite a bit since it uses the fastest texturing path.