20 instances of the same abstraction:
(these are counts of the totals for the whole patch)
40 gemheads
100 pix_texture
120 rectangle
20 pix_buffer
60 pix_buffer_read
20 pix_buffer_write
160 translate
80 scale
20 color
40 alpha
160 separator

That many separators will likely stall the GPU quite a bit since that basically resets the rendering pipeline. That many pix_texture objects constantly uploading textures will also stall in the driver on Linux and Windows (your patch would probably go like 0.2 fps on Windows).

I really have no idea how the nv driver works, what your saying tells me
that each gem GL function should execute a function in the nvidia driver.
But since the driver is closed source without symbols would there be any
way of getting those calls?

The profiler will take note of any memory location in the driver that takes time and then it should also keep track of what calls lead up to it. So if you do a glColor4f() in GEM that will make the driver do something with that call.

The symbols in the driver aren't all that important unless there is a bug to alert Nvidia about.

attached is the patch, though it depends on pix_video uses in a weird way,
and my serial control camera

A Pan/Tilt/Zoom camera?

cgc