Hi Py,
I think WebPd core could definitely be used for that, and thus the refactoring I am doing these days is a good thing as it will create a good base to start these kind of projects.
Personally, I am not very familiar with graphics on the web (especially when it comes to WebGL), but I am sure that for anyone with a good experience in this, it should be pretty straightforward, for example using one of the many good libraries that make WebGL, SVG or canvas easier (d3.js, three.js, ...), to replicate some GEM objects in WebPd. And would be nice indeed! But I see this as something more in the future ...