Sounds like a winner to me. At one point I had things working with no audio dropouts but then all of a sudden on some patches particularly ones that have
a) unresolved sends/receives
b) mouse input from gem or via canvas dragging in editable window
Here's how I did it, I have a web interface that controls which of the patches is running:
.hc