Hi Alexandre
Nice patch! I've just been working on something very similar, but yours is much classier.
Sorry this is coming so late... I've only just got to this, but I think I know what's going on...
It looks like a mismatch between envelope and sample reading. The clicks seem to happen because the amplitude envelope and the tabread4~ aren't synchronized. What happens is the tabread object gets its new starting position while the envelope is still in mid-grain. The solution is to make sure each voice is inactive before starting a new grain.
David Golightly made a great granular delay line patch (I think based on a paper by Ross Bencina) maybe a year and a bit ago which used a scheduler to send info to each voice. As I remember, the scheduler kept track of which voices were playing and sent the messages to the next available note. Another way is to let the voice pass the message to the next voice if it's already busy.
I hope this makes some sense.
cheers dafydd