What I would do is use your typical sound playback algorithm, ([phasor~] feeding [tabread~] and relevant arithmetic. But take a side chain from the [phasor~] to another [tabread~] via a [*~ 10000](or whatever length). and draw your pitch curve into an array (with it's length in that [*~] and with a y range of something like 0 - 2). sending the output of the second tabread back to the phasor(which should happily change it's speed sample-by-sample without losing it's position) by a multiply (from your original speed calculation.
Something like...
[O]
|
[arraysize sound]
| \
| [s sample_length]
[expr 44100 / $f1]
|
| [r pitch_curve]
| /
[*~]
|
[phasor~]
| \
| [s phasor]
|
| [r sample_length]
| /
[*~]
|
[tabread~ sample]
| \
[dac~]
[r phasor]
|
| [O]
| |
| [arraysize pitch]
| /
[*~]
|
[tabread pitch]
|
[s pitch_curve]
(I've put that together quickly and in the middle of the night, might well have made a mistake or two)
Of course, while this will change the speed of playback based on the position of playback, it is very difficult to express it as musical pitch differences. setting the scope of your pitch curve to 0 - 2 means that by multiplying it by the 'correct' playback speed you can go down to no movement at all (which will probably stall this sytem) and all the slow speeds related to this and up to twice the original playback speed (one octave above). Which will be good for creative sound, as I say, hard to scale as semiquavers.
Andrew