mid-March 2022: I'm finally getting back to normal and returning to projects I was working on before the pandemic. 2 years of shifting priorities... ya know
The project I was working towards is doing long-length cross-covariance and deconvolution calculations in a "fast-forward" pd instance and then passing the results back to a real-time process, with minimal risk to real-time
A pd process that is running in fast forward does not poll and so does not listen for connections of some kinds. When fast forward is turned off by the patch, it starts using the regular scheduler and starts listening again.
At the time, 2 years ago, I had gotten this kind of behavior from batch mode and communication through shmem and some signal handlers (raisestop, sigcont). With [ggee/shell] to start/stop the processes and send sigcont, it rounded out an (linux-specific, and very kludgey) solution. A batch mode process could wake up, read some shmem ID's and be told what to do, transfer data in, do some calculations, write the output to other shmem ID's, and then sleep again. To make the programming a bit easier to read, it was also possible to send lists using FUDI over shmem.
My conclusion there was that shmem can be used for asynchronous inter-process communication with minimal risk to real-time. It's very good as a fundamental object--it does not block, it does not synchronize. Notable limitations:
- Every process needs to know/use the same size for shmem ID's.
- Once allocated, a shmem cannot be re-sized.
- Writing to/from an extremely large array all at once poses a risk
to real-time.
I'd like to write a pair of management abstractions for using fast forward and shmem, then, that make it easy to stage in/out large, variable-length data
Anybody else have best practices for IPC when using fast forward? Having a listener on the 2nd process between computing sprints in the "fast forward" process completely changes how I can do things.
other: is there a good way to start/stop processes other than [ggee/shell]? cross-platform?
Chuck