I had the same problem with cpu usage peaking when the patch was silent on the rpi with pd 0.43-something (whatever was in the standard rpi repos) Turned out it was the [hip~] that was causing the problem. (I used two [hip~ 5] before the [dac~] and simply removing them completely solved the problem)
The [hip~] bug might be solved in the 0.44-3 though I have not tested it yet.
Unfortunately my raspberrys are packed into boxes right now, so I can't test.
During next week they will be back with me and I continue working on the setup - looking forward to testing your tip!
// Halfdan