You can filter out some noise if that happens. I have done several arduino projects (namely public installations or sculptures with sensors), one thing I learned is that you can always filter the (some) noise, by ignoring some values and so forth.
Another fun aspect is that in public installations whenever I connected the arduino to the AC power outlets (instead of battery) the noise was much higher. (which makes sense, since AC powers of houses, steet, etc.. are actually noisy (have you ever noticed that AC induces noise on most laptop on-board sound-cards - due to ground loop and so forth) )
>use i wanted to use a pot to control a delay length (you can hear gliches if the delay length jumps from one value to another rapidly).
But don't panic about it, its workable. You can defined thresholds, and if it varies withing a certain range you will not consider as the pot has moved, if you do not have noise: even better. But electricity is not perfect, components are definitely not perfect either :)
Best,