You don't, the coefficients can be complex too. However, I discovered that mirroring (*) every pole and zero results in just real values without imaginary part. I don't have any mathematical proof for this, but it probably wouldn't be too hard to find such.

*) adding another pole/zero for each complex one, like z=-j if you already have a z=j.