Think of this: in the most extreme case where a single bit is used to represent signal values, you get an output signal of ones and zeros no matter what input level. If the input level was low it becomes louder but if the input level was far above unity gain it becomes softer. You could say that bit reduction acts as a compressor/limiter itself. If your input signal is hot, the amplitude effect of bit reduction will be smaller but there is still the psycho-acoustic effect of the extra harmonics. My approach is to pump up the volume before bit reduction (using a compressor/limiter, see
http://www.katjaas.nl/compander/compander.html). For heavy bit reduction I compensate output level but I don't have a mathematically justified recipe for that.