>> I'm anxious to know what limit is reached in the coefficients of the filter that causes the undefined result (NaN).
>
>I haven't seen the code, but I just want to make you notice that adding together -Infinity and +Infinity results in a NaN ; so does subtracting
>two infinities of the same sign.
>
>So, the NaN might happen when two expressions that are supposed to partially cancel each other, happen to both overflow, in different directions.
>
>There are various possible causes for NaN, but with formulas that only involve +, - and *, the possibilities are a lot more limited.
Hmmm.
I think that's why it
puzzles me so. Can you see a / anywhere here?
while (n--) {
i1=(*in++);
fc1 = (*fc++);
res1 = (*res++);
q = 1.0f - fc1;
p = fc1 + 0.8f * fc1 * q;
fcoeff = p + p - 1.0f;
q = res1 * (1.0f + 0.5f * q * (1.0f - q + 5.6f * q * q));
i1 -= q * xb4;
t1 = xb1;
xb1 = (i1 + xb0) * p - xb1 * fcoeff;
t2 = xb2;
xb2 = (xb1 + t1) * p - xb2 * fcoeff;
t1 = xb3;
xb3 = (xb2 + t2) * p - xb3 * fcoeff;
xb4 = (xb3 + t1) * p - xb4 * fcoeff;
xb4 = xb4 - xb4 * xb4 * xb4 * 0.166667f;
xb0 = i1;
*out++ = xb4;
}
No reciprocals, no divisions, just +,-,*.
I cannot
help thinking I must be reaching a "limit" in the calculus sense...
Ed
______________________________________________________________________
| Mathieu BOUCHARD ----- téléphone : +1.514.383.3801 ----- Montréal, QC