On Thu, 13 Dec 2007, Mathieu Bouchard wrote:
longitude has to be from -180 to 180. The epsilon is then the previous power of two divided by 2^23. In metres this is 0.61 metre near equator. This is the worst case. For latitude the precision is twice better than longitude at equator. In northern europe and in alaska, the longitude precision is the same as the latitude precision. It's twice more precise to use signed values than unsigned values, which is why I wouldn't use longitudes from 0 to 360.
Oh duh, my computation was wrong.
A degree is about 111319 metres of longitude around the equator, or 111138 metres of latitude. The max longitude error is 2^-16 degree or 1.69 metre and the max latitude error is 2^-17 degree or 0.85 metre. I don't remember how I computed it the first time.
You can get twice better worst case by using values ranging from -1 to +1 (because 1 is a power of two, so it lies at the boundary of a new precision level)
_ _ __ ___ _____ ________ _____________ _____________________ ... | Mathieu Bouchard - tél:+1.514.383.3801, Montréal QC Canada