Re: Conversion

Hi,

I think the calculator in Windows "Programs / Accesories" must do the
math in software instead of using hardware. Fixed point decimal math
is carried out to 32 digit precision, hex math is fixed point, 64 bit
precision. Pi is correct to 32 decimal places as well. I have tested
a few transcendental functions for accuracy and it seems to be there
to 32 decimal places.

Besides Intel would never make a mistake like that; I am using the
very latest Pentium 3.999999999999965319 CPU.

Mariss

--- In CAD_CAM_EDM_DRO@y..., Steve Blackmore <steve@p...> wrote:
> On Fri, 21 Jun 2002 17:39:01 -0000, you wrote:
>
> >Hi,
> >
> >Loaded Google with "inch mm convert", looked thru the first dozen
or
> >so returns until I found
http://galaxies.com/Info/Math/Conversion.htm
> >
> >I got 39.3700787401575" per meter, took the 1/x of that using
Windows
> >calculator because of its nearly unlimited digit resolution.
>
> And it's inaccuracy <G>
>
> Quote
>
>
> Mike Carlton (Intel)
> USC Information Sciences Institute
>
> The Pentium (along with every other major microprocessor) uses the
> IEEE 754 standard to represent floating point numbers. A
floating-
> point number stores a real number as sign*1.XXX*2^YYY, i.e. one
> plus a fraction, raised to a power of two, along with the sign.
> The XXX part is the mantissa and the YYY part is the exponent.
>
> The IEEE standard defines a single-precision number to have a
> 23-bit mantissa. This provides 24-bit precision (counting the
> leading one) and is equal to approximately 7 decimal digits.
>
>
> Steve Blackmore
> --
> Pilot Consultancy Limited ***** Computer Consultants
> Telephone 01744 752256 Mobile 07831 768478
> Email: steve@p... WWW: http://www.pilotltd.net