Thursday, July 22, 2004

I was having a discussion on MSDN Newsgroup in response to a post that said that simple arithmetic in VS.NET command window yields wrong results.

?11075.04+1814.20
12889.2484

was the result (and this is what happens!)  I have seen this problem before though I can't recollect exactly where.

Code like this:

double x = 11075.04+1814.20;

MessageBox.Show(x.ToString());

if(x == 1468.62) { MessageBox.Show("true"); } else { MessageBox.Show("false"); }

will show the expected 12889.24 as the first messagebox result and then show false!  The if condition basically does a bit-wise comparison and hence fails (since the actual answer is 12889.2484)

The problem has its roots in how computers are designed to handle floating point numbers.

Jon's homepage - a wonderful resource for some very interesting articles - has a complete description of the issue and the reasons behind it:  http://www.yoda.arachsys.com/csharp/floatingpoint.html 

 

Thursday, July 22, 2004 10:34:45 AM (India Standard Time, UTC+05:30)
1468.62? where did you get that number from?
Sidharth
Thursday, July 22, 2004 3:59:10 PM (India Standard Time, UTC+05:30)
yeah yeah :)

realized that this morning. was too late in the night and after a lot of UT with some OM, was happy enough to see i had made two posts! :)

Had to be 12889.24 - and it still gives you false :)
Thursday, July 22, 2004 5:06:56 PM (India Standard Time, UTC+05:30)
I must admit, your blog's manual notification feature is quite interesting. Especially the annotation feature. :-)

And I do know of the pitfalls of floating point calculations. Floating point numbers are meant for engineering/scientific application where exactness is not necessary. If someone wants exact values he should use some sort of BigDecimal type.
Sidharth
Name
E-mail
Home page

Comment (HTML not allowed)  

Enter the code shown (prevents robots):