Answered by:
14.0 / (14 * 0.01) = ?
Question

Hello everyone.
Got issue with calculation items success ratio. Here's source code:
List<ResponseMail> conversationsForCurrentPeriod = _responsesForCurrentPeriod; if (conversationsForCurrentMonth.Count <= 0) { return 0; } double successCount = conversationsForCurrentMonth.Count(obj => obj.ResponseInterval > 0); double result = successCount / (conversationsForCurrentMonth.Count * 0.01); return result;
So in case I got 14 records in '_responsesForCurrentPeriod' collection and all of them are successful (their ResponseInterval is greater then 0) I got successCount = 14. So the 'result' variable is calculated by operation 14.0 / (14 * 0.01). The result of this operation should be 100 but in my particular case I'm resulted with 99.999999999999986.
Please tell me whether this is a bug or what ?
Thanks in advance. Edited by eks77ern Saturday, December 20, 2014 5:51 PM
Answers

this is a standard (IEEE 754) spec not a bug. Most numerical values cannot be stored in a finite amount of space thus data are truncated when storing, causing rounding errors that you must take into consideration when writing code.
Visual C++ MVP Edited by Sheng Jiang 蒋晟 Saturday, December 20, 2014 10:04 PM
 Marked as answer by Fred BaoModerator Monday, December 29, 2014 9:11 AM

If acceptable, consider decimal type instead of double:
int successCount = 14;
decimal result = successCount / ( 14 * 0.01m );
 Proposed as answer by Blackwood Thursday, December 25, 2014 3:07 PM
 Marked as answer by Fred BaoModerator Monday, December 29, 2014 9:11 AM
All replies

this is a standard (IEEE 754) spec not a bug. Most numerical values cannot be stored in a finite amount of space thus data are truncated when storing, causing rounding errors that you must take into consideration when writing code.
Visual C++ MVP Edited by Sheng Jiang 蒋晟 Saturday, December 20, 2014 10:04 PM
 Marked as answer by Fred BaoModerator Monday, December 29, 2014 9:11 AM

If acceptable, consider decimal type instead of double:
int successCount = 14;
decimal result = successCount / ( 14 * 0.01m );
 Proposed as answer by Blackwood Thursday, December 25, 2014 3:07 PM
 Marked as answer by Fred BaoModerator Monday, December 29, 2014 9:11 AM