# Why I do not get the exact operation

• ### Question

• Hi folks,

I am transforming feet-in to cm, but I have a case where I am getting a lot of decimals in the transformed value, although using the calculator, it gets an exact value. Here is the function:

public double ConvertFTLevelToCM(int ftLevel, intinLevel)

{

double levelInCM;

levelInCM = ftLevel * 30.48f + inLevel * 2.54f;

}

If I set ftLevel = 22, inLevel = 6, using a calculator it gets an exact value of 685.8, but using the function it gets 685.79998970031738

The problem I am facing with this is that I have to truncate the result. It should truncate from the exact value of 685.8, which is 685.8 exactly and correct, but using the result value of 685.79998970031738, the trunc value is 685.7, which is not correct.

I am using Microsoft Visual c# 2008 and wonder why c# does not drop the exact value of 685.8 and looking for a general workaround when I should have exact results in the conversions.

Thanks,

Marco

Monday, July 4, 2011 11:35 PM

• This is standard behavior - see the Floating Point Guide for details: http://floating-point-gui.de/

There are a few things you could do to help.  I'd recommend not using floats (use 30.48 instead of 30.48f), since your math will cause a double conversion anyways.  This will allow things to be more accurate.

If you need the most accuracy, try using decimal (instead of double) for all of your math.  For example, try this:

```public decimal ConvertFTLevelToCM(int ftLevel, int inLevel)
{
decimal levelInCM;
levelInCM = ftLevel * 30.48m + inLevel * 2.54m;

}
```

That being said, double will give you a LOT of precision - it's more a matter of how you use/display the result.  Instead of truncating, you should always round as needed, etc.

Reed Copsey, Jr. - http://reedcopsey.com
• Proposed as answer by Tuesday, July 5, 2011 4:15 AM
• Marked as answer by Tuesday, July 12, 2011 2:21 PM
Tuesday, July 5, 2011 12:09 AM

### All replies

• This is standard behavior - see the Floating Point Guide for details: http://floating-point-gui.de/

There are a few things you could do to help.  I'd recommend not using floats (use 30.48 instead of 30.48f), since your math will cause a double conversion anyways.  This will allow things to be more accurate.

If you need the most accuracy, try using decimal (instead of double) for all of your math.  For example, try this:

```public decimal ConvertFTLevelToCM(int ftLevel, int inLevel)
{
decimal levelInCM;
levelInCM = ftLevel * 30.48m + inLevel * 2.54m;

}
```

That being said, double will give you a LOT of precision - it's more a matter of how you use/display the result.  Instead of truncating, you should always round as needed, etc.

Reed Copsey, Jr. - http://reedcopsey.com
• Proposed as answer by Tuesday, July 5, 2011 4:15 AM
• Marked as answer by Tuesday, July 12, 2011 2:21 PM
Tuesday, July 5, 2011 12:09 AM
•

Hi,

I am writing to check the status of the issue on your side. Would you mind letting us know the result of the suggestions?

If not, any more concerns, please feel free to let us know.

Have a nice day!

Paul Zhou [MSFT]
MSDN Community Support | Feedback to us
Get or Request Code Sample from Microsoft
Please remember to mark the replies as answers if they help and unmark them if they provide no help.

Tuesday, July 12, 2011 5:40 AM
• Everything worked fine and the solution is as expected. Marked as Answered.

Thanks,

Marco

Tuesday, July 12, 2011 2:22 PM