locked
Decimal to decimal conversion RRS feed

  • Question

  • I have a decimal number d = 80.12345. and class property decimal FixedPrice{get;set;}

    When I assign FixedPrice = d;, the value of Fixed price  = 80.1235.

    However, i want to store it as 80.12345.

    Can it be explained why a decimal number when assigned to another decimal is loosing its scale?


    Thanks, Ankita

    Monday, July 8, 2013 7:15 AM

Answers

  • Ah. It was the suspicion I talked about in my July 08 post (that you perhaps used a currency data type rather than really a decimal).

    Please use "Mark as answer" on any response that best fit  so that the thread can appear as "closed".


    Please always mark whatever response solved your issue so that the thread is properly marked as "Answered".


    Wednesday, July 10, 2013 9:17 AM

All replies

  • maybe you can try to use "Decimal.Parse(values)" while assiging the decimal values. Nomral decimal to decimal conversion should not round off values.

    regards

    joon

    Monday, July 8, 2013 7:44 AM
  • How did you determine that FixedPrice was 80.1235?  Did you write it to the console?  Did you use the debugger?  Did you format it as a string? or ??? (and similarly, how did you determine that d was 80.12345?)


    Paul Linton

    Monday, July 8, 2013 7:59 AM
  • d is being calculated in code and is assigned with 80.12345 .

    then FixedPrice = d.Value.HasValue ? d.Value : 0;


    Thanks, Ankita


    • Edited by Ankita Kumar Monday, July 8, 2013 8:18 AM Not completed
    Monday, July 8, 2013 8:15 AM
  • I'll try again

    How did you determine that FixedPrice was 80.1235?  Did you write it to the console?  Did you use the debugger?  Did you format it as a string? or ??? (and similarly, how did you determine that d was 80.12345?)


    Paul Linton

    Monday, July 8, 2013 8:16 AM
  • d is being calculated in code and is assigned with 80.12345 .

    then FixedPrice = d.Value.HasValue ? d.Value : 0;


    Thanks, Ankita


    if d is already a decimal, then maybe you can directly assign d to FixedPrice. The value is getting round off due to some kind of datatype conversion.

    regards

    joon

    Monday, July 8, 2013 8:38 AM
  • I see you have edited your reply.  Slowly information is divulged.

    It would seem that d is not of type decimal but is something else which contains a property called Value which is of type Nullable<decimal>.  If you really assign FixedPrice in the way you have shown and if the type of d.Value is really Nullable<decimal> and d really is 80.12345 (although you still will not reveal how you know this) then FixedPrice is 80.12345 despite what you say.  The method you are using to find the value of FixedPrice is wrong.  Can you tell us how you determine that the value of FixedPrice is 80.1235 (bearing in mind that it does not have that value if all the assumptions that we have had to guess about your code are correct)?  I know I have already asked twice but, hey, third time lucky.


    Paul Linton

    Monday, July 8, 2013 8:48 AM
  • Hi ,I think I figured out the issue. I modified 0 with default (decimal)

    FixedPrice = d.Value.HasValue ? d.Value : default(decimal);

    I do not understand how it resolved though!


    Thanks, Ankita

    Monday, July 8, 2013 9:24 AM
  • Hi Ankita Kumar

    decimal FixedPrice { get; set; }
    decimal d = 80.12345m
    
    FixedPrice = d;
     The code works I tested it.

    There you go, happy?



    • Edited by Dzunisani Monday, July 8, 2013 9:32 AM
    Monday, July 8, 2013 9:31 AM
  • I think you can define number of decimal places you need, Check this link

    Mark Answered, if it solves your question and Vote if you found it helpful.
    Rohit Arora

    Monday, July 8, 2013 9:33 AM
  • What is you code? when I run this in my PS I get 80.12345!

     

    Noam B.



    Do not Forget to Vote as Answer/Helpful, please. It encourages us to help you...

    Monday, July 8, 2013 10:20 AM
  • Hi,

    It would have been easier if you showed some sample to show how you store FixedPrice (I suspect you were using the currency datatype).


    Please always mark whatever response solved your issue so that the thread is properly marked as "Answered".

    Monday, July 8, 2013 11:42 AM
  • Let me try and examine precisely why there is a disconnect between Ankita Kumar and the answerers. 

    First of all, this doesn't compile.

    decimal price = 80.12345; // error CS0664
    

    Why?  Because 80.12345 is a numeric literal of type double which can't be used to initialize a variable of type decimal because of the potential loss of precision due to the different storage formats of the numbers.  To make it a numeric literal of type decimal, you need to include the suffix m or M, like this:

    decimal price = 80.12345m; // OK

    Secondly, with that particular value (80.12345), you do not lose any precision going to either the float or double data types.

    This can be verified like this:

    float myFloat = (float)price;
    Debug.Assert( myFloat.ToString() == price.ToString() );
    double myDouble = (double)price;
    Debug.Assert( myDouble.ToString() == price.ToString() );
    

    Thirdly, this code doesn't compile:

    Nullable<decimal> d = price;
    decimal FixedPrice = d.Value.HasValue ? d.Value : default(decimal);
    

    The full error is:

    error CS1061 : 'decimal' does not contain a definition for 'HasValue' and no extension method 'HasValue' accepting a first argument of type 'decimal' could be found (are you missing a using directive or an assembly reference?)

    There's a simple fix for this:  to obtain the .HasValue property from d rather than d.Value, like this:

    decimal FixedPrice = d.HasValue ? d.Value : default(decimal);
    

    Even with 0 rather than default(decimal) it still works fine, and doesn't give your purported rounding problem.

    decimal FixedPrice = d.HasValue ? d.Value : 0;
    Debug.Assert( d.ToString() == price.ToString() ); // OK
    

    Now having said all that, there is a suspicious thing, that log base 2 of 8012345 is ~22.934 And a single precision float number has exactly 23 bits of fraction, so 80.12345 is actually VERY close to the limits of the precision of a single-precision floating point number. And that's also why .ToString() stops expanding to the decimal representation of a float after that many digits.  In fact, the .ToString() representation of these numbers are all identical to their single-precision float counterparts:

    80.12345
    80.12346
    80.12347
    80.12348
    80.12349

    None of them round up to 80.1235.  You still have a couple of bits of precision, so this is suspiciously unlikely to be a rounding error.

    In fact, I'm willing to bet that the reason you observed 80.12345 at all as an answer is because that is the precision to which floating point numbers are displayed in the first place.

    So I think the key piece of information here is that you said: 

    d is being calculated in code and is assigned with 80.12345

    So if it is calculated in code then you should show the calculation that you used.  Because it seems likely that you lost precision around the 23 bit mark, or falsely displayed a number as a single-precision float somewhere.  So you haven't conveyed to us the actual computation that you used.

    And the one resounding thing that prevents me from claiming that this is a single rounding error is that even at the lowest precision (23-bits of fraction in a single-precision float) there are still 5 code points between your value and your supposed rounding error:

    Here is a quick list of some binary representation of some single-precision floats (shown here in hex) and their .ToString() representations.

    42a03f34 = "80.12344"
    42a03f35 = "80.12345" // Your original
    42a03f36 = "80.12346"  !
    42a03f37 = "80.12347"  !
    42a03f38 = "80.12347"  !
    42a03f39 = "80.12348"  !
    42a03f3a = "80.12349"  !
    42a03f3b = "80.1235" // Your supposed rounding error
    42a03f3c = "80.1235"
    42a03f3d = "80.12351"

    It doesn't make sense that there would be this much error from a single rounding mistake.  I would expect that 80.12345 could be come the code point before it or after it (80.12344 or 80.12346) but not all the way to 80.1235.  That seems a little extreme.  It would take 3 separate rounding errors to compound to this error, even at single precision.  I suppose that's possible, but you would likely notice these rounding errors in other places.

    Something's fishy here.  Either you're not telling us the whole story, or I'm about to learn something profound about rounding errors that I didn't know before.

    [It's also worth noting that the debugger uses doubles for the representations of many numbers internally when formatting information for the user.]

    Monday, July 8, 2013 1:36 PM
  • Hi,

    Yes d is a Nullable<decimal> type. This is being calculated in the code with value 80.12345 and then it is assigned to d. Fixed price is just a decimal type property.


    Thanks, Ankita

    Tuesday, July 9, 2013 7:24 AM
  • If d is Nullable<decimal> then the code you showed earlier

    FixedPrice = d.Value.HasValue ? d.Value : default(decimal);

    will not compile.  So, is the code correct or is d of type Nullable<decimal>?

    Is there any chance that you might answer my question?  What makes you think that FixedPrice is 80.1235?  (Because it isn't)


    Paul Linton

    Tuesday, July 9, 2013 7:29 AM
  • Sorry, would like to make that correction.

    FixedPrice = d.HasValue ? d.Value : default(decimal);


    Thanks, Ankita

    Tuesday, July 9, 2013 7:32 AM
  • Does refusing to tell us what makes you think that FixedPrice is 80.1235 mean that you do not want your problem solved?  If so, please let us know and we can all move on to helping someone who really does want help.

    Paul Linton

    Tuesday, July 9, 2013 7:35 AM
  • By the way, you have just managed to write the .Net function GetValueOrDefault().  Your code is the same as

    FixedPrice = d.GetValueOrDefault();


    Paul Linton

    Tuesday, July 9, 2013 7:37 AM
  • If you want this solved, then it's probably time to copy and paste some code here I think.  You've already demonstrated one error with the d.Value.HasValue where that clearly wasn't the code you were running, so we (PaulLinton in particular) are suspicious that you are not giving us the correct details of what you have done.  With copy and paste, we can be sure we are debugging the code you are running.

    Please share the code.

    If you don't want to share the code, then please try to reproduce your problem in a small, stand-alone test program, and copy and paste the code for that.

    Tuesday, July 9, 2013 12:49 PM
  • Hi,

    Would second that and in my opinion even always show the smallest amount of code that shows the issue rather than an excerpt of your real code. Either you'll find yourself the issue by creating the snippet or you'll have some code to show that goes straight to the point (showing too much unrelated code will decrease the willingness to help as it shows you don't even care about isolating the offending lines as a easily runnable sample).


    Please always mark whatever response solved your issue so that the thread is properly marked as "Answered".

    Tuesday, July 9, 2013 1:55 PM
  • Hi,

    Well sorry i cant put the actual code here in the forum, but the code essentially calculates the fixed price based on the sample code provided. D is a value which the user inputs from the UI.

    I myself tried the code in a console program, with 80.12345M ( as Dzuniman suggested) and it worked fine.

    The problem  seemed to be on the backend in actual code where the fixedprice was being stored as money(data type).It stores only upto 4 decimal digits and hence to was round off the 5th digit.

    Appreciate the help I got from you all.

    Cheeers


    Thanks, Ankita

    Wednesday, July 10, 2013 8:56 AM
  • Ah. It was the suspicion I talked about in my July 08 post (that you perhaps used a currency data type rather than really a decimal).

    Please use "Mark as answer" on any response that best fit  so that the thread can appear as "closed".


    Please always mark whatever response solved your issue so that the thread is properly marked as "Answered".


    Wednesday, July 10, 2013 9:17 AM