locked
classic ASP giving a wrong answer to a maths calc! RRS feed

 > 

  • Question

  • Question
    Sign in to vote
    0
    Sign in to vote
    User1987415599 posted

    Hi, Can anyone give me a work-around for this please?

    If you have a classic ASP page with the following:

    <% response.write 1.0668 - 1.01885 &"<br>" %>

    This 'should' give you the answer of:

    0.04795

    but in reality gives:

    4.79499999999999E-02

    Through experimentation I can see that the issue relates to the number of decimal places in the second number AND that fact it ends in a five.  (If I change that end number from a 5 to a 4 for example -it works it out correctly.  Also, if I reduce the number of decimal places in the second number (but still have it ending with a 5), again, it works out it correctly.

    So the 'bug' appears to be related to a number with "at least" 5 decimal places ending in a 5.

    I've tried the usual cdbl() and formatnumber() without any success.

    Wednesday, January 31, 2018 10:57 AM

Answers

  • Question
    Sign in to vote
    0
    Sign in to vote
    User1987415599 posted

    ok all sorted.

    For any interested, the lack of non-zero digits in the answer was showing that number (with the "E" sufffix giving a clue to the number of zero digits present).

    To see this as an actual number I then had to surround this with a formatnumber tag and all appeared ok!

    • Marked as answer by Anonymous Tuesday, September 28, 2021 12:00 AM
    Wednesday, January 31, 2018 11:08 AM