No announcements
Found 2771147 threads
-
0 VotesAdding 2 floating point numbers in query produces wrong result
I have a query tha tads 2 FP (single 2) numbers (0.0 and 89.4) and the result is 89.4000. see query ...Proposed | 14 Replies | 627 Views | Created by Steve Staab - Monday, April 25, 2016 11:45 PM | Last reply by Albert D. Kallal - Sunday, May 8, 2016 5:59 PM
-
0 Votes
Trailing digits when adding certain floating point numbers in javascript
I'm was making a simple input/output gadget when I ran into a strange problem in Javascript. Essentially when adding two floating point numbers ... -
0 Votes
tncating floating point numbers
takes up way too much screen-estate displaying all the uncessary detail. I would like to have maybe 2 or 3 numbers after the decimal point and i'm wondering which ...Answered | 2 Replies | 2829 Views | Created by amp1928 - Monday, May 31, 2010 11:54 PM | Last reply by amp1928 - Tuesday, June 1, 2010 12:37 AM -
0 Votes
Wrong floating point assembler generated
What are you expecting anyway, floating point numbers aren't accurate. There are a few reasons why you could have this type of change in the results and all of them ...Answered | 2 Replies | 2828 Views | Created by rumpelstilzchen2010 - Thursday, June 9, 2011 4:51 PM | Last reply by Darran Rowe - Thursday, June 9, 2011 11:49 PM -
3 Votes
Floating-point numbers and equality
Floating-point numbers should never be directly compared for equality, because the IEEE floating-point representation does not support an exact equality ...Answered | 1 Replies | 3108 Views | Created by Rainmater - Friday, November 2, 2012 8:41 AM | Last reply by Sehnsucht_Fr - Friday, November 2, 2012 11:12 AM -
0 Votes
Floating Point, Additions and Random Numbers?
In the addition of real numbers like 0.1234 + 0.1234 + 0.1234 + ........ appears random number to the right of the point like this result 12.34000000000005Answered | 1 Replies | 3621 Views | Created by Determinant - Wednesday, June 11, 2008 11:39 PM | Last reply by WayneAKing - Thursday, June 12, 2008 1:17 AM -
0 Votes
Int/Floating-point numbers conversion
integer and floating-point calculation. I found it very hard because there are common years as well as leap years. And each month has different ...Answered | 7 Replies | 1747 Views | Created by VBui - Wednesday, January 4, 2012 8:21 AM | Last reply by Barry-Schwarz - Thursday, January 5, 2012 5:00 AM -
0 Votes
loop for floating point numbers
0.3 in all situations. If you want exact calculations on a "floating-point" type field, then consider using a "currency" type ...Answered | 3 Replies | 275 Views | Created by IntenseNJ - Monday, May 16, 2016 3:42 PM | Last reply by Viorel_ - Monday, May 16, 2016 5:35 PM -
0 Votes
Matrix transform produces wrong result (SL4)
* mt.Matrix.M22 + mt.Matrix.OffsetY; I get this result: X = 425.011518239975 Y = 48.025378704071 I ...Answered | 3 Replies | 1017 Views | Created by asthomas - Sunday, September 11, 2011 1:40 PM | Last reply by asthomas - Thursday, September 15, 2011 9:28 AM -
0 Votes
skeleton in x64 Asm floating point numbers in Visual C++
Dear Community Is this skeleton of program.cpp is completely in order to addressing 64 bit floating point number in Asm x64 ...Answered | 1 Replies | 619 Views | Created by RB _1 - Wednesday, November 19, 2014 7:04 PM | Last reply by davewilk - Wednesday, November 19, 2014 8:02 PM -
1 Votes
Floating point errors with irrational numbers
Yes. Please study What Every Computer Scientist Should Know About Floating Point Arithmetic And I do mean study. Don't just ...Answered | 1 Replies | 346 Views | Created by GadMolkho - Thursday, January 22, 2015 8:41 PM | Last reply by Brian Muth - Thursday, January 22, 2015 9:41 PM -
0 Votes
Floating Point Precision issue
I am working on a multithreaded application in VC++ that does a lot of floating point computation. I was wondering whether it is possible that the order ...Answered | 2 Replies | 3883 Views | Created by rKarthik - Monday, August 14, 2006 12:57 AM | Last reply by rKarthik - Monday, August 14, 2006 2:22 AM -
0 Votes
Floating point
term floating point refers to the fact that the radix point (decimal point, or, more commonly in computers, binary point) can "float": ...Unanswered | 7 Replies | 3404 Views | Created by Dave Pf - Friday, July 18, 2008 4:13 PM | Last reply by SMD- - Wednesday, July 23, 2008 12:41 PM -
0 Votes
Library with Floating Point numbers with multiprecision arithmetic
"floating point numbers and not for integer" Use the J# BigDecimal class.Answered | 4 Replies | 3101 Views | Created by Oscarfh - Wednesday, January 14, 2009 12:04 PM | Last reply by Oscarfh - Wednesday, January 14, 2009 2:51 PM -
0 Votes
tabletable.compute("Var()", filter ) produces wrong result
Just a gues, what result value type are you using, that we don't see? That it does things half does not mean that the answer from Martin is incorrect, there ...Proposed | 7 Replies | 4297 Views | Created by topcan5 - Thursday, May 14, 2009 11:54 PM | Last reply by topcan5 - Wednesday, May 27, 2009 2:48 PM -
0 Votes
floating point control register
I am having a problem with floating point exceptions after creating an instance of a COM object within in a Windows form application. The result of the ... -
0 Votes
Floating point precision
How do I find out the floating point precision of my machine?Answered | 1 Replies | 2820 Views | Created by mpt_fz - Wednesday, July 29, 2009 5:25 PM | Last reply by Reed Copsey, Jr - Wednesday, July 29, 2009 5:34 PM -
0 Votes
Wrong query result
=20 The red filters satisfy the result ...Answered | 1 Replies | 6114 Views | Created by ardmore - Wednesday, April 16, 2008 3:35 PM | Last reply by Kalman Toth - Wednesday, April 16, 2008 3:47 PM -
0 Votes
Problem in CEILING in some floating point numbers
hi, The following script provide incorrect output. This problem occurs only in 2 type numbers. (1) 3 comes after decimal ...Answered | 8 Replies | 529 Views | Created by Anees A - Thursday, May 17, 2018 4:35 AM | Last reply by Anees A - Thursday, May 17, 2018 2:39 PM -
0 Votes
Wrong Result Summation in Query
How to calculate Quantity in 2 Table which have Same ItemID WithOut Wrong calculateAnswered | 10 Replies | 7608 Views | Created by Ahmed_Soft3 - Monday, July 12, 2010 2:31 AM | Last reply by Tom Li - MSFT - Friday, December 10, 2010 2:46 AM - Items 1 to 20 of 2771147 Next ›
No announcements
