Infragistics Home

Infragistics Forums

Infragistics community online discussions.
Welcome to Infragistics Forums Sign in | FAQ
in Search

Unexpected result for SUM function

Last post 06-27-2008 8:59 by [Infragistics] Mike Saltzman. 1 replies.
Page 1 of 1 (2 items)
Sort Posts: Previous Next
  • 06-26-2008 10:21

    Unexpected result for SUM function

    Hi,

    I´m working on a spreadsheet like application where users can create its formulas just by typing it in a formula bar, according to our sintax and application data context. Recently, a customer has reported an error (as he sad) in the following formula:

    SUM(A2:A10) , where values for A2 to A10 are: 3287.87, 502.72, 354.83, 857.21, 266.52, 33.32, 83.29, 43.37 and -5429.13

    In this case, we just replace the text "A2:A10" by its values, resulting in "SUM(3287,87;502,72;354,83;857,21;266,52;33,32;83,29;43,37;-5429,13)" expression and then call the UltraCalcManager.Calculate method getting the unexpected value -9,09494701772928E-13.

    I´m aware that this is a double data type imprecision issue but unfortunately I can´t use this as an argumente with my customer. And to make it worst, when he does the same in the MS Excel, he gets the expected result for formula SUM(A2:A10). It seems they do some kind of work around to avoid the inprecise result value.

    Any suggestion?

     Att, Clodoaldo.

    • Post Points: 20
  • 06-27-2008 8:59 In reply to

    Re: Unexpected result for SUM function

    Answer

    What result are you getting from Excel that is different from the CalcManager resut? When I try this formula, I get exactly the same result in both Excel and CalcManager:

    By default, Excel will format the number differently and hide some of the decimal places, but if you apply a number format to the cell so that it shows all of the digits, it returns exactly the same result as the CalcManager does. 

    Mike Saltzman
    R&D Engineer
    Infragistics, Inc.
    • Post Points: 5
Page 1 of 1 (2 items)
Powered by Community Server (Commercial Edition), by Telligent Systems