broken link to A.Neumaier article in built-in sum comment

[enzbus]

Bug report

Bug description:

The new implementation of sum on Python 3.12 (cfr. #100425 , #100426 , #107785 ) is not associative on simple input values. This minimal code shows the bug:

On Python 3.11:

>>> a = [0.1, -0.2, 0.3, -0.4, 0.5]
>>> a.append(-sum(a))
>>> sum(a) == 0
True

On Python 3.12:

>>> a = [0.1, -0.2, 0.3, -0.4, 0.5]
>>> a.append(-sum(a))
>>> sum(a) == 0
False

I'm sure this affects more users than the "improved numerical accuracy" on badly scaled input data which most users don't ever deal with, and for which exact arithmetic is already available in the Standard Library
-> https://docs.python.org/3/library/decimal.html.

I'm surprised this low-level change was accepted with so little review. There are other red flags connected with this change:

• The link to the new algorithm's description in cPython's official code is dead ->

  cpython/Python/bltinmodule.c

  Line 2614 in 289af86

  // https://www.mat.univie.ac.at/~neum/scan/01.pdf

• The researcher to which the algorithm is credited has an empty academic page, with no PDFs -> https://www.mat.univie.ac.at/~neum/

Is anybody interested in keeping the quality of cPython's codebase high? When I learned Python, I remember one of the first thing in the official tutorial was that Python is a handy calculator, and now to me it seems broken. @gvanrossum ?

CPython versions tested on:

3.12

Operating systems tested on:

Linux

Linked PRs

• gh-111933: fix broken link to A.Neumaier article #111937
• [3.12] gh-111933: fix broken link to A.Neumaier article (gh-111937) #111993

[skirpichev]

The researcher to which the algorithm is credited has an empty academic page, with no PDFs -> https://www.mat.univie.ac.at/~neum/

FYI, citation: Neumaier, A. (1974), Rundungsfehleranalyse einiger Verfahren zur Summation endlicher Summen. Z. angew. Math. Mech., 54: 39-51. https://doi.org/10.1002/zamm.19740540106.

Full text is still available in the archive: https://web.archive.org/web/20220804051351/https://www.mat.univie.ac.at/~neum/scan/01.pdf Should we use this url or above citation (no free text access)?

BTW, I doubt there is an issue, besides a broken link. Associativity is broken for addition of floating point numbers, see e.g. TAOCP, Vol. 2, 4.2.2. The Tutorial has a dedicated section for floating point arithmetic, which cover this "issue".

exact arithmetic is already available in the Standard Library -> https://docs.python.org/3/library/decimal.html

The decimal module actually not about exact arithmetics, rather the fractions module.

[Eclips4]

cc @rhettinger

[terryjreedy]

One might naively expect sum(a) to be .9 -.6 = .3. But it is actually 0.29999999999999993, so the error is about 7e-17. The sum(a2) is about 3e-17, so the absolute error has gone down. 0.0 error is something of an accident.

Now add -.3 to list a and sum. Instead of the naively expected 0.0, I got -5.6e-17 in 3.11 and -2.8e-17 in 3.12 (both rounded to 2 digits), which is half the error. This is at least as realistic example than the original one and has nothing to do with badly scaled data.

@enzbus In the future, please first ask about possible bugs on help forums such as https://discuss.python.org/c/users/7. And please omit disparaging comments wherever you post.

[sweeneyde]

You also don't have to modify the example too much to see a case where the accuracy of the sum function improved with the change:

>>> # Python 3.11
>>> sum([0.1, -0.2, 0.3, -0.4, 0.5, -0.1, 0.2, -0.3, 0.4, -0.5])
-5.551115123125783e-17

>>> # Python 3.12
>>> sum([0.1, -0.2, 0.3, -0.4, 0.5, -0.1, 0.2, -0.3, 0.4, -0.5])
0.0

[enzbus]

I'm actually getting the error on random inputs, see the commit above which references this bug report. The point remains, this is a regression, adds little value (as I explain above), is the implementation of a 1974 paper in german (really?) which is not online any more, and should not have happened.

[enzbus]

@terryjreedy please re-open, at least to see if this affects other users. Please note that there were no disparaging comments. I noted that the Standard Library already provides arbitrary precision arithmetic in the decimal module, this change refers a suspicious article in German which I couldn't even find (and I went on the academic webpage of the researcher). I think this should not have passed quality review, since this bug is real (I provided a minimal code to show it, but I encountered it on random inputs where the comparison is exact on python < 3.12), especially since it adds little value.

[skirpichev]

@terryjreedy, lets not forgot the real issue with the broken link, see PR #111937.