In numpy sum documentation https://numpy.org/doc/stable/reference/generated/numpy.sum.html
It mentions that instead of naive summation it use pairwise summation for better error rate
I am confused on why pairwise summation is better for reducing error
Naive Summation
In naive summation, numbers are added sequentially giving average error of O(sqrt(N)) and worst case O(N)
sum=(((x1+x2)+x3)+…+xn)Pairwise Summation has average error of O(log(sqrt(N))) and worst case of O(logN)
From google, pairwise improves on naive summation by dividing the list of numbers into pairs, summing each pair, and then recursively summing the results. The process is as follows:
Initial Pairing: Start with the original list of numbers.
{x1,x2,x3,x4,…,xn}
{x1,x2,x3,x4,…,xn}
Sum Pairs: Sum the numbers in pairs.
{(x1+x2),(x3+x4),…}{(x1+x2),(x3+x4),…}
Recursive Summation: Repeat the process with the sums from the previous step until a single sum remains.{((x1+x2)+(x3+x4)),…}{((x1+x2)+(x3+x4)),…}
Can someone explain the intuition on where is the improvement?
For example
Lets use the following number, say that the mantissa is 3
1.01x10^10 + 1.01x10-10 + 1.01x10^10 + 1.01x10-10Naive addition
(((1.01x10^10 + 1.01x10-10) + 1.01x10^10) + 1.01x10-10)(((1.01x10^10 + 1.01x10^10) + 1.01x10-10)(((2.02x10^10 + 1.01x10-10)2.02x10^10Pairwise summation
(1.01x10^10 + 1.01x10-10) + (1.01x10^10 + 1.01x10-10)1.01x10^10 + 1.01x10^10 2.02x10^10Both result in 2.02x10^10