incorrect Kendall's tau p-value

[Peter]

With statistics::Kendall I get the following result Score: 0.260358 One-sided P: 0.447281 Two-sided P: 0.894562

whereas via ​https://astatsa.com/CorrelationTest I get Kendall's rank correlation sample estimate τ= : 0.249107 alternate hypothesis: true τ ≠0 z -statistic: 4.11244903545 p-value: 0.000039

The difference between score and tau is acceptable, but something is wrong for the p-value.

[Peter]

data

[Peter]

In 3972:

add test data for statistics::Kendall (refs #955)

[Peter]

In 3973:

add dependency for lazycheck (refs #955)

[Peter]

The problem is that the wrong formula is used in the correction of variance.

There is a term counting triplets

\sum t(t-1)(t-2) * \sum u(u-1)(u-2)

which should be divided by 9n(n-1)(n-2). Not doing that overestimates the variance and since that factor growths like n cubed, the error becomes disastrous for large n.

[Peter]

Also, note that the term above is only non-zero when there is at least one triplet in both x and y (i.e. one x-value is populated by at least three data points and the same for y). If there are not ties in both variables or the tie is only between two data points, this term is zero and the bug does not come into play.

[Peter]

In 3975:

fixes #955.

The correction term involving count_triple() was not divided with the
correct factor (9n(n-1)(n-2)) resulting in overestimated variance and
thus overestimated p-values. On top of changing that single line, also
rewriting the most of the function to make it easier to read.