weighted ROC p-value does not equal permutation test p-value?

[Peter]

The title might not be true but it is not straight forward to work out.

When having binary weights i.e. all non-zero weight are equal, then this should not be a problem. Work out what happens when this is not the case? Calculating the p-value exactly and approximately are two different stories, and need to be examined separately.

In both cases examine whether p-value is equal to permutation p-value. In the approximative case it is of course not identical but check if some correction should be done. In the exact case check if the p-value equals permutation p-value, if not try to modify implementation.

[Peter]

Just to clarify: when we say permute in a weighted context, it means weight and class label stay together and data values are permuted. This imnplies that sum w over one class is const wrt permutations and that missing values (w=0) are ignored.

[Peter]

(In [2708]) tests for weighted ROC p-value. refs #211

[Peter]

(In [2709]) refs #211. Changed how number of samples, n, is calculated in large sample approximation p-value calculation. Before the sum_w was used and now sum(w)*sum(w)/sum(ww) is used.

[Peter]

(In [2712]) correct test. refs #211

[Peter]

(In [2718]) implement ROC p-value handling weights and ties. closes #211

[Peter]

(In [2722]) fix implementation of ROC::remove. refs #211