Opened 16 years ago

Closed 11 years ago

# Tie in ROC

Reported by: Owned by: Peter Peter minor yat 0.8 statistics trunk

### Description

Behaviour when ties are present appears to be random.

### comment:1 Changed 14 years ago by Peter

Ties for the score is taking care of in r998. The p-value was not modified. We should check in some text-book what is standard.

### comment:2 Changed 12 years ago by Peter

Hollander and Wolfe say that exact p-value is calculating by going through all permutations (ch 4.1 comment 5). That seems a naive too me; there gotta be a way to trim down the tree (similar to in the no-tie case).

### comment:3 Changed 11 years ago by Peter

This code

```theplu::yat::statistics::ROC roc;
for (size_t i=0; i<20; ++i)
cout << roc.area() << "\n";
```

outputs 0 when I expected 0.5 since as all values are tied.

### comment:4 Changed 11 years ago by Peter

Milestone: yat 0.x+ → yat 0.8

### comment:5 Changed 11 years ago by Peter

Status: new → assigned

### comment:6 Changed 11 years ago by Peter

(In [2549]) ROC: adding a test for ties case. refs #144

### comment:7 Changed 11 years ago by Peter

(In [2551]) refs #144. Fix ROC::area for the tied case

### comment:8 Changed 11 years ago by Peter

In the large sample approximation the mean of the null distribution is unaffected of ties but the variance is modified to mn(N+1)/12-[mn/(12N(N-1)) * sum(t(t-1)(t+1))] where the last sum runs over tied groups and t is the size of the group.

### comment:9 Changed 11 years ago by Peter

I noticed that in presence of ties the distribution is no longer symmetric, which means Ptwo-sided does not simply equal 2*Pone-sided. Instead in the in the two-tailed case we need to sum over both tails. In the large sample approximation there is no need as we already approximate the distribution of score/area with a Gaussian (which is symmetric).

### comment:10 Changed 11 years ago by Peter

(In [2556]) ROC: implement correction for ties in large sample approximation o p-value. refs #144.

### comment:11 Changed 11 years ago by Peter

Resolution: → fixed assigned → closed

(In [2585]) handle ties in exact calculation of ROC p-value. fixes #144

### comment:12 Changed 11 years ago by Peter

Resolution: fixed closed → reopened

Needs more docs. Docs for function `ROC::p_value` is, for example, not accurate.

### comment:13 Changed 11 years ago by Peter

Resolution: → fixed reopened → closed

(In [2594]) improve docs for ROC and sister class AUC. closes #144

Note: See TracTickets for help on using tickets.