## #211 closed defect (fixed)

# weighted ROC p-value does not equal permutation test p-value?

Reported by: | Peter | Owned by: | Peter |
---|---|---|---|

Priority: | minor | Milestone: | yat 0.9 |

Component: | statistics | Version: | trunk |

Keywords: | Cc: |

### Description

The title might not be true but it is not straight forward to work out.

When having binary weights i.e. all non-zero weight are equal, then this should not be a problem. Work out what happens when this is not the case? Calculating the p-value exactly and approximately are two different stories, and need to be examined separately.

In both cases examine whether p-value is equal to permutation p-value. In the approximative case it is of course not identical but check if some correction should be done. In the exact case check if the p-value equals permutation p-value, if not try to modify implementation.

### Change History (7)

### comment:1 Changed 12 years ago by

Milestone: | yat 0.x+ → yat 0.9 |
---|---|

Status: | new → assigned |

### comment:2 Changed 12 years ago by

### comment:4 Changed 12 years ago by

### comment:6 Changed 11 years ago by

Resolution: | → fixed |
---|---|

Status: | assigned → closed |

**Note:**See TracTickets for help on using tickets.

Just to clarify: when we say permute in a weighted context, it means weight and class label stay together and data values are permuted. This imnplies that sum w over one class is const wrt permutations and that missing values (w=0) are ignored.