Opened 13 years ago

Closed 13 years ago

#437 closed request (fixed)

Rank Normalization

Reported by: Peter Owned by: Peter
Priority: major Milestone: yat 0.5
Component: normalizer Version: trunk
Keywords: Cc:

Description (last modified by Peter)

needs #439

This a normalization that works on 1D containers, for example, a Vector.

The short story is to replace each value with its rank, i.e., the smallest value is replaced with 0, the next smallest is replaced with 1/N (where N is number of elements in the Container), et cetera.

In the weighted case, the rank naturally translates to sum(w_i)/sum(w) where the first sum runs over elements for which x_i<x. Possibly one could add half the values of the weights corresponding to x_i=x just for symmetry reasons (and make the corresponding modification in the unweighted case)

Change History (9)

comment:1 Changed 13 years ago by Peter

Milestone: yat 0.x+yat 0.5
Status: newassigned

well I actually need this now... I'll call the class normalization::Spearman

comment:2 Changed 13 years ago by Peter

Description: modified (diff)

comment:3 Changed 13 years ago by Peter

(In [1496]) unweighted version of the Spearman Normalization. refs #437.

comment:4 Changed 13 years ago by Peter

(In [1507]) preparing for weighted version - refs #437

comment:5 Changed 13 years ago by Peter

(In [1508]) refs #437

comment:6 Changed 13 years ago by Peter

Resolution: fixed
Status: assignedclosed

(In [1509]) fixes #437 and added support for having known issues in test::Suite

comment:7 Changed 13 years ago by Peter

Resolution: fixed
Status: closedreopened

I think we should add half the equal weights as mentioned in description. This will cost some speed, but I think it is worth it. See this weighted example in which the NaNs? have w=0

1  2
2  NaN
4  NaN

that currently is transformed to

0    0
1/3  NaN
2/3  NaN

but with in modified version we will get

1/6  1/2
1/2  NaN
5/6  NaN

which I think makes more sense, because the only value in second column should not be biased to lower (or upper) end.

comment:8 Changed 13 years ago by Peter

(In [1511]) refs #437 modified test to reflect new spec

comment:9 Changed 13 years ago by Peter

Resolution: fixed
Status: reopenedclosed

fixed in [1512]

Note: See TracTickets for help on using tickets.