[MRG] Add drop_intermediate kwarg to metrics.precision_recall_curve
#24668
Conversation
| # with the same tps value have the same recall and thus x coordinate. | ||
| # They appear as a vertical line on the plot. | ||
| optimal_idxs = np.where( | ||
| np.r_[True, np.logical_or(np.diff(tps[:-1]), np.diff(tps[1:])), True] |
There was a problem hiding this comment.
For my education: why does taking the "second derivative" in roc_curve work, but here it doesn't?
There was a problem hiding this comment.
More precisely, why does using the second derivative work for roc_curve? What we are looking for is two (or more) points where there is no change, so the first derivative seems like the natural thing to use :-/
There was a problem hiding this comment.
Thinking about this some more, could we use np.r_[True, np.diff(tps, 2), True] instead?
For tps = [1, 2, 3, 3, 3, 5, 6] we'd get [1, 3, 3, 5, 6] (the two gets dropped because its is on the line between 1 and 3. For tps = [1,2.1,3,3,3,5,6] we get [1., 2.1, 3., 3., 5., 6.].
I guess for plotting purposes it is fine to remove the 2?! Is there a reason to have different behaviour regarding the removal of points in roc_curve and this (with np.logical_or(np.diff(tps[:-1]), np.diff(tps[1:])) the 2 is kept)?
There was a problem hiding this comment.
The difference is that both axes of an ROC curve have constant denominators:
fpr = fps / fps[-1](linearly correlated withfps)tpr = tps / tps[-1](linearly correlated withtps)
By contrast, precision has a non-constant denominator (note that recall = tpr):
precision = tps / (tps + fps)(not linearly correlated with eithertpsorfps)
If you extend your example by one to tps = [1, 2, 3, 3, 3, 5, 6, 7], then you will get:
tps = [1, 3, 3, 3, 5, 6, 7]
fps = [0, 0, 1, 2, 2, 2, 2]
tpr = [1/7, 3/7, 3/7, 3/7, 5/7, 6/7, 7/7]
fpr = [0/2, 0/2, 1/2, 2/2, 2/2, 2/2, 2/2]
precision = [1/1, 3/3, 3/4, 3/5, 5/7, 6/8, 7/9]
np.r_[True, np.logical_or(np.diff(tps[:-1]), np.diff(tps[1:])), True] results in [1, 3, 3, 5, 6, 7].
np.r_[True, np.diff(tps, 2), True] results in [1, 3, 3, 5, 7].
The second method incorrectly drops the 6, which is not actually on a line in the precision-recall curve:
There was a problem hiding this comment.
Today I learnt! Thanks for taking the time to explain it
Not sure if this question is to me? I'm not sure what justifies a "what's new" entry but happy to provide one if needed. |
It was aimed at someone "in the know", because I also don't know the inclusion criteria. |
Reference Issues/PRs
Fixes #21825
What does this implement/fix? Explain your changes.
Adds a
drop_intermediatekwarg tometrics.precision_recall_curvesimilar to the one that already exists formetrics.roc_curve. This removes unnecessary points on the curve to reduce its size.