Talk:Benchmark

From OWASP
Revision as of 20:34, 13 July 2016 by Eelgheez (talk | contribs)

Jump to: navigation, search

The meaning of the diagonal

I don't think it's fair to call the diagonal line in the FPR/TPR chart a "random guess" line. The FPR == TPR equation translates to FP/(FP+TN) == TP/(TP+FN), meaning FP*FN == TN*TP, or FP/TP == TN/FN. The FPR > TPR area below the line does not put the tool into a "worse than guessing" shame list. The last equation suggests a different interpretation of that area, "the noise rate in reporting suspects exceeds the silence rate about non-issues".

The "worse than guessing" interpretation seems to come from the following scenario. We have n real and m fake vulnerabilities. For each of these vulnerabilities let the tool (or a monkey) decide if it is real. I guess this scenario ignores that the tool does not get the list of these vulnerabilities as its input. --Eelgheez (talk) 20:24, 13 July 2016 (CDT)