onyx.util.rocutils – Utilities for generating ROC and DET curves

onyx.util.rocutils.make_DET_data(reference, ratios)

reference is a list of 0/1 values which are the correct classifications values is a parallel list of numeric values, with higher values intending to map toward classifications of 1.

Returns data for a DET curve in the form of a list of triples, where each triple contains the fraction of false positives as a percent, the fraction of false negatives, and the threshold value that generated those rates. The triples are ordered by threshold from lowest (fewest false positives) to highest (fewest misses)

Note that a typical DET curve would plot false_pos on the X axis and false_neg on the Y axis, oftentimes with a normal deviate scale.

>>> ref = [0,0,0,0,0,1,1,1,1,1]
>>> values = [2, 3, 4, 9, 4, 5, 6, 9, 9, 3]
>>> res = make_DET_data(ref, values)
>>> res
[(0.0, 100.0, 9), (20.0, 19.999999999999996, 4), (80.0, 0.0, 2)]

onyx.util.rocutils.make_ROC_data(reference, ratios)

reference is a list of 0/1 values which are the correct classifications values is a parallel list of numeric values, with higher values intending to map toward classifications of 1.

Returns data for a ROC curve in the form of a list of triples, where each triple contains an interesting threshold value, the fraction of correct identifications (true positives) as a percent, and the fraction of false positives, at that threshold. The triples are ordered by threshold from lowest (fewest false positives) to highest (most true positives)

Note that a typical ROC curve would plot false_pos on the X axis and true_pos on the Y axis using a linear scale.

>>> ref = [0,0,0,0,0,1,1,1,1,1]
>>> values = [2, 3, 4, 9, 4, 5, 6, 9, 9, 3]
>>> res = make_ROC_data(ref, values)
>>> res
[(0.0, 0.0, 9), (20.0, 80.0, 4), (80.0, 100.0, 2)]

onyx.util.rocutils.write_data_as_csv(data, stream, header_type='DET')

Write either ROC or DET data as comma-separated text, suitable for import into a spreadsheet or other tool. Writes DET header fields be default, use header_type of “ROC” or None for ROC headers or no headers, respectively.

>>> ref = [0,0,0,0,0,1,1,1,1,1]
>>> values = [2, 3, 4, 9, 4, 5, 6, 9, 9, 3]
>>> res = make_DET_data(ref, values)
>>> s = StringIO.StringIO()
>>> write_data_as_csv(res, s)
>>> out = s.getvalue()
>>> print out
False Alarm Rate,  Miss Rate,  Threshold
0.0,  100.0,  9
20.0,  20.0,  4
80.0,  0.0,  2
<BLANKLINE>
>>> s.seek(0)
>>> res = make_ROC_data(ref, values)
>>> write_data_as_csv(res, s, header_type="ROC")
>>> out = s.getvalue()
>>> print out
False Pos Rate, True Pos Rate, Threshold
0.0,  0.0,  9
20.0,  80.0,  4
80.0,  100.0,  2
<BLANKLINE>
>>> s.close()