CPerformanceMeasures Class Reference

Inheritance diagram for CPerformanceMeasures:

Inheritance graph
[legend]

List of all members.


Detailed Description

class to implement various performance measures: Receiver Operating Curve (ROC), Precision Recall Curve (PRC), Detection Error Tradeoff (DET), Cross Correlation coefficient (CC), Weighted Relative Accuracy (WRAcc), Balanced Error (BAL)

based on: Fawcett, T: March 2004, ROC Graphs: Notes and Practical Considerations for Researchers Sonnenburg, S: Feburary 2008, various discussions

Definition at line 31 of file PerformanceMeasures.h.


Public Member Functions

 CPerformanceMeasures ()
 CPerformanceMeasures (CLabels *true_labels, CLabels *output)
virtual ~CPerformanceMeasures ()
void init (CLabels *true_labels, CLabels *output)
bool set_true_labels (CLabels *true_labels)
CLabelsget_true_labels () const
bool set_output (CLabels *output)
CLabelsget_output () const
INT get_num_labels () const
void get_ROC (DREAL **result, INT *num, INT *dim)
DREAL get_auROC ()
DREAL get_aoROC ()
void get_PRC (DREAL **result, INT *num, INT *dim)
DREAL get_auPRC ()
DREAL get_aoPRC ()
void get_DET (DREAL **result, INT *num, INT *dim)
DREAL get_auDET ()
DREAL get_aoDET ()
void get_all_accuracy (DREAL **result, INT *num, INT *dim)
DREAL get_accuracy (DREAL threshold=0)
void get_all_error (DREAL **result, INT *num, INT *dim)
DREAL get_error (DREAL threshold=0)
void get_all_fmeasure (DREAL **result, INT *num, INT *dim)
DREAL get_fmeasure (DREAL threshold=0)
void get_all_CC (DREAL **result, INT *num, INT *dim)
DREAL get_CC (DREAL threshold=0)
void get_all_WRAcc (DREAL **result, INT *num, INT *dim)
DREAL get_WRAcc (DREAL threshold=0)
void get_all_BAL (DREAL **result, INT *num, INT *dim)
DREAL get_BAL (DREAL threshold=0)

Static Public Attributes

static CParallel parallel
static CIO io
static CVersion version

Protected Member Functions

void init_nolabels ()
template<class T>
DREAL trapezoid_area (T x1, T x2, T y1, T y2)
void create_sortedROC ()
void compute_ROC (DREAL **result)
void compute_accuracy (DREAL **result, INT *num, INT *dim, bool do_error=false)
void compute_PRC (DREAL **result)
void compute_DET (DREAL **result)
void compute_confusion_matrix (DREAL threshold, INT *tp, INT *fp, INT *fn, INT *tn)

Protected Attributes

CLabelsm_true_labels
CLabelsm_output
INT m_num_labels
INT m_all_true
INT m_all_false
INTm_sortedROC
DREAL m_auROC
DREAL m_auPRC
DREAL m_auDET

Constructor & Destructor Documentation

CPerformanceMeasures::CPerformanceMeasures (  ) 

default constructor

Definition at line 16 of file PerformanceMeasures.cpp.

CPerformanceMeasures::CPerformanceMeasures ( CLabels true_labels,
CLabels output 
)

constructor

Parameters:
true_labels true labels as seen in real world
output output labels/hypothesis from a classifier

Definition at line 22 of file PerformanceMeasures.cpp.

CPerformanceMeasures::~CPerformanceMeasures (  )  [virtual]

Definition at line 29 of file PerformanceMeasures.cpp.


Member Function Documentation

void CPerformanceMeasures::init ( CLabels true_labels,
CLabels output 
)

initialise performance measures

Parameters:
true_labels true labels as seen in real world
output output labels/hypothesis from a classifier

Definition at line 53 of file PerformanceMeasures.cpp.

bool CPerformanceMeasures::set_true_labels ( CLabels true_labels  ) 

set true labels as seen in real world

Parameters:
true_labels true labels
Returns:
if setting was successful

Definition at line 58 of file PerformanceMeasures.h.

CLabels* CPerformanceMeasures::get_true_labels (  )  const

get true labels as seen in real world

Returns:
true labels as seen in real world

Definition at line 69 of file PerformanceMeasures.h.

bool CPerformanceMeasures::set_output ( CLabels output  ) 

set output labels/hypothesis from a classifier

Parameters:
output output labels
Returns:
if setting was successful

Definition at line 76 of file PerformanceMeasures.h.

CLabels* CPerformanceMeasures::get_output (  )  const

get classifier's output labels/hypothesis

Returns:
output labels

Definition at line 87 of file PerformanceMeasures.h.

INT CPerformanceMeasures::get_num_labels (  )  const

get number of labels in output/true labels

Returns:
number of labels in output/true labels

Definition at line 93 of file PerformanceMeasures.h.

void CPerformanceMeasures::get_ROC ( DREAL **  result,
INT num,
INT dim 
)

get Receiver Operating Curve for previously given labels (swig compatible)

ROC point = false positives / all false labels, true positives / all true labels

caller has to free

Parameters:
result where computed ROC values will be stored
num number of labels/examples
dim dimensions == 2 (false positive rate, true positive rate)

Definition at line 191 of file PerformanceMeasures.cpp.

DREAL CPerformanceMeasures::get_auROC (  ) 

return area under Receiver Operating Curve

calculated by adding trapezoids

Returns:
area under ROC

Definition at line 115 of file PerformanceMeasures.h.

DREAL CPerformanceMeasures::get_aoROC (  ) 

return area over Reveiver Operating Curve

value is 1 - auROC

Returns:
area over ROC

Definition at line 132 of file PerformanceMeasures.h.

void CPerformanceMeasures::get_PRC ( DREAL **  result,
INT num,
INT dim 
)

get Precision Recall Curve for previously given labels (swig compatible)

PRC point = true positives / all true labels, true positives / (true positives + false positives)

caller has to free

Parameters:
result where computed ROC values will be stored
num number of labels/examples
dim dimension == 2 (recall, precision)

Definition at line 263 of file PerformanceMeasures.cpp.

DREAL CPerformanceMeasures::get_auPRC (  ) 

return area under Precision Recall Curve

calculated by adding trapezoids

Returns:
area under PRC

Definition at line 157 of file PerformanceMeasures.h.

DREAL CPerformanceMeasures::get_aoPRC (  ) 

return area over Precision Recall Curve

value is 1 - auPRC

Returns:
area over PRC

Definition at line 174 of file PerformanceMeasures.h.

void CPerformanceMeasures::get_DET ( DREAL **  result,
INT num,
INT dim 
)

get Detection Error Tradeoff curve for previously given labels (swig compatible)

DET point = false positives / all false labels, false negatives / all false labels

caller has to free

Parameters:
result where computed DET values will be stored
num number of labels/examples
dim dimension == 2 (false positive rate, false negative rate)

Definition at line 311 of file PerformanceMeasures.cpp.

DREAL CPerformanceMeasures::get_auDET (  ) 

return area under Detection Error Tradeoff curve

calculated by adding trapezoids

Returns:
area under DET curve

Definition at line 199 of file PerformanceMeasures.h.

DREAL CPerformanceMeasures::get_aoDET (  ) 

return area over Detection Error Tradeoff curve

value is 1 - auDET

Returns:
area over DET curve

Definition at line 216 of file PerformanceMeasures.h.

void CPerformanceMeasures::get_all_accuracy ( DREAL **  result,
INT num,
INT dim 
)

get classifier's accuracies (swig compatible)

accuracy = (true positives + true negatives) / all labels

caller has to free

Parameters:
result storage of accuracies in 2 dim array: (output, accuracy), sorted by output
num number of accuracy points
dim dimension == 2 (output, accuracy)

Definition at line 399 of file PerformanceMeasures.cpp.

DREAL CPerformanceMeasures::get_accuracy ( DREAL  threshold = 0  ) 

get classifier's accuracy at given threshold

Parameters:
threshold all values below are considered negative examples (default 0)
Returns:
classifer's accuracy at threshold

Definition at line 359 of file PerformanceMeasures.cpp.

void CPerformanceMeasures::get_all_error ( DREAL **  result,
INT num,
INT dim 
)

get classifier's error rates (swig compatible)

value is 1 - accuracy

caller has to free

Parameters:
result storage of errors in 2 dim array: (output, error), sorted by output
num number of accuracy points
dim dimension == 2 (output, error)

Definition at line 404 of file PerformanceMeasures.cpp.

DREAL CPerformanceMeasures::get_error ( DREAL  threshold = 0  ) 

get classifier's error at threshold

value is 1 - accuracy0

Parameters:
threshold all values below are considered negative examples (default 0)
Returns:
classifer's error at threshold

Definition at line 263 of file PerformanceMeasures.h.

void CPerformanceMeasures::get_all_fmeasure ( DREAL **  result,
INT num,
INT dim 
)

get classifier's F-measure (swig compatible)

F-measure = 2 / (1 / precision + 1 / recall)

caller has to free

Parameters:
result storage of F-measure in 2 dim array (output, fmeasure), sorted by output
num number of accuracy points
dim dimension == 2 (output, fmeasure)

Definition at line 440 of file PerformanceMeasures.cpp.

DREAL CPerformanceMeasures::get_fmeasure ( DREAL  threshold = 0  ) 

get classifier's F-measure at threshold 0

Returns:
classifer's F-measure at threshold 0

Definition at line 411 of file PerformanceMeasures.cpp.

void CPerformanceMeasures::get_all_CC ( DREAL **  result,
INT num,
INT dim 
)

get classifier's Cross Correlation coefficients (swig compatible)

CC = ( true positives * true negatives

  • false positives * false negatives ) / sqrt( (true positives + false positives) * (true positives + false negatives) * (true negatives + false positives) * (true negatives + false negatives) )

also check http://en.wikipedia.org/wiki/Correlation

caller has to free

Parameters:
result storage of CCs in 2 dim array: (output, CC), sorted by output
num number of CC points
dim dimension == 2 (output, CC)

Definition at line 477 of file PerformanceMeasures.cpp.

DREAL CPerformanceMeasures::get_CC ( DREAL  threshold = 0  ) 

get classifier's Cross Correlation coefficient at threshold

Returns:
classifer's CC at threshold

Definition at line 463 of file PerformanceMeasures.cpp.

void CPerformanceMeasures::get_all_WRAcc ( DREAL **  result,
INT num,
INT dim 
)

get classifier's Weighted Relative Accuracy (swig compatible)

WRAcc = ( true positives / (true positives + false negatives) )

  • ( false positives / (false positives + true negatives) )

caller has to free

Parameters:
result storage of WRAccs in 2 dim array: (output, WRAcc), sorted by output
num number of WRAcc points
dim dimension == 2 (output, WRAcc)

Definition at line 523 of file PerformanceMeasures.cpp.

DREAL CPerformanceMeasures::get_WRAcc ( DREAL  threshold = 0  ) 

get classifier's Weighted Relative Accuracy at threshold 0

Returns:
classifer's WRAcc at threshold 0

Definition at line 504 of file PerformanceMeasures.cpp.

void CPerformanceMeasures::get_all_BAL ( DREAL **  result,
INT num,
INT dim 
)

get classifier's Balanced Error (swig compatible)

BAL = 0.5 * ( true positives / all true labels + true negatives / all false labels )

caller has to free

Parameters:
result storage of BAL in 2 dim array: (output, BAL), sorted by output
num number of BAL points
dim dimension == 2 (output, BAL)

Definition at line 566 of file PerformanceMeasures.cpp.

DREAL CPerformanceMeasures::get_BAL ( DREAL  threshold = 0  ) 

get classifier's Balanced Error at threshold 0

Returns:
classifer's BAL at threshold 0

Definition at line 550 of file PerformanceMeasures.cpp.

void CPerformanceMeasures::init_nolabels (  )  [protected]

initialise values independent from true labels/output

Definition at line 43 of file PerformanceMeasures.cpp.

template<class T>
DREAL CPerformanceMeasures::trapezoid_area ( x1,
x2,
y1,
y2 
) [protected]

calculate trapezoid area for auROC

Parameters:
x1 x coordinate of point 1
x2 x coordinate of point 2
y1 y coordinate of point 1
y2 y coordinate of point 2
Returns:
trapezoid area for auROC

Definition at line 133 of file PerformanceMeasures.cpp.

void CPerformanceMeasures::create_sortedROC (  )  [protected]

create index for ROC sorting

Definition at line 113 of file PerformanceMeasures.cpp.

void CPerformanceMeasures::compute_ROC ( DREAL **  result  )  [protected]

compute ROC points and auROC

Definition at line 198 of file PerformanceMeasures.cpp.

void CPerformanceMeasures::compute_accuracy ( DREAL **  result,
INT num,
INT dim,
bool  do_error = false 
) [protected]

compute ROC accuracy/error

Parameters:
result where the result will be stored
num number of points
dim dimension == 2 (output, accuracy/error)
do_error if error instead of accuracy shall be computed

Definition at line 371 of file PerformanceMeasures.cpp.

void CPerformanceMeasures::compute_PRC ( DREAL **  result  )  [protected]

compute PRC points and auPRC

Parameters:
result where the result will be stored

Definition at line 271 of file PerformanceMeasures.cpp.

void CPerformanceMeasures::compute_DET ( DREAL **  result  )  [protected]

compute DET points and auDET

Parameters:
result where the result will be stored

Definition at line 319 of file PerformanceMeasures.cpp.

void CPerformanceMeasures::compute_confusion_matrix ( DREAL  threshold,
INT tp,
INT fp,
INT fn,
INT tn 
) [protected]

compute confusion matrix

caller has to delete[]

Parameters:
threshold threshold to compute against
tp storage of true positives or NULL if unused
fp storage of false positives or NULL if unused
fn storage of false negatives or NULL if unused
tn storage of true negatives or NULL if unused

Definition at line 140 of file PerformanceMeasures.cpp.


Member Data Documentation

true labels/examples as seen in real world

Definition at line 432 of file PerformanceMeasures.h.

output labels/hypothesis from a classifier

Definition at line 434 of file PerformanceMeasures.h.

number of true labels/outputs/accuracies/ROC points

Definition at line 436 of file PerformanceMeasures.h.

number of positive examples in true_labels

Definition at line 439 of file PerformanceMeasures.h.

number of negative examples in true_labels

Definition at line 441 of file PerformanceMeasures.h.

array of size num_labels with indices of true_labels/output sorted to fit ROC algorithm

Definition at line 445 of file PerformanceMeasures.h.

area under ROC; 1 - area over ROC

Definition at line 447 of file PerformanceMeasures.h.

area under PRC; 1 - area over PRC

Definition at line 449 of file PerformanceMeasures.h.

area under DET; 1 - area over DET

Definition at line 451 of file PerformanceMeasures.h.

CParallel CSGObject::parallel [static, inherited]

Definition at line 105 of file SGObject.h.

CIO CSGObject::io [static, inherited]

Definition at line 106 of file SGObject.h.

CVersion CSGObject::version [static, inherited]

Definition at line 107 of file SGObject.h.


The documentation for this class was generated from the following files:

SHOGUN Machine Learning Toolbox - Documentation