Package mvpa :: Package clfs :: Module smlr :: Class SMLR
[hide private]
[frames] | no frames]

Class SMLR

source code


Sparse Multinomial Logistic Regression Classifier.

This is an implementation of the SMLR algorithm published in Krishnapuram et al. (2005, IEEE Transactions on Pattern Analysis and Machine Intelligence). Be sure to cite that article if you use this for your work.

Nested Classes [hide private]

Inherited from misc.state.Stateful: __metaclass__

Instance Methods [hide private]
 
__init__(self, **kwargs)
Initialize an SMLR classifier.
source code
 
_pythonStepwiseRegression(self, w, X, XY, Xw, E, auto_corr, lambda_over_2_auto_corr, S, M, maxiter, convergence_tol, resamp_decay, min_resamp, verbose, seed=None)
The (much slower) python version of the stepwise regression.
source code
 
_train(self, dataset)
Train the classifier using dataset (Dataset).
source code
 
_getFeatureIds(self)
Virtual method to return feature_ids used while training
source code
 
_predict(self, data)
Predict the output for the provided data.
source code
 
getSensitivityAnalyzer(self, **kwargs)
Returns a sensitivity analyzer for SMLR.
source code

Inherited from base.Classifier: __str__, isTrained, predict, regression, train, train2predict, trained, untrain

Inherited from misc.state.Parametrized: __repr__

Inherited from misc.state.Stateful: __getattribute__, __setattr__, reset

Inherited from object: __delattr__, __hash__, __new__, __reduce__, __reduce_ex__

Class Variables [hide private]
  _clf_internals = ['smlr', 'linear', 'has_sensitivity', 'multic...
Describes some specifics about the classifier -- is that it is doing regression for instance....
  lm = Parameter(.1, min= 1e-10, allowedtype= 'float', doc= """T...
  convergence_tol = Parameter(1e-3, min= 1e-10, max= 1.0, allowe...
  resamp_decay = Parameter(0.5, allowedtype= 'float', min= 0.0, ...
  min_resamp = Parameter(0.001, allowedtype= 'float', min= 1e-10...
  maxiter = Parameter(10000, allowedtype= 'int', min= 1, doc= ""...
  has_bias = Parameter(True, allowedtype= 'bool', doc= """Whethe...
  fit_all_weights = Parameter(False, allowedtype= 'bool', doc= "...
  implementation = Parameter("C", allowedtype= 'basestring', cho...
  seed = Parameter(None, allowedtype= 'None or int', doc= """See...
  biases = property(lambda self: self.__biases)
  weights = property(lambda self: self.__weights)

Inherited from base.Classifier: feature_ids, predicting_time, predictions, retrainable, trained_dataset, trained_labels, training_confusion, training_time, values

Instance Variables [hide private]
  __ulabels
Unigue labels from the training set.
  __weights_all
Contains all weights including bias values
  __weights
Just the weights, without the biases
  __biases
The biases, will remain none if has_bias is False

Inherited from base.Classifier (private): _regression, _train2predict

Properties [hide private]

Inherited from misc.state.Stateful: descr

Inherited from object: __class__

Method Details [hide private]

__init__(self, **kwargs)
(Constructor)

source code 
Initialize an SMLR classifier.
Parameters:
  • enable_states - Names of the state variables which should be enabled additionally to default ones
  • disable_states - Names of the state variables which should be disabled
  • descr - Description of the instance
Overrides: object.__init__

_pythonStepwiseRegression(self, w, X, XY, Xw, E, auto_corr, lambda_over_2_auto_corr, S, M, maxiter, convergence_tol, resamp_decay, min_resamp, verbose, seed=None)

source code 
The (much slower) python version of the stepwise regression. I'm keeping this around for now so that we can compare results.

_train(self, dataset)

source code 
Train the classifier using dataset (Dataset).
Overrides: base.Classifier._train

_getFeatureIds(self)

source code 

Virtual method to return feature_ids used while training

Is not intended to be called anywhere but from _posttrain, thus classifier is assumed to be trained at this point

Overrides: base.Classifier._getFeatureIds
(inherited documentation)

_predict(self, data)

source code 
Predict the output for the provided data.
Overrides: base.Classifier._predict

getSensitivityAnalyzer(self, **kwargs)

source code 
Returns a sensitivity analyzer for SMLR.
Overrides: base.Classifier.getSensitivityAnalyzer

Class Variable Details [hide private]

_clf_internals

Describes some specifics about the classifier -- is that it is doing regression for instance....
Value:
['smlr', 'linear', 'has_sensitivity', 'multiclass', 'does_feature_sele\
ction']

lm

Value:
Parameter(.1, min= 1e-10, allowedtype= 'float', doc= """The penalty te\
rm lambda.  Larger values will give rise to
             more sparsification.""")

convergence_tol

Value:
Parameter(1e-3, min= 1e-10, max= 1.0, allowedtype= 'float', doc= """Wh\
en the weight change for each cycle drops below this value
             the regression is considered converged.  Smaller values
             lead to tighter convergence.""")

resamp_decay

Value:
Parameter(0.5, allowedtype= 'float', min= 0.0, max= 1.0, doc= """Rate \
of decay in the probability of resampling a zero weight.
             1.0 will immediately decrease to the min_resamp from 1.0,\
 0.0
             will never decrease from 1.0.""")

min_resamp

Value:
Parameter(0.001, allowedtype= 'float', min= 1e-10, max= 1.0, doc= "Min\
imum resampling probability for zeroed weights")

maxiter

Value:
Parameter(10000, allowedtype= 'int', min= 1, doc= """Maximum number of\
 iterations before stopping if not
             converged.""")

has_bias

Value:
Parameter(True, allowedtype= 'bool', doc= """Whether to add a bias ter\
m to allow fits to data not through
             zero""")

fit_all_weights

Value:
Parameter(False, allowedtype= 'bool', doc= """Whether to fit weights f\
or all classes or to the number of
            classes minus one.  Both should give nearly identical resu\
lts, but
            if you set fit_all_weights to True it will take a little l\
onger
            and yield weights that are fully analyzable for each class\
.  Also,
...

implementation

Value:
Parameter("C", allowedtype= 'basestring', choices= ["C", "Python"], do\
c= """Use C or Python as the implementation of
             stepwise_regression. C version brings significant speedup\
 thus is
             the default one.""")

seed

Value:
Parameter(None, allowedtype= 'None or int', doc= """Seed to be used to\
 initialize random generator, might be
             used to replicate the run""")