public class SimpleEvaluator extends Object
Simplified Java API to train-test evaluation. The train-test evaluator is somewhat difficult to use directly from Java; this class is intended to make it easier.
Constructor and Description |
---|
SimpleEvaluator()
Create a simple evaluator with a custom configuration.
|
Modifier and Type | Method and Description |
---|---|
SimpleEvaluator |
addAlgorithm(AlgorithmInstance algo)
Adds an algorithmInfo to the experiment being built.
|
SimpleEvaluator |
addAlgorithm(String name,
LenskitConfiguration config)
An algorithm instance constructed with a name and Lenskit configuration
|
SimpleEvaluator |
addDataSet(Crossfolder cross)
Adds a crossfolder’s results to the experiment.
|
SimpleEvaluator |
addDataSet(DataSet data)
Adds a single
DataSet to the TrainTestExperiment . |
SimpleEvaluator |
addDataSet(StaticDataSource source,
int partitions)
Add a new data set to be cross-folded.
|
SimpleEvaluator |
addDataSet(StaticDataSource source,
int partitions,
double holdout)
Add a new data set to be cross-folded.
|
SimpleEvaluator |
addDataSet(StaticDataSource train,
StaticDataSource test)
Add a data set to the experiment.
|
SimpleEvaluator |
addDataSet(String name,
StaticDataSource source,
int partitions)
Add a new data set to be cross-folded.
|
SimpleEvaluator |
addDataSet(String name,
StaticDataSource source,
int partitions,
double holdout)
Add a new data set to be cross-folded.
|
SimpleEvaluator |
addMetric(PredictMetric<?> metric)
Add a metric to the experiment.
|
Table |
execute()
If this is called more than once it will call of these commands again and most likely throw an exception.
|
TrainTestExperiment |
getExperiment()
Provides raw unrestricted access to the experiment.
|
Path |
getWorkDir()
Get the working directory for the evaluator.
|
SimpleEvaluator |
setOutput(Path file)
Set an output file for aggregate metrics.
|
SimpleEvaluator |
setUserOutput(Path file)
Set an output file for per-user evaluation metrics.
|
SimpleEvaluator |
setWorkDir(Path dir)
Set the working directory for the evaluator.
|
public SimpleEvaluator()
Create a simple evaluator with a custom configuration.
public Path getWorkDir()
Get the working directory for the evaluator.
public SimpleEvaluator setWorkDir(Path dir)
Set the working directory for the evaluator.
dir
- The directory in which the evaluator will save its output and temporary files.public SimpleEvaluator addAlgorithm(AlgorithmInstance algo)
Adds an algorithmInfo to the experiment being built.
If any exception is thrown while the command is called it is rethrown as a runtime error.
algo
- The algorithm to add to the experiment.public SimpleEvaluator addAlgorithm(String name, LenskitConfiguration config)
An algorithm instance constructed with a name and Lenskit configuration
name
- The name of the algorithm.config
- Lenskit configurationpublic SimpleEvaluator addDataSet(Crossfolder cross)
Adds a crossfolder’s results to the experiment.
cross
- The crossfold task.public SimpleEvaluator addDataSet(String name, StaticDataSource source, int partitions, double holdout)
Add a new data set to be cross-folded. This method creates a new Crossfolder
and passes it to addDataSet(Crossfolder)
. All crossfold parameters that are not taken as arguments by this method are left at their defaults.
name
- The name of the crossfoldsource
- The source for the crossfoldpartitions
- The number of partitionsholdout
- The holdout fractionpublic SimpleEvaluator addDataSet(StaticDataSource source, int partitions, double holdout)
Add a new data set to be cross-folded. This method creates a new Crossfolder
and passes it to addDataSet(Crossfolder)
. All crossfold parameters that are not taken as arguments by this method are left at their defaults.
source
- The source for the crossfoldpartitions
- The number of partitionsholdout
- The holdout fractionpublic SimpleEvaluator addDataSet(String name, StaticDataSource source, int partitions)
Add a new data set to be cross-folded. This method creates a new Crossfolder
and passes it to addDataSet(Crossfolder)
. All crossfold parameters that are not taken as arguments by this method are left at their defaults.
Note: Prior to LensKit 2.2, this method used a holdout fraction of 0.2. In LensKit 2.2, it was changed to use the Crossfolder
’s default holdout.
name
- The name of the crossfoldsource
- The source for the crossfoldpartitions
- The number of partitionspublic SimpleEvaluator addDataSet(StaticDataSource source, int partitions)
Add a new data set to be cross-folded. This method creates a new Crossfolder
and passes it to addDataSet(Crossfolder)
. All crossfold parameters that are not taken as arguments by this method are left at their defaults.
Note: Prior to LensKit 2.2, this method used a holdout fraction of 0.2. In LensKit 2.2, it was changed to use the Crossfolder
’s default holdout.
source
- The source for the crossfoldpartitions
- The number of partitionspublic SimpleEvaluator addDataSet(DataSet data)
Adds a single DataSet
to the TrainTestExperiment
.
data
- The data set to be added to the command.public SimpleEvaluator addDataSet(StaticDataSource train, StaticDataSource test)
Add a data set to the experiment.
The name for the data source will default to ‘generic-data-source’. Because of this, be careful of calling this method more than once.
train
- The source of training data.test
- The source of test data.public SimpleEvaluator addMetric(PredictMetric<?> metric)
Add a metric to the experiment.
metric
- The metric to be added.public SimpleEvaluator setOutput(Path file)
Set an output file for aggregate metrics.
file
- An output file for aggregate metrics.public SimpleEvaluator setUserOutput(Path file)
Set an output file for per-user evaluation metrics.
file
- A file to receive per-user evaluation metrics.public TrainTestExperiment getExperiment()
Provides raw unrestricted access to the experiment.
public Table execute()
If this is called more than once it will call of these commands again and most likely throw an exception.