org.lenskit.eval.traintest.predict

Class NDCGPredictMetric

java.lang.Object

org.lenskit.eval.traintest.metrics.Metric<X>

org.lenskit.eval.traintest.predict.PredictMetric<MeanAccumulator>

org.lenskit.eval.traintest.predict.NDCGPredictMetric

public class NDCGPredictMetric
extends PredictMetric<MeanAccumulator>

Evaluate a recommender’s predictions with normalized discounted cumulative gain.

This is a prediction evaluator that uses base-2 nDCG to evaluate recommender accuracy. The items are ordered by predicted preference and the nDCG is computed using the user’s real rating as the gain for each item. Doing this only over the queried items, rather than in the general recommend condition, avoids penalizing recommenders for recommending items that would be better if the user had known about them and provided ratings (e.g., for doing their job).

nDCG is computed per-user and then averaged over all users.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	NDCGPredictMetric.Spec Specification for configuring nDCG metrics.

Field Summary

Fields

Modifier and Type	Field and Description
static String	DEFAULT_COLUMN

Constructor Summary

Constructors

Constructor and Description
NDCGPredictMetric() Create a new log_2 nDCG metric with column name “Predict.nDCG”.
NDCGPredictMetric(Discount disc) Create a new nDCG metric with column name “Predict.nDCG”.
NDCGPredictMetric(Discount disc, String name) Create a new nDCG metric.
NDCGPredictMetric(NDCGPredictMetric.Spec spec) Construct a predict metric from a spec.

Method Summary

Modifier and Type	Method and Description
MeanAccumulator	createContext(AlgorithmInstance algorithm, DataSet dataSet, Recommender recommender) Create the context for an experimental condition (algorithm/data set pair).
MetricResult	getAggregateMeasurements(MeanAccumulator context) Get the aggregate results from an accumulator.
MetricResult	measureUser(TestUser user, ResultMap predictions, MeanAccumulator context) Measure a single result.

Methods inherited from class org.lenskit.eval.traintest.metrics.Metric

getAggregateColumnLabels, getColumnLabels, getRequiredRoots

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_COLUMN

public static final String DEFAULT_COLUMN

See Also:

Constant Field Values

Constructor Detail

NDCGPredictMetric

public NDCGPredictMetric()

Create a new log_2 nDCG metric with column name “Predict.nDCG”.

NDCGPredictMetric

public NDCGPredictMetric(Discount disc)

Create a new nDCG metric with column name “Predict.nDCG”.

Parameters:

disc - The discount.

NDCGPredictMetric

public NDCGPredictMetric(NDCGPredictMetric.Spec spec)

Construct a predict metric from a spec.

Parameters:

spec - The metric spec.

NDCGPredictMetric

public NDCGPredictMetric(Discount disc,
                         String name)

Create a new nDCG metric.

Parameters:

disc - The discount.

name - The column name.

Method Detail

createContext

@Nullable
public MeanAccumulator createContext(AlgorithmInstance algorithm,
                                                DataSet dataSet,
                                                Recommender recommender)

Description copied from class: Metric

Create the context for an experimental condition (algorithm/data set pair). The default implementation returns null.

Overrides:

createContext in class Metric<MeanAccumulator>

Parameters:

algorithm - The algorithm.

dataSet - The data set.

recommender - The LensKit recommender, if applicable. This can be null for an external algorithm that does not provide a LensKit recommender.

Returns:

The accumulator. This will be passed to the individual measurement methods. If the metric does need to accumulate any results, this method can return null.

getAggregateMeasurements

@Nonnull
public MetricResult getAggregateMeasurements(MeanAccumulator context)

Description copied from class: Metric

Get the aggregate results from an accumulator. The default implementation returns MetricResult.empty().

Overrides:

getAggregateMeasurements in class Metric<MeanAccumulator>

Parameters:

context - The context for an experimental condition.

Returns:

The aggregate results from the accumulator.

measureUser

@Nonnull
public MetricResult measureUser(TestUser user,
                                          ResultMap predictions,
                                          MeanAccumulator context)

Description copied from class: PredictMetric

Measure a single result. The result may come from either prediction or recommendation.

Specified by:

measureUser in class PredictMetric<MeanAccumulator>

Parameters:

user - The user’s test data.

predictions - The predictions.

Returns:

A list of fields to add to the result’s output.