khalib

khalib is a classifier probability calibration package powered by the Khiops AutoML suite.

Features

• KhalibClassifier: A scikit-learn estimator to calibrate classifiers with a similar interface as CalibratedClassifierCV.

• calibration_error : A function to estimate the Estimated Calibration Error (ECE).

• build_reliability_diagram : A function to build a reliability diagram.

These features are based on Khiops’s non-parametric supervised histograms, so there is no need to specify the number and width of the bins, as they are automatically estimated from data.

Installation

• Install Khiops

• Install the khalib python library:

pip install khalib

How does it work

khalib proposes histogram-based calibration and its error estimation. Its differentiating factor is that uses Khiops to construct the histogram in which \(P(Y = 1 | S)\) is estimated. These histograms have the following properties:

• They balance class purity, model complexity and data fitness.

• They are non-parametric: The optimal histogram is searched without constraint in number of bins or bin width. This implies that the user doesn’t need to set a number of bins nor their widths.

See the Quickstart and API reference to learn how to use the library.