Quickstart

These examples are minimal examples for the OrdBoostClassifier and the OrdBoostRegressor models.

OrdBoostClassifier simple example

Use OrdBoostClassifier for discrete ordinal problems (like Likert survey responses or stage rankings) and score probabilistic performance via discrete CRPS.

First let us generate some synthetic ordinal data to train and test the model.

import numpy as np

from ordboost import OrdBoostClassifier

# --- Generate data
# Ordinal targets (e.g., pain scale 0-4)
rng = np.random.default_rng(42)  # For reproducibility
X_train = rng.random((250, 5))
y_train = rng.choice([0, 1, 2, 3, 4], size=250, p=[0.1, 0.2, 0.4, 0.2, 0.1])
X_test = rng.random((3, 5))

Now that the data has been generated, we can very easily fit the classifier. We impose monotonicity with the "isotonic" method and fit the model on the train set.

model = OrdBoostClassifier(monotonicity="isotonic").fit(X_train, y_train)

Finally, we can make predictions with the model and predict the median value. Three different ways to achieve this are shown in the example. The predict() function with the "median" method is the most direct way. After generating the predictive distributions for the test samples, we can either use the median() method, or the ppf for quantile q = 0.5.

# Three ways to make predictions
predictions = model.predict(X_test, method="median")
dist = model.predict_dist(X_test)  # Predictive distributions
medians = dist.median()
ppf_median = dist.ppf(0.5)  # Percent Point Function

print(f"Sample 1 median from predict(): {predictions[1]}")
print(f"Sample 1 median from median(): {medians[1]}")
print(f"Sample 1 median from ppf(0.5): {ppf_median[1]}")

The results should be:

Sample 1 median from predict(): 2
Sample 1 median from median(): 2
Sample 1 median from ppf(0.5): 2

Full example:

import numpy as np

from ordboost import OrdBoostClassifier

# --- Generate data
# Ordinal targets (e.g., pain scale 0-4)
rng = np.random.default_rng(42)  # For reproducibility
X_train = rng.random((250, 5))
y_train = rng.choice([0, 1, 2, 3, 4], size=250, p=[0.1, 0.2, 0.4, 0.2, 0.1])
X_test = rng.random((3, 5))

# --- Fit model
model = OrdBoostClassifier(monotonicity="isotonic").fit(X_train, y_train)

# --- Predict
# Three ways to make predictions
predictions = model.predict(X_test, method="median")
dist = model.predict_dist(X_test)  # Predictive distributions
medians = dist.median()
ppf_median = dist.ppf(0.5)  # Percent Point Function

print(f"Sample 1 median from predict(): {predictions[1]}")
print(f"Sample 1 median from median(): {medians[1]}")
print(f"Sample 1 median from ppf(0.5): {ppf_median[1]}")

OrdBoostRegressor simple example

Use OrdBoostRegressor for continuous problems and estimate prediction intervals for the outcome. Let's start by generating some synthetic data to fit and test the model.

import numpy as np

from ordboost import OrdBoostRegressor

# --- Generate data
# 1. Generate synthetic continuous regression data with non-linear skew
rng = np.random.default_rng(42)
X_train = rng.standard_normal((300, 3))
y_train = np.exp(X_train[:, 0] * 0.6) + rng.normal(0.0, 0.5, size=300)

X_test = rng.standard_normal((1, 3))

Now we use the training data to fit the OrdBoostRegressor. The model is created with a EmpiricalMeanBinMapper and 5 bins (4 bin edges).

# 2. Instantiate and fit OrdBoostRegressor
reg = OrdBoostRegressor(
    bin_edges=[-0.5, 1.5, 3.0, 5.5],
    mapper="mean",
    learning_rate=0.05,
    max_iter=50,
    random_state=42,
)
reg.fit(X_train, y_train)

Once the model is fitted, we can make predictions. The OrdBoostRegressor uses the same functions as the OrdBoostClassifier to make predictions and estimate prediction intervals. However, the cumulative distribution function (CDF) can be used to estimate probabilities, such as P(Y <= 3).

We used the default method for the predict function which computes the prediction using the mean, and then specify the median method to get the median prediction. These produce different results.

# 3. Generate continuous point predictions
y_pred_mean = reg.predict(X_test)
y_pred_median = reg.predict(X_test, method="median")

# 80% prediction intervals
dist = reg.predict_dist(X_test)
lower_80, upper_80 = dist.interval(alpha=0.20)
prob_under_3 = dist.cdf(3.0)

# Display predictions for test sample
print(
    f"Predicted mean: {y_pred_mean[0]:.2f} "
    f"[80% PI: {lower_80[0]:.2f}, {upper_80[0]:.2f}]"
)
print(
    f"Predicted median: {y_pred_median[0]:.2f} "
    f"[80% PI: {lower_80[0]:.2f}, {upper_80[0]:.2f}]"
)
print(f"P(Y <= 3.0): {prob_under_3[0]:.2%}")

The results should be:

Predicted mean: 0.68 [80% PI: -0.24, 1.40]
Predicted median: 0.72 [80% PI: -0.24, 1.40]
P(Y <= 3.0): 99.59%

Full example:

import numpy as np

from ordboost import OrdBoostRegressor

# --- Generate data
# 1. Generate synthetic continuous regression data with non-linear skew
rng = np.random.default_rng(42)
X_train = rng.standard_normal((300, 3))
y_train = np.exp(X_train[:, 0] * 0.6) + rng.normal(0.0, 0.5, size=300)

X_test = rng.standard_normal((1, 3))

# --- Fit model
# 2. Instantiate and fit OrdBoostRegressor
reg = OrdBoostRegressor(
    bin_edges=[-0.5, 1.5, 3.0, 5.5],
    mapper="mean",
    learning_rate=0.05,
    max_iter=50,
    random_state=42,
)
reg.fit(X_train, y_train)

# --- Predict
# 3. Generate continuous point predictions
y_pred_mean = reg.predict(X_test)
y_pred_median = reg.predict(X_test, method="median")

# 80% prediction intervals
dist = reg.predict_dist(X_test)
lower_80, upper_80 = dist.interval(alpha=0.20)
prob_under_3 = dist.cdf(3.0)

# Display predictions for test sample
print(
    f"Predicted mean: {y_pred_mean[0]:.2f} "
    f"[80% PI: {lower_80[0]:.2f}, {upper_80[0]:.2f}]"
)
print(
    f"Predicted median: {y_pred_median[0]:.2f} "
    f"[80% PI: {lower_80[0]:.2f}, {upper_80[0]:.2f}]"
)
print(f"P(Y <= 3.0): {prob_under_3[0]:.2%}")

Now that the basics have been covered, look at the evaluation examples to see how the models can be evaluated.

---