Plot Gpc Isoprobability¶
================================================================= Iso-probability lines for Gaussian Processes classification (GPC)¶
A two-dimensional classification example showing iso-probability lines for the predicted probabilities.
Imports for GPC Iso-Probability Contour Visualization¶
Non-stationary kernel for polynomial decision boundaries: The kernel ConstantKernel * DotProduct**2 creates a non-stationary covariance function that depends on the absolute positions of inputs rather than just their distance. The squared DotProduct kernel corresponds to a second-degree polynomial feature mapping, enabling the Gaussian process to model quadratic decision boundaries – appropriate for the true classification boundary g(x) = 5 - x2 - 0.5*x1^2, which is a parabola in the 2D feature space. The ConstantKernel prefix scales the overall signal variance, and both its magnitude and the DotProduct’s sigma_0 parameter are optimized during fitting.
Iso-probability contours reveal uncertainty structure: The predict_proba output is evaluated on a dense mesh grid and visualized as a grayscale image where brightness indicates the probability of the positive class. The three contour lines at p=0.334, p=0.5, and p=0.666 show how the posterior probability transitions from one class to the other. The p=0.5 contour (dashed) represents the decision boundary, while the gap between the 0.334 and 0.666 contours indicates regions of high uncertainty where the model is least confident. Comparing the learned p=0.5 contour against the true boundary g(x)=0 (dash-dot line) reveals how accurately the GP recovers the underlying classification rule from only 8 training points.
# Authors: The scikit-learn developers
# SPDX-License-Identifier: BSD-3-Clause
import numpy as np
from matplotlib import cm
from matplotlib import pyplot as plt
from sklearn.gaussian_process import GaussianProcessClassifier
from sklearn.gaussian_process.kernels import ConstantKernel as C
from sklearn.gaussian_process.kernels import DotProduct
# A few constants
lim = 8
True Classification Boundary Function¶
Quadratic decision surface for binary classification: The function g(x) = 5 - x2 - 0.5*x1^2 defines the ground-truth classification rule where samples are labeled positive when g(x) > 0 (inside a downward-opening parabolic region) and negative otherwise. This non-linear boundary cannot be captured by a linear kernel but is well-suited for the squared DotProduct kernel, which implicitly maps inputs to a polynomial feature space where the parabolic boundary becomes expressible.
def g(x):
"""The function to predict (classification will then consist in predicting
whether g(x) <= 0 or not)"""
return 5.0 - x[:, 1] - 0.5 * x[:, 0] ** 2.0
# Design of experiments
X = np.array(
[
[-4.61611719, -6.00099547],
[4.10469096, 5.32782448],
[0.00000000, -0.50000000],
[-6.17289014, -4.6984743],
[1.3109306, -6.93271427],
[-5.03823144, 3.10584743],
[-2.87600388, 6.74310541],
[5.21301203, 4.26386883],
]
)
# Observations
y = np.array(g(X) > 0, dtype=int)
# Instantiate and fit Gaussian Process Model
kernel = C(0.1, (1e-5, np.inf)) * DotProduct(sigma_0=0.1) ** 2
gp = GaussianProcessClassifier(kernel=kernel)
gp.fit(X, y)
print("Learned kernel: %s " % gp.kernel_)
# Evaluate real function and the predicted probability
res = 50
x1, x2 = np.meshgrid(np.linspace(-lim, lim, res), np.linspace(-lim, lim, res))
xx = np.vstack([x1.reshape(x1.size), x2.reshape(x2.size)]).T
y_true = g(xx)
y_prob = gp.predict_proba(xx)[:, 1]
y_true = y_true.reshape((res, res))
y_prob = y_prob.reshape((res, res))
# Plot the probabilistic classification iso-values
fig = plt.figure(1)
ax = fig.gca()
ax.axes.set_aspect("equal")
plt.xticks([])
plt.yticks([])
ax.set_xticklabels([])
ax.set_yticklabels([])
plt.xlabel("$x_1$")
plt.ylabel("$x_2$")
cax = plt.imshow(y_prob, cmap=cm.gray_r, alpha=0.8, extent=(-lim, lim, -lim, lim))
norm = plt.matplotlib.colors.Normalize(vmin=0.0, vmax=0.9)
cb = plt.colorbar(cax, ticks=[0.0, 0.2, 0.4, 0.6, 0.8, 1.0], norm=norm)
cb.set_label(r"${\rm \mathbb{P}}\left[\widehat{G}(\mathbf{x}) \leq 0\right]$")
plt.clim(0, 1)
plt.plot(X[y <= 0, 0], X[y <= 0, 1], "r.", markersize=12)
plt.plot(X[y > 0, 0], X[y > 0, 1], "b.", markersize=12)
plt.contour(x1, x2, y_true, [0.0], colors="k", linestyles="dashdot")
cs = plt.contour(x1, x2, y_prob, [0.666], colors="b", linestyles="solid")
plt.clabel(cs, fontsize=11)
cs = plt.contour(x1, x2, y_prob, [0.5], colors="k", linestyles="dashed")
plt.clabel(cs, fontsize=11)
cs = plt.contour(x1, x2, y_prob, [0.334], colors="r", linestyles="solid")
plt.clabel(cs, fontsize=11)
plt.show()