Chapter 6: Implicit Differentiation¶
When Functions are Defined by Relationships, Not Formulas¶
Most calculus courses focus on explicit functions like \(y = x^2\), where \(y\) is isolated on one side. But many important relationships in mathematics and science are defined implicitly – the equation \(x^2 + y^2 = 1\) defines a circle, and there is no single function \(y = f(x)\) that captures the entire curve. Implicit differentiation lets us find slopes and rates of change directly from such equations by differentiating both sides with respect to \(x\) and applying the chain rule to any term involving \(y\) (since \(y\) is implicitly a function of \(x\)).
Why this matters for ML/AI: Constraint surfaces and level sets appear throughout optimization. When a neural network’s loss landscape is described implicitly by \(L(w_1, w_2, \ldots) = c\), the implicit function theorem (which depends on implicit differentiation) tells us how parameters relate to each other along level curves of constant loss. This is also the foundation for techniques like Lagrange multipliers used in constrained optimization (e.g., SVMs).
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
sns.set_style('whitegrid')
plt.rcParams['figure.figsize'] = (14, 10)
The Circle Example¶
Differentiate both sides with respect to x:
Using chain rule on y²:
Solve for dy/dx:
def implicit_circle():
theta = np.linspace(0, 2*np.pi, 100)
x = np.cos(theta)
y = np.sin(theta)
# Slope from implicit differentiation
slope = -x / (y + 1e-10)
fig, ax = plt.subplots(figsize=(10, 10))
ax.plot(x, y, 'b-', linewidth=3, label='x² + y² = 1')
# Show tangent at a point
t0 = np.pi/4
x0, y0 = np.cos(t0), np.sin(t0)
slope0 = -x0/y0
x_tan = np.linspace(x0-0.5, x0+0.5, 50)
y_tan = y0 + slope0 * (x_tan - x0)
ax.plot(x_tan, y_tan, 'r--', linewidth=2, label=f'Tangent (slope={slope0:.2f})')
ax.plot(x0, y0, 'ro', markersize=10)
ax.set_aspect('equal')
ax.grid(True, alpha=0.3)
ax.legend()
ax.set_title('Implicit Differentiation: Circle', fontweight='bold')
plt.show()
print(f"At ({x0:.2f}, {y0:.2f}): dy/dx = -x/y = {slope0:.2f}")
implicit_circle()