Run this notebook: Open in Colab Open in Kaggle

Chapter 8: Integration and the Fundamental Theorem

Integration: Accumulating Infinitesimal Contributions

While differentiation asks “how fast is something changing right now?”, integration asks the complementary question: “given a rate of change, how much has accumulated?” The integral \(\int_a^b f(x)\,dx\) computes the signed area under the curve \(f(x)\) between \(x = a\) and \(x = b\) by summing infinitely many infinitesimally thin rectangles.

The Fundamental Theorem of Calculus ties these two ideas together in one of the most profound results in mathematics:

\[\int_a^b f'(x)\,dx = f(b) - f(a)\]

This says that to compute the total accumulation of a rate of change, you only need to evaluate the antiderivative at the endpoints. In machine learning, integration appears in computing expected values over probability distributions, calculating areas under ROC curves, defining loss functions in variational inference, and understanding the cumulative effect of gradient updates during training.

import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

sns.set_style('whitegrid')
plt.rcParams['figure.figsize'] = (14, 10)

Approximating Area with Riemann Sums

The integral is defined as the limit of Riemann sums: partition the interval \([a, b]\) into \(n\) sub-intervals of width \(\Delta x = \frac{b-a}{n}\), evaluate \(f\) at a sample point in each sub-interval, multiply by the width, and sum. As \(n \to \infty\), the sum converges to the exact area:

\[\int_a^b f(x)\,dx = \lim_{n \to \infty} \sum_{i=1}^{n} f(x_i) \cdot \Delta x\]

The code below uses matplotlib.patches.Rectangle to draw these approximating rectangles for \(f(x) = x^2\) on \([0, 3]\) with increasing values of \(n\). Watch how the jagged staircase of rectangles smooths into a perfect fit as \(n\) grows – this is the geometric essence of integration. Numerical integration (quadrature) is how computers evaluate integrals that have no closed-form antiderivative, which is the typical case in Bayesian ML when computing posterior distributions.

def riemann_sum():
    def f(x):
        return x**2
    
    a, b = 0, 3
    fig, axes = plt.subplots(1, 3, figsize=(15, 5))
    
    for idx, n in enumerate([5, 20, 100]):
        ax = axes[idx]
        x = np.linspace(a, b, 1000)
        ax.plot(x, f(x), 'b-', linewidth=2)
        
        # Rectangles
        x_rect = np.linspace(a, b, n)
        dx = (b - a) / n
        for xi in x_rect[:-1]:
            height = f(xi)
            rect = plt.Rectangle((xi, 0), dx, height, 
                                facecolor='lightblue', edgecolor='blue', alpha=0.5)
            ax.add_patch(rect)
        
        area = sum(f(xi) * dx for xi in x_rect[:-1])
        ax.set_title(f'n={n}, Area≈{area:.2f}', fontweight='bold')
        ax.grid(True, alpha=0.3)
    
    plt.tight_layout()
    plt.show()
    print(f"Exact: ∫₀³ x² dx = [x³/3]₀³ = 9")

riemann_sum()