Run this notebook: Open in Colab Open in Kaggle

Chapter 7: Limits and L’Hopital’s Rule

Limits: The Rigorous Engine Behind Calculus

Every concept in calculus – derivatives, integrals, series convergence – ultimately rests on the notion of a limit. A limit describes the value a function approaches as its input approaches some target, even if the function is not defined at that exact point. This chapter revisits limits formally and introduces a powerful shortcut for evaluating tricky indeterminate forms.

L’Hopital’s Rule handles the frustrating cases where direct substitution yields \(\frac{0}{0}\) or \(\frac{\infty}{\infty}\). The insight is elegant: if \(\lim_{x \to a} \frac{f(x)}{g(x)}\) gives an indeterminate form, then the limit equals \(\lim_{x \to a} \frac{f'(x)}{g'(x)}\), provided this latter limit exists. Geometrically, both \(f\) and \(g\) are vanishing at the same point, so their ratio depends on how fast each vanishes – which is exactly what their derivatives measure. In ML, limits appear when analyzing learning rate schedules, convergence guarantees of gradient descent, and the behavior of loss functions near critical points.

import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

sns.set_style('whitegrid')
plt.rcParams['figure.figsize'] = (14, 10)

Example: sin(x)/x as x → 0

\[ \lim_{x \to 0} \frac{\sin(x)}{x} \]

Direct substitution: 0/0 (indeterminate)

Using L’Hôpital’s Rule: $\( \lim_{x \to 0} \frac{\sin(x)}{x} = \lim_{x \to 0} \frac{\cos(x)}{1} = 1 \)$

def visualize_lhopital():
    x = np.linspace(-3, 3, 1000)
    x = x[np.abs(x) > 0.001]  # Avoid division by zero
    y = np.sin(x) / x
    
    plt.figure(figsize=(10, 6))
    plt.plot(x, y, 'b-', linewidth=2, label='sin(x)/x')
    plt.axhline(1, color='r', linestyle='--', label='Limit = 1')
    plt.axvline(0, color='gray', linestyle=':', alpha=0.5)
    plt.plot(0, 1, 'ro', markersize=10, label='lim as x→0')
    plt.grid(True, alpha=0.3)
    plt.legend()
    plt.title("L'Hôpital's Rule: lim sin(x)/x = 1", fontweight='bold')
    plt.show()

visualize_lhopital()