Run this notebook: Open in Colab Open in Kaggle

Chapter 5: Derivatives of Exponential

Exponential Functions

Functions of the form f(x) = aˣ are special!

Why e is Special

For f(x) = eˣ, the derivative is… itself!

\[ \frac{d}{dx} e^x = e^x \]

This makes e the most natural base for exponentials.

import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

sns.set_style('whitegrid')
plt.rcParams['figure.figsize'] = (14, 10)

The Derivative of \(2^x\): Discovering the Pattern

Exponential functions like \(f(x) = 2^x\) grow in a fundamentally different way from polynomials – their rate of growth is proportional to their current value. To find the derivative, we apply the limit definition:

\[\frac{d}{dx} 2^x = \lim_{dx \to 0} \frac{2^{x+dx} - 2^x}{dx} = 2^x \cdot \lim_{dx \to 0} \frac{2^{dx} - 1}{dx}\]

That remaining limit evaluates to a constant, approximately \(0.693\), which turns out to be \(\ln(2)\). So \(\frac{d}{dx} 2^x = \ln(2) \cdot 2^x\). The visualization below confirms this by plotting \(2^x\) alongside its numerically computed derivative and checking their ratio. In machine learning, understanding exponential derivatives is essential because the softmax function \(\sigma(z_i) = \frac{e^{z_i}}{\sum_j e^{z_j}}\) relies entirely on exponential behavior, and its gradients flow through these same derivative rules during backpropagation.

def visualize_exponential_derivative():
    x = np.linspace(-2, 3, 100)
    f = 2**x
    
    # Approximate derivative
    dx = 0.01
    df_dx = (2**(x+dx) - 2**x) / dx
    
    fig, axes = plt.subplots(1, 2, figsize=(14, 6))
    
    ax1 = axes[0]
    ax1.plot(x, f, 'b-', linewidth=2, label='f(x) = 2ˣ')
    ax1.plot(x, df_dx, 'r--', linewidth=2, label="f'(x)")
    ax1.legend()
    ax1.grid(True, alpha=0.3)
    ax1.set_title('2ˣ and its Derivative', fontweight='bold')
    
    ax2 = axes[1]
    ratio = df_dx / f
    ax2.plot(x, ratio, 'g-', linewidth=2)
    ax2.axhline(np.log(2), color='r', linestyle='--', label=f'ln(2) ≈ {np.log(2):.3f}')
    ax2.legend()
    ax2.grid(True, alpha=0.3)
    ax2.set_title("Ratio f'(x)/f(x)", fontweight='bold')
    plt.tight_layout()
    plt.show()
    print(f"For 2ˣ: d(2ˣ)/dx = ln(2)·2ˣ ≈ {np.log(2):.3f}·2ˣ")

visualize_exponential_derivative()

Why e is Natural

The number e ≈ 2.71828… is chosen so that:

\[ \frac{d}{dx} e^x = e^x \]

Definition: e is the unique number where the derivative equals the function!

General rule: $\( \frac{d}{dx} a^x = \ln(a) \cdot a^x \)$

When a = e, ln(e) = 1, so d(eˣ)/dx = eˣ ✓

def show_e_special():
    print("Why e is special:\n")
    bases = [2, np.e, 3, 10]
    print("Base a | ln(a)  | d(aˣ)/dx")
    print("-" * 35)
    for a in bases:
        print(f"  {a:4.2f} | {np.log(a):6.3f} | {np.log(a):.3f}·{a:.2f}ˣ")
    print("\nNotice: For e, ln(e) = 1, so d(eˣ)/dx = 1·eˣ = eˣ !")

show_e_special()