Chapter 4: Fourier Series and Complex Exponential¶
Decomposing Any Function into Frequencies¶
Fourier series represent one of the most far-reaching ideas in mathematics: any periodic function can be decomposed into a weighted sum of sines and cosines at integer multiples of a fundamental frequency:
The coefficients \(a_n\) and \(b_n\) are computed via inner products (integrals) that measure how much each frequency component contributes to the overall signal. This is the mathematical foundation of spectral analysis – breaking a complex signal into its constituent frequencies.
Why this is essential for ML/AI: The Discrete Fourier Transform (DFT) and Fast Fourier Transform (FFT) are workhorses of modern AI. Audio processing models (speech recognition, music generation) operate on spectral representations. Positional encodings in Transformers use sinusoidal functions at different frequencies directly inspired by Fourier decomposition. Fourier features have also been shown to help neural networks learn high-frequency patterns that they otherwise struggle to capture (the “spectral bias” problem).
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from matplotlib.animation import FuncAnimation
from IPython.display import HTML
sns.set_style('whitegrid')
plt.rcParams['figure.figsize'] = (14, 10)
np.set_printoptions(precision=3, suppress=True)
Euler’s Formula: Unifying Sines, Cosines, and Exponentials¶
Euler’s formula \(e^{i\theta} = \cos(\theta) + i\sin(\theta)\) is one of the most beautiful identities in mathematics. It shows that complex exponentials encode both sine and cosine simultaneously, allowing us to write the Fourier series in a more compact complex form:
where \(c_n = \frac{1}{2\pi}\int_{-\pi}^{\pi} f(x) e^{-inx} dx\) are complex Fourier coefficients. The magnitude \(|c_n|\) tells you how much frequency \(n\) contributes, and the phase \(\arg(c_n)\) tells you the shift.
This complex exponential notation simplifies computation enormously and is the form used in practice. When you call np.fft.fft() in NumPy, you are computing these complex Fourier coefficients. In deep learning, complex-valued representations and Fourier-based layers (like FNet, which replaces attention with FFT) exploit this elegant formulation for computational efficiency.