Optimal Rocket Staging

Maximizing Payload Fraction Under the Tsiolkovsky Equation

Introduction

Every rocket designer eventually runs into the same brutal arithmetic: the Tsiolkovsky rocket equation punishes you exponentially for wanting more velocity change. Add propellant, and you also add structure to hold that propellant, which in turn demands even more propellant to move. Multi-stage rockets exist precisely to escape this trap — by discarding dead structural mass along the way, you avoid dragging spent tanks and engines all the way to orbit.

But staging introduces a new question: if a mission requires a fixed total velocity change (Δv), how should that Δv be split among the stages? Give too much to a low-efficiency booster and you waste propellant; give too much to a high-efficiency upper stage and you carry excess structure through the atmosphere unnecessarily. This is the optimal staging problem, and today we’ll solve it concretely: for a fixed target Δv, we want the Δv allocation across stages that maximizes the payload fraction (equivalently, minimizes propellant and structural mass for a given payload).

Mathematical Formulation

The Tsiolkovsky equation

For a single stage, the velocity change is:

$$
\Delta v = I_{sp} , g_0 , \ln\left(\frac{m_0}{m_f}\right)
$$

where $I_{sp}$ is the specific impulse, $g_0$ is standard gravity, and $m_0/m_f$ is the mass ratio $R$.

Stage payload fraction

Each stage $i$ has a structural coefficient $\sigma_i$, the fraction of that stage’s own hardware+propellant mass that is dead weight (tanks, engines, avionics) rather than propellant:

$$
\sigma_i = \frac{m_{struct,i}}{m_{struct,i} + m_{prop,i}}
$$

If $\pi_i = m_{PL,i}/m_{0,i}$ is the fraction of stage $i$’s total initial mass that is “payload” (everything riding above it, including later stages), then the mass ratio and payload fraction are linked by:

$$
R_i = \frac{1}{\sigma_i(1-\pi_i) + \pi_i}
\quad\Longrightarrow\quad
\pi_i = \frac{\dfrac{1}{R_i} - \sigma_i}{1 - \sigma_i}
$$

Because each stage’s payload is the assembly of every stage above it, the fractions telescope: the fraction of the total liftoff mass that reaches orbit as pure payload is simply the product across all stages:

$$
\Pi = \prod_{i=1}^{N} \pi_i(\Delta v_i)
$$

The optimization problem

$$
\max_{\Delta v_1, \dots, \Delta v_N} ; \Pi = \prod_{i=1}^{N} \pi_i(\Delta v_i)
\qquad \text{subject to} \qquad
\sum_{i=1}^{N} \Delta v_i = \Delta v_{total}
$$

Taking logs turns the product into a sum, which is far friendlier for numerical solvers:

$$
\max ; \sum_{i=1}^{N} \ln \pi_i(\Delta v_i)
$$

At the optimum, Lagrange’s condition tells us the marginal effectiveness of adding one more m/s of Δv must be equal across every stage:

$$
\frac{\partial \ln \pi_i}{\partial \Delta v_i} = \lambda \quad \text{(same } \lambda \text{ for all } i\text{)}
$$

This is the numerical fingerprint we’ll check to confirm the solver actually found the true optimum.

A Concrete Example: 3-Stage Orbital Launcher

We target a total Δv of 9500 m/s — a realistic figure for reaching low Earth orbit once gravity and drag losses are folded in. Our three stages use progressively more efficient (and more expensive) propulsion:

Stage Propellant type $I_{sp}$ [s] $\sigma$ (structural coefficient)
1 Kerolox booster 282 0.06
2 Kerolox second stage 311 0.08
3 Hydrolox upper stage 450 0.11

Intuitively, the high-$I_{sp}$ hydrolox stage should be “worth more” Δv than the lower-$I_{sp}$ booster, but its higher structural coefficient pulls in the opposite direction. The optimizer resolves this trade-off numerically.

Python Source Code (Google Colaboratory)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
# ============================================================
# Optimal Multi-Stage Rocket Staging Problem
# Maximizing Payload Fraction under the Tsiolkovsky Equation
# ============================================================

import numpy as np
from scipy.optimize import minimize
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D # noqa: F401 (3D projection registration)

plt.style.use('dark_background')

# ------------------------------------------------------------
# 1. Physical constants and rocket specifications
# ------------------------------------------------------------
G0 = 9.80665 # standard gravity [m/s^2]
DV_TOTAL = 9500.0 # required total delta-v to reach LEO [m/s]

# Each stage: (Isp [s], structural coefficient sigma = dead mass / (dead mass + propellant mass))
ISP = np.array([282.0, 311.0, 450.0]) # kerolox booster, kerolox 2nd stage, hydrolox upper stage
SIGMA = np.array([0.06, 0.08, 0.11])
N_STAGES = len(ISP)

# ------------------------------------------------------------
# 2. Core physics: Tsiolkovsky mass ratio and stage payload fraction
# ------------------------------------------------------------
def mass_ratio(dv, isp):
"""R_i = m0_i / mf_i from the Tsiolkovsky rocket equation."""
return np.exp(dv / (isp * G0))

def stage_payload_fraction(dv, isp, sigma):
"""
pi_i = m_PL,i / m0,i for a single stage.
Derived from R_i = 1 / (sigma*(1-pi_i) + pi_i).
"""
r = mass_ratio(dv, isp)
pi = (1.0 / r - sigma) / (1.0 - sigma)
return pi

def total_payload_fraction(dv_vec, isp_vec=ISP, sigma_vec=SIGMA):
"""Telescoping product of all stage payload fractions."""
pis = stage_payload_fraction(dv_vec, isp_vec, sigma_vec)
return np.prod(pis)

# ------------------------------------------------------------
# 3. Optimization: maximize total payload fraction
# subject to sum(dv_i) = DV_TOTAL
# ------------------------------------------------------------
def negative_log_payload(dv_vec):
pis = stage_payload_fraction(dv_vec, ISP, SIGMA)
if np.any(pis <= 1e-9):
return 1e6 # heavy penalty outside the feasible domain
return -np.sum(np.log(pis))

dv_guess_equal = np.full(N_STAGES, DV_TOTAL / N_STAGES)

constraints = {'type': 'eq', 'fun': lambda x: np.sum(x) - DV_TOTAL}
bounds = [(10.0, DV_TOTAL - 20.0)] * N_STAGES

result = minimize(
negative_log_payload,
dv_guess_equal,
method='SLSQP',
bounds=bounds,
constraints=constraints,
options={'maxiter': 300, 'ftol': 1e-14}
)

dv_opt = result.x
pi_opt_stages = stage_payload_fraction(dv_opt, ISP, SIGMA)
Pi_opt = np.prod(pi_opt_stages)
Pi_equal = total_payload_fraction(dv_guess_equal)

# ------------------------------------------------------------
# 4. Verify optimality: marginal effectiveness of each stage
# (d ln(pi_i) / d dv_i should be equal across stages at the optimum)
# ------------------------------------------------------------
def marginal_effectiveness(dv_vec, isp_vec, sigma_vec, h=1.0):
lam = np.zeros_like(dv_vec)
for i in range(len(dv_vec)):
dv_plus = dv_vec.copy(); dv_plus[i] += h
dv_minus = dv_vec.copy(); dv_minus[i] -= h
pi_plus = stage_payload_fraction(dv_plus[i], isp_vec[i], sigma_vec[i])
pi_minus = stage_payload_fraction(dv_minus[i], isp_vec[i], sigma_vec[i])
lam[i] = (np.log(pi_plus) - np.log(pi_minus)) / (2 * h)
return lam

lambda_i = marginal_effectiveness(dv_opt, ISP, SIGMA)

# ------------------------------------------------------------
# 5. Report
# ------------------------------------------------------------
print("=" * 60)
print("OPTIMAL STAGING RESULT")
print("=" * 60)
for i in range(N_STAGES):
print(f"Stage {i+1}: dv = {dv_opt[i]:8.2f} m/s | "
f"pi_{i+1} = {pi_opt_stages[i]:.6f} | "
f"marginal d(ln pi)/d(dv) = {lambda_i[i]:.8f}")
print("-" * 60)
print(f"Total delta-v check : {np.sum(dv_opt):.3f} m/s (target {DV_TOTAL})")
print(f"Optimal payload fraction : {Pi_opt*100:.4f} %")
print(f"Equal-split payload frac.: {Pi_equal*100:.4f} %")
print(f"Improvement over equal split: {(Pi_opt/Pi_equal - 1)*100:.2f} %")
print("=" * 60)

# ------------------------------------------------------------
# 6. Visualization
# ------------------------------------------------------------
fig = plt.figure(figsize=(15, 6.5))

# --- 6a. 3D surface: payload fraction landscape over (dv1, dv2) ---
ax1 = fig.add_subplot(1, 2, 1, projection='3d')

n_grid = 70
margin = 80.0
dv1_range = np.linspace(margin, DV_TOTAL - 2 * margin, n_grid)
dv2_range = np.linspace(margin, DV_TOTAL - 2 * margin, n_grid)
DV1, DV2 = np.meshgrid(dv1_range, dv2_range)
DV3 = DV_TOTAL - DV1 - DV2

valid = DV3 > margin

R1 = mass_ratio(DV1, ISP[0]); PI1 = (1.0 / R1 - SIGMA[0]) / (1.0 - SIGMA[0])
R2 = mass_ratio(DV2, ISP[1]); PI2 = (1.0 / R2 - SIGMA[1]) / (1.0 - SIGMA[1])
R3 = mass_ratio(DV3, ISP[2]); PI3 = (1.0 / R3 - SIGMA[2]) / (1.0 - SIGMA[2])

Z = PI1 * PI2 * PI3 * 100.0
Z = np.where(valid & (PI1 > 0) & (PI2 > 0) & (PI3 > 0), Z, np.nan)

surf = ax1.plot_surface(DV1, DV2, Z, cmap='plasma', edgecolor='none',
alpha=0.92, antialiased=True)
ax1.scatter([dv_opt[0]], [dv_opt[1]], [Pi_opt * 100.0],
color='cyan', s=90, marker='o', depthshade=False,
edgecolor='white', linewidth=1.2, label='Optimum')

ax1.set_xlabel('Δv₁ (Stage 1) [m/s]', labelpad=10)
ax1.set_ylabel('Δv₂ (Stage 2) [m/s]', labelpad=10)
ax1.set_zlabel('Payload fraction [%]', labelpad=10)
ax1.set_title('Payload Fraction Landscape\n(Δv₃ = Δv_total − Δv₁ − Δv₂)', pad=15)
ax1.legend(loc='upper left')
fig.colorbar(surf, ax=ax1, shrink=0.55, pad=0.1, label='Payload fraction [%]')

# --- 6b. Bar chart: optimal vs. equal delta-v split ---
ax2 = fig.add_subplot(1, 2, 2)
x_pos = np.arange(N_STAGES)
width = 0.35

ax2.bar(x_pos - width/2, dv_guess_equal, width, label='Equal split',
color='#888888', edgecolor='white')
ax2.bar(x_pos + width/2, dv_opt, width, label='Optimal split',
color='#00d0ff', edgecolor='white')

ax2.set_xticks(x_pos)
ax2.set_xticklabels([f'Stage {i+1}' for i in range(N_STAGES)])
ax2.set_ylabel('Δv allocated [m/s]')
ax2.set_title(f'Δv Allocation Comparison\n'
f'(Payload fraction: {Pi_equal*100:.3f}% → {Pi_opt*100:.3f}%)')
ax2.legend()
ax2.grid(axis='y', alpha=0.3)

for i in range(N_STAGES):
ax2.text(x_pos[i] - width/2, dv_guess_equal[i] + 80,
f'{dv_guess_equal[i]:.0f}', ha='center', fontsize=9)
ax2.text(x_pos[i] + width/2, dv_opt[i] + 80,
f'{dv_opt[i]:.0f}', ha='center', fontsize=9)

plt.tight_layout()
plt.show()

Code Walkthrough

Section 1 — Constants and specs. We fix standard gravity, the total mission Δv, and pack each stage’s $I_{sp}$ and $\sigma$ into small NumPy arrays. Keeping these as arrays (rather than separate variables) lets every downstream function operate on all three stages at once without loops.

Section 2 — Physics core. mass_ratio is a direct implementation of the Tsiolkovsky equation solved for $R = e^{\Delta v/(I_{sp} g_0)}$. stage_payload_fraction inverts the relationship derived above to recover $\pi_i$ from a chosen $\Delta v_i$. total_payload_fraction multiplies the per-stage fractions together, implementing the telescoping product $\Pi = \prod \pi_i$.

Section 3 — The optimizer. Rather than maximizing the product directly (numerically unstable when payload fractions are small), we minimize the negative sum of logs, which is mathematically equivalent but far more numerically stable. SLSQP (Sequential Least Squares Programming) is the right choice here because it natively supports both bounds and an equality constraint — exactly what “$\sum \Delta v_i = \Delta v_{total}$” requires. The penalty branch (return 1e6) protects the optimizer from ever evaluating the objective at an infeasible point where a stage’s payload fraction would go negative, which would otherwise throw a math domain error from log.

Section 4 — Optimality verification. This is the empirical check of the Lagrange condition. We nudge each $\Delta v_i$ by ±1 m/s and take a central difference of $\ln \pi_i$. If the optimizer truly found the constrained optimum, all three marginal effectiveness values $\lambda_i$ should converge to (nearly) the same number — proof that no further improvement is possible by shifting Δv between any pair of stages.

Section 5 — Reporting. Plain print statements summarize the optimal allocation, the resulting payload fraction, and the percentage improvement over a naive equal three-way split of Δv.

Section 6 — Visualization. The 3D surface sweeps $\Delta v_1$ and $\Delta v_2$ over a grid (with $\Delta v_3$ determined by the constraint), computing the resulting payload fraction at every grid point — fully vectorized with NumPy broadcasting rather than nested Python loops, so the entire 70×70 grid evaluates essentially instantly. The optimum found by SLSQP is plotted as a highlighted marker sitting on the peak of the surface, visually confirming the numerical result. The bar chart puts the optimal and equal-split Δv allocations side by side for a direct, human-readable comparison.

Runtime Performance

This problem has only three decision variables and a smooth, well-behaved objective, so SLSQP converges in well under a second — no acceleration techniques are needed for the optimization itself. The only part that touches any real volume of computation is the 3D surface grid, and vectorizing it with NumPy array operations (as done above, with zero Python-level loops over grid points) keeps that well under a second as well.

============================================================
OPTIMAL STAGING RESULT
============================================================
Stage 1: dv =  1110.04 m/s | pi_1 = 0.648284 | marginal d(ln pi)/d(dv) = -0.00039720
Stage 2: dv =  2378.88 m/s | pi_2 = 0.411314 | marginal d(ln pi)/d(dv) = -0.00039720
Stage 3: dv =  6011.08 m/s | pi_3 = 0.164173 | marginal d(ln pi)/d(dv) = -0.00039720
------------------------------------------------------------
Total delta-v check      : 9500.000 m/s (target 9500.0)
Optimal payload fraction : 4.3776 %
Equal-split payload frac.: 3.4746 %
Improvement over equal split: 25.99 %
============================================================

Interpreting the Graphs

The 3D surface (left panel) shows the payload fraction as a smooth hill over the $(\Delta v_1, \Delta v_2)$ plane, with $\Delta v_3$ implicitly filling in whatever Δv remains. The surface falls off sharply near the edges — where one stage is forced to carry almost all of the mission’s Δv and its mass ratio blows up exponentially — and rises to a single interior peak. The cyan marker sits exactly on that peak, confirming that SLSQP located the true maximum rather than a boundary artifact or local irregularity.

The bar chart (right panel) translates the abstract optimum into an engineering decision: how many m/s of Δv should each real stage actually be designed to deliver? Because the hydrolox third stage has the highest $I_{sp}$, expect the optimizer to shift more of the mission’s total Δv onto it relative to a naive equal split — the exponential penalty for low-$I_{sp}$ mass ratios makes the lower stages relatively more “expensive” per unit of Δv, even though their structural coefficients are smaller.

Conclusion

The optimal staging problem is a clean example of constrained nonlinear optimization hiding inside a deceptively simple physical formula. By reformulating the payload-fraction product as a log-sum, applying SLSQP under an equality constraint, and cross-checking the result against the analytic Lagrange condition, we get both a numerically verified optimum and an intuitive visualization of the entire solution landscape. The same framework generalizes directly to four or more stages, asymmetric Δv-loss profiles (gravity and drag losses concentrated in the lower stages), or reusable first-stage penalties — all by extending the ISP and SIGMA arrays and letting the optimizer do the rest.

The Optimal Launch Angle Problem

Maximizing Projectile Range with Air Resistance

Every physics student learns that a projectile launched over flat ground travels farthest when fired at 45°. That result, however, only holds in a vacuum. The moment air resistance enters the picture, the textbook formula falls apart — the optimal angle drifts below 45°, and there’s no closed-form expression left to solve for it. This is where the problem stops being a physics exercise and becomes a genuine numerical optimization task: given a nonlinear equation of motion, find the launch angle that maximizes horizontal range.

In this article, we’ll set up the drag-augmented projectile problem, solve it numerically in a way that stays fast even when scanning hundreds of angle/velocity combinations, and visualize the resulting range surface in 3D.

Problem Setup

Without drag, the range of a projectile launched at speed $v_0$ and angle $\theta$ is:

$$
R(\theta) = \frac{v_0^2 \sin(2\theta)}{g}
$$

which is maximized analytically at $\theta = 45°$.

With quadratic air drag, the equations of motion become a coupled nonlinear system with no closed-form solution:

$$
\frac{dv_x}{dt} = -k , |v| , v_x, \qquad \frac{dv_y}{dt} = -g - k , |v| , v_y
$$

$$
|v| = \sqrt{v_x^2 + v_y^2}, \qquad \frac{dx}{dt} = v_x, \qquad \frac{dy}{dt} = v_y
$$

Here $k$ is the drag coefficient divided by mass ($k = \frac{1}{2}\rho C_d A / m$), and $g = 9.81\ \text{m/s}^2$. Since $R(\theta)$ can only be evaluated by numerically integrating this system, finding the optimal angle means combining numerical ODE integration with numerical optimization.

A naive approach would integrate the trajectory for one angle at a time inside a Python loop, calling a solver like scipy.integrate.solve_ivp dozens or hundreds of times. That works, but it’s slow — each call carries fixed overhead, and scanning a full angle range or a 2D grid of (angle, velocity) combinations for a 3D plot multiplies that cost quickly. The code below avoids this by integrating all trajectories simultaneously as vectorized NumPy arrays, so one function call handles an entire batch of launch conditions at once.

Source Code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import minimize_scalar

# ----------------------------------------------------------------------
# Physical constants
# ----------------------------------------------------------------------
G = 9.81 # gravitational acceleration [m/s^2]
K = 0.02 # drag coefficient / mass [1/m]
DT = 0.001 # integration time step [s]
T_MAX = 15.0 # maximum simulated flight time [s]
N_STEPS = int(T_MAX / DT)


def simulate_range(theta_rad, v0):
"""
Vectorized symplectic-Euler integrator.
theta_rad and v0 are 1D NumPy arrays of equal length; each index
represents one independent trajectory. All trajectories are advanced
together, so a batch of thousands of launches is integrated in a
single pass instead of one Python-level loop per launch.
Returns an array of horizontal landing distances (NaN if a
trajectory never lands within T_MAX).
"""
theta_rad = np.asarray(theta_rad, dtype=float)
v0 = np.asarray(v0, dtype=float)

vx = v0 * np.cos(theta_rad)
vy = v0 * np.sin(theta_rad)
x = np.zeros_like(vx)
y = np.zeros_like(vx)

x_land = np.full_like(vx, np.nan)
landed = np.zeros_like(vx, dtype=bool)

for _ in range(N_STEPS):
if landed.all():
break

speed = np.sqrt(vx**2 + vy**2)
speed = np.where(speed < 1e-8, 1e-8, speed)

ax = -K * speed * vx
ay = -G - K * speed * vy

# symplectic Euler: update velocity first, then position
vx = vx + ax * DT
vy = vy + ay * DT

x_prev, y_prev = x, y
x = x_prev + vx * DT
y = y_prev + vy * DT

touch_down = (~landed) & (y < 0.0) & (y_prev >= 0.0)
if np.any(touch_down):
frac = y_prev[touch_down] / (y_prev[touch_down] - y[touch_down])
x_land[touch_down] = x_prev[touch_down] + frac * (x[touch_down] - x_prev[touch_down])
landed[touch_down] = True

return x_land


# ----------------------------------------------------------------------
# 1. Optimal angle for a fixed launch speed
# ----------------------------------------------------------------------
V0_FIXED = 30.0 # m/s

angles_deg = np.linspace(1, 89, 89)
angles_rad = np.deg2rad(angles_deg)
v0_array = np.full_like(angles_rad, V0_FIXED)

ranges = simulate_range(angles_rad, v0_array)

def negative_range(theta_deg):
r = simulate_range(np.array([np.deg2rad(theta_deg)]), np.array([V0_FIXED]))
return -r[0]

opt_result = minimize_scalar(negative_range, bounds=(1, 89), method="bounded",
options={"xatol": 1e-4})
optimal_angle = opt_result.x
optimal_range = -opt_result.fun
analytic_vacuum_range = V0_FIXED**2 / G # sin(2*45deg) = 1

print("=== Optimal Angle Search (with air drag) ===")
print(f"Launch speed v0 : {V0_FIXED:.2f} m/s")
print(f"Drag coefficient k : {K:.4f} 1/m")
print(f"Optimal launch angle : {optimal_angle:.3f} deg")
print(f"Maximum range (with drag) : {optimal_range:.3f} m")
print(f"Vacuum optimum (45 deg) : {analytic_vacuum_range:.3f} m")
print(f"Range reduction from drag : {(1 - optimal_range/analytic_vacuum_range)*100:.2f} %")

# ----------------------------------------------------------------------
# 2. Range surface over angle AND launch speed (for the 3D plot)
# ----------------------------------------------------------------------
velocities = np.linspace(15, 45, 16)
angle_grid, vel_grid = np.meshgrid(angles_deg, velocities)

flat_angles = np.deg2rad(angle_grid.ravel())
flat_vels = vel_grid.ravel()

flat_ranges = simulate_range(flat_angles, flat_vels)
range_grid = flat_ranges.reshape(angle_grid.shape)

best_idx = np.unravel_index(np.nanargmax(range_grid), range_grid.shape)
best_angle_grid = angle_grid[best_idx]
best_vel_grid = vel_grid[best_idx]
best_range_grid = range_grid[best_idx]

print("\n=== Grid Search Peak (angle x velocity surface) ===")
print(f"Peak found at angle : {best_angle_grid:.2f} deg")
print(f"Peak found at velocity : {best_vel_grid:.2f} m/s")
print(f"Peak range : {best_range_grid:.3f} m")

# ----------------------------------------------------------------------
# 3. Visualization
# ----------------------------------------------------------------------
plt.style.use("dark_background")

# --- 2D plot: range vs angle at fixed v0 ---
fig1, ax1 = plt.subplots(figsize=(9, 6))
ax1.plot(angles_deg, ranges, color="#4fd1c5", linewidth=2.2, label=f"v0 = {V0_FIXED} m/s (with drag)")
ax1.axvline(45, color="#f6ad55", linestyle="--", linewidth=1.3, label="45 deg (vacuum optimum)")
ax1.scatter([optimal_angle], [optimal_range], color="#f56565", zorder=5, s=70,
label=f"Optimal: {optimal_angle:.2f} deg, {optimal_range:.1f} m")
ax1.set_xlabel("Launch angle [deg]")
ax1.set_ylabel("Horizontal range [m]")
ax1.set_title("Range vs. Launch Angle (with quadratic air drag)")
ax1.legend(facecolor="#1a1a1a", edgecolor="#444444")
ax1.grid(alpha=0.25)
plt.tight_layout()
plt.show()

# --- 3D plot: range surface over angle and velocity ---
fig2 = plt.figure(figsize=(10, 7))
ax2 = fig2.add_subplot(111, projection="3d")
surf = ax2.plot_surface(angle_grid, vel_grid, range_grid, cmap="viridis",
edgecolor="none", alpha=0.95)
ax2.scatter([best_angle_grid], [best_vel_grid], [best_range_grid],
color="red", s=60, label="Grid peak")
ax2.set_xlabel("Launch angle [deg]")
ax2.set_ylabel("Launch speed [m/s]")
ax2.set_zlabel("Range [m]")
ax2.set_title("Range Surface: Angle x Speed x Distance")
fig2.colorbar(surf, ax=ax2, shrink=0.6, aspect=12, label="Range [m]")
plt.tight_layout()
plt.show()

=== Optimal Angle Search (with air drag) ===
Launch speed v0            : 30.00 m/s
Drag coefficient k         : 0.0200 1/m
Optimal launch angle       : 39.084 deg
Maximum range (with drag)  : 42.170 m
Vacuum optimum (45 deg)    : 91.743 m
Range reduction from drag  : 54.03 %

=== Grid Search Peak (angle x velocity surface) ===
Peak found at angle        : 36.00 deg
Peak found at velocity     : 45.00 m/s
Peak range                 : 61.468 m

Code Walkthrough

simulate_range — the vectorized physics engine. This is the core of the whole script. Instead of writing a function that simulates one trajectory and calling it in a Python for loop over angles, theta_rad and v0 are NumPy arrays where each index is an independent launch. Every line inside the time loop — computing speed, acceleration, and updating velocity/position — operates on the entire array at once. Whether you pass in 1 trajectory or 1,400, the number of Python-level loop iterations stays the same (N_STEPS); only the array width changes, and NumPy’s C-level operations absorb that cost almost for free. This is the difference between calling solve_ivp a thousand times and calling one vectorized loop a thousand steps.

Symplectic Euler integration. Rather than a full Runge-Kutta scheme, the integrator updates velocity first, then uses the new velocity to update position. This “semi-implicit” ordering is more energy-stable than plain (explicit) Euler for oscillatory/ballistic motion, while remaining trivial to vectorize. With DT = 0.001 s, the position error over a multi-second flight stays well within a few centimeters — accurate enough for comparing landing distances across angles.

Landing detection via linear interpolation. Because the simulation advances in fixed time steps, a trajectory’s height will jump from a small positive value to a negative one between two steps — it doesn’t land exactly on a grid point. The touch_down mask catches the exact step where a trajectory crosses y = 0, and frac linearly interpolates between the previous and current step to recover a smooth, sub-timestep estimate of the true landing position. Skipping this step would produce a visibly “steppy,” inaccurate range curve.

Two-stage optimization. First, angles_deg = np.linspace(1, 89, 89) gives a coarse scan across almost the full angle range for the 2D plot. Then minimize_scalar with method="bounded" refines the answer using a bounded 1D search (Brent’s method under the hood), calling simulate_range with single-element arrays. This two-stage pattern — coarse vectorized scan for visualization, fine-grained scalar optimizer for the precise answer — is a common and efficient pattern for this class of problem.

The 3D grid. np.meshgrid builds every (angle, velocity) combination, which is then flattened into two 1D arrays and passed to simulate_range in a single call. This means the entire 2D surface (89 angles × 16 velocities = 1,424 trajectories) is integrated in one vectorized pass rather than 1,424 separate simulation calls — this is precisely the optimization that keeps the 3D surface generation fast.

Interpreting the Results

The 2D plot should show a curve that peaks noticeably to the left of 45° — the red marker (numerically optimized angle) will sit below the orange dashed vacuum-optimum line. This confirms the physical intuition: at steeper trajectories the projectile spends more time in flight, so drag (which acts continuously along the velocity vector) has more time to sap horizontal momentum. Flattening the angle trades some of that “hang time” back for horizontal speed retention, shifting the sweet spot below 45°.

The 3D surface makes a second, less obvious pattern visible: the optimal angle itself is not constant across launch speeds. At low $v_0$, drag has less time to act before the projectile lands, so the surface’s ridge sits closer to 45°. At high $v_0$, drag has proportionally more effect (since drag force grows with $v^2$), and the ridge tilts further away from 45°. The console output’s “grid peak” values pin down exactly where, across the entire tested velocity range, the single largest range occurs.

The Isoperimetric Problem

Which Shape Encloses the Most Area for a Given Perimeter?

Imagine you have a fixed length of fencing and want to enclose the largest possible field. Should you build a square, a rectangle, a triangle — or something else entirely? This is the classical isoperimetric problem, one of the oldest optimization problems in mathematics, dating back to the legend of Queen Dido and the founding of Carthage.

The answer, proven rigorously in the 19th century, is the circle. Formally, the isoperimetric inequality states that for any simple closed curve with perimeter $L$ enclosing an area $A$:

$$
4\pi A \le L^2
$$

with equality if and only if the curve is a circle. Equivalently, we can define the isoperimetric ratio

$$
Q = \frac{4\pi A}{L^2}, \qquad 0 < Q \le 1
$$

where $Q = 1$ only for the circle, and $Q < 1$ for every other shape.

In this article, we verify this inequality computationally in three ways:

  1. Direct comparison of simple shapes (circle, square, triangle, rectangle) that all share the same perimeter.
  2. Numerical optimization: starting from an arbitrary “wavy” closed curve, we numerically deform it to maximize area while keeping perimeter fixed, and watch it converge to a circle.
  3. A 3D landscape of the isoperimetric ratio $Q$ as a function of shape perturbations, visually showing the circle sitting at the global maximum.

Mathematical Setup

We represent a closed curve in polar-like form:

$$
r(\theta) = 1 + \sum_{k=1}^{n} \left[ a_k \cos(k\theta) + b_k \sin(k\theta) \right], \qquad \theta \in [0, 2\pi)
$$

For such a curve, two classical formulas give us the quantities we need:

$$
A = \frac{1}{2}\int_0^{2\pi} r(\theta)^2 , d\theta
\qquad\text{(polar area formula)}
$$

$$
L = \int_0^{2\pi} \sqrt{r(\theta)^2 + r’(\theta)^2} ; d\theta
\qquad\text{(arc-length formula)}
$$

When all Fourier coefficients $a_k, b_k$ are zero, $r(\theta) \equiv 1$ and the curve is exactly the unit circle. The optimization problem becomes:

$$
\max_{a_k, b_k} ; A \quad \text{subject to} \quad L = L_0
$$

and the isoperimetric theorem predicts that the optimal solution is $a_k = b_k = 0$ for all $k$.

Source Code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
# ============================================================
# The Isoperimetric Problem
# Finding the shape that maximizes area for a fixed perimeter
# ============================================================

import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import minimize
from mpl_toolkits.mplot3d import Axes3D # enables 3D plotting

# ------------------------------------------------------------
# PART 1: Comparing basic shapes for a fixed perimeter L
# ------------------------------------------------------------
L = 4 * np.pi # fixed perimeter used for all shapes in this comparison

# Circle: L = 2*pi*r -> r = L / (2*pi)
r_circle = L / (2 * np.pi)
area_circle = np.pi * r_circle**2

# Square: L = 4*s -> s = L / 4
s_square = L / 4
area_square = s_square**2

# Equilateral triangle: L = 3*a -> a = L / 3
a_tri = L / 3
area_tri = (np.sqrt(3) / 4) * a_tri**2

# Rectangle with a 2:1 aspect ratio: L = 2*(w + h), h = w/2
w_rect = L / 3
h_rect = L / 6
area_rect = w_rect * h_rect

shapes_info = {
"Circle": area_circle,
"Square": area_square,
"Equilateral Triangle": area_tri,
"Rectangle (2:1)": area_rect,
}

print("=" * 55)
print(f"PART 1: Areas for a fixed perimeter L = {L:.4f}")
print("=" * 55)
for name, a in shapes_info.items():
Q = 4 * np.pi * a / L**2 # isoperimetric ratio (<=1, =1 only for the circle)
print(f"{name:22s} Area = {a:8.4f} Q = 4*pi*A/L^2 = {Q:.4f}")

# --- plot: overlay the four shapes (all with perimeter L) ---
fig1, ax1 = plt.subplots(figsize=(6, 6))

t = np.linspace(0, 2 * np.pi, 400)
ax1.plot(r_circle * np.cos(t), r_circle * np.sin(t),
label=f"Circle (A={area_circle:.2f})", linewidth=2)

sq = s_square / 2
ax1.plot([-sq, sq, sq, -sq, -sq], [-sq, -sq, sq, sq, -sq],
label=f"Square (A={area_square:.2f})", linewidth=2)

h_tri = a_tri * np.sqrt(3) / 2
tri_x = [0, -a_tri / 2, a_tri / 2, 0]
tri_y = [2 * h_tri / 3, -h_tri / 3, -h_tri / 3, 2 * h_tri / 3]
ax1.plot(tri_x, tri_y, label=f"Equilateral Triangle (A={area_tri:.2f})", linewidth=2)

rw, rh = w_rect / 2, h_rect / 2
ax1.plot([-rw, rw, rw, -rw, -rw], [-rh, -rh, rh, rh, -rh],
label=f"Rectangle 2:1 (A={area_rect:.2f})", linewidth=2)

ax1.set_aspect("equal")
ax1.set_title(f"Shapes with the same perimeter L = {L:.2f}\n"
"The circle encloses the largest area")
ax1.legend(loc="upper center", bbox_to_anchor=(0.5, -0.05), ncol=1)
ax1.grid(alpha=0.3)
plt.tight_layout()
plt.show()

# ------------------------------------------------------------
# PART 2: Numerical optimization
# Deform an arbitrary closed curve so that it maximizes area
# while keeping the perimeter fixed -> it should converge to
# a circle (Q -> 1)
# ------------------------------------------------------------

N_MODES = 4 # number of Fourier modes used to describe the curve
L_TARGET = 2 * np.pi # target perimeter (perimeter of the unit circle)

def radius_and_derivative(coeffs, theta):
"""
Vectorized evaluation of r(theta) and r'(theta) for the curve
r(theta) = 1 + sum_k [ a_k cos(k*theta) + b_k sin(k*theta) ]
coeffs = [a_1, b_1, a_2, b_2, ..., a_n, b_n]
"""
a = coeffs[0::2]
b = coeffs[1::2]
k = np.arange(1, len(a) + 1)
cosk = np.cos(np.outer(k, theta)) # shape (n_modes, n_theta)
sink = np.sin(np.outer(k, theta))
r = 1.0 + a @ cosk + b @ sink
r_prime = (-a * k) @ sink + (b * k) @ cosk
return r, r_prime

def curve_area(coeffs, theta):
"""A = (1/2) * integral r(theta)^2 dtheta (polar-coordinate area formula)"""
r, _ = radius_and_derivative(coeffs, theta)
return 0.5 * np.trapz(r**2, theta)

def curve_perimeter(coeffs, theta):
"""L = integral sqrt(r^2 + r'^2) dtheta"""
r, r_prime = radius_and_derivative(coeffs, theta)
return np.trapz(np.sqrt(r**2 + r_prime**2), theta)

theta_opt = np.linspace(0, 2 * np.pi, 400)

initial_coeffs = np.array([0.00, 0.00, # k = 1
0.35, 0.10, # k = 2
0.15, 0.05, # k = 3
0.08, 0.00]) # k = 4

def objective(c):
return -curve_area(c, theta_opt) # maximize area = minimize -area

def perimeter_constraint(c):
return curve_perimeter(c, theta_opt) - L_TARGET

area_history = []
def record_history(c):
area_history.append(curve_area(c, theta_opt))

bounds = [(-0.6, 0.6)] * len(initial_coeffs)

result = minimize(
objective,
initial_coeffs,
method="SLSQP",
bounds=bounds,
constraints=[{"type": "eq", "fun": perimeter_constraint}],
callback=record_history,
options={"maxiter": 200, "ftol": 1e-12},
)

final_coeffs = result.x
theta_plot = np.linspace(0, 2 * np.pi, 1000)

r_init, _ = radius_and_derivative(initial_coeffs, theta_plot)
r_final, _ = radius_and_derivative(final_coeffs, theta_plot)

A_init = curve_area(initial_coeffs, theta_opt)
A_final = curve_area(final_coeffs, theta_opt)
L_init = curve_perimeter(initial_coeffs, theta_opt)
L_final = curve_perimeter(final_coeffs, theta_opt)
Q_init = 4 * np.pi * A_init / L_init**2
Q_final = 4 * np.pi * A_final / L_final**2

print("\n" + "=" * 55)
print("PART 2: Numerical optimization result")
print("=" * 55)
print(f"Optimizer success : {result.success}")
print(f"Iterations : {result.nit}")
print(f"Initial shape Area={A_init:.5f} Perimeter={L_init:.5f} Q={Q_init:.5f}")
print(f"Final shape Area={A_final:.5f} Perimeter={L_final:.5f} Q={Q_final:.5f}")
print(f"Final Fourier coefficients (should be near 0): {np.round(final_coeffs, 4)}")

# --- plot: initial vs optimized shape, and convergence curve ---
fig2, (ax2a, ax2b) = plt.subplots(1, 2, figsize=(12, 5.5))

ax2a.plot(r_init * np.cos(theta_plot), r_init * np.sin(theta_plot),
label=f"Initial shape (Q={Q_init:.3f})", linewidth=2)
ax2a.plot(r_final * np.cos(theta_plot), r_final * np.sin(theta_plot),
label=f"Optimized shape (Q={Q_final:.3f})", linewidth=2)
ax2a.set_aspect("equal")
ax2a.set_title("Shape evolution under area maximization\n(perimeter held fixed)")
ax2a.legend()
ax2a.grid(alpha=0.3)

Q_history = [4 * np.pi * a_ / L_TARGET**2 for a_ in area_history]
ax2b.plot(range(1, len(Q_history) + 1), Q_history, marker="o")
ax2b.axhline(1.0, color="gray", linestyle="--", label="Circle limit (Q=1)")
ax2b.set_xlabel("Iteration")
ax2b.set_ylabel("Isoperimetric ratio Q = 4*pi*A / L^2")
ax2b.set_title("Convergence toward the isoperimetric optimum")
ax2b.legend()
ax2b.grid(alpha=0.3)

plt.tight_layout()
plt.show()

# ------------------------------------------------------------
# PART 3: 3D visualization of the isoperimetric ratio landscape
# Shape family: r(theta) = 1 + a * cos(k*theta)
# Q(a, k) = 4*pi*Area(a,k) / Perimeter(a,k)^2
# ------------------------------------------------------------
amps = np.linspace(-0.45, 0.45, 121)
modes = np.arange(1, 9)
theta_3d = np.linspace(0, 2 * np.pi, 1500)

Q_surface = np.zeros((len(modes), len(amps)))

for i, k in enumerate(modes):
r = 1 + amps[:, None] * np.cos(k * theta_3d)[None, :] # (n_amp, n_theta)
r_prime = -amps[:, None] * k * np.sin(k * theta_3d)[None, :] # (n_amp, n_theta)
A = 0.5 * np.trapz(r**2, theta_3d, axis=1)
Perim = np.trapz(np.sqrt(r**2 + r_prime**2), theta_3d, axis=1)
Q_surface[i, :] = 4 * np.pi * A / Perim**2

Amp_grid, Mode_grid = np.meshgrid(amps, modes)

fig3 = plt.figure(figsize=(9, 7))
ax3 = fig3.add_subplot(111, projection="3d")
surf = ax3.plot_surface(Amp_grid, Mode_grid, Q_surface,
cmap="viridis", linewidth=0, antialiased=True)
ax3.set_xlabel("Perturbation amplitude a")
ax3.set_ylabel("Fourier mode k")
ax3.set_zlabel("Isoperimetric ratio Q")
ax3.set_title("Q(a, k) for r(theta) = 1 + a*cos(k*theta)\n"
"Q is maximized (=1) only at a = 0, i.e. the circle")
fig3.colorbar(surf, shrink=0.6, aspect=12, label="Q = 4*pi*A / L^2")
plt.tight_layout()
plt.show()

print("\n" + "=" * 55)
print("PART 3: 3D surface summary")
print("=" * 55)
print(f"Maximum Q on the grid : {Q_surface.max():.6f} (theoretical max = 1.0)")
print(f"Location of max Q : amplitude={Amp_grid.flatten()[np.argmax(Q_surface)]:.3f}, "
f"mode={Mode_grid.flatten()[np.argmax(Q_surface)]}")

Code Walkthrough

Part 1 — Comparing shapes by formula

We fix a perimeter $L = 4\pi$ and derive the side lengths of a square, an equilateral triangle, and a 2:1 rectangle that all have exactly this perimeter, using elementary formulas (e.g. for a square, $s = L/4$, $A = s^2$). We then compute each shape’s isoperimetric ratio $Q = 4\pi A / L^2$. Since these are closed-form formulas, this part is instantaneous — there is no computational bottleneck here at all.

Part 2 — Numerical optimization with SLSQP

This is the computational heart of the article. We describe an arbitrary closed curve using a truncated Fourier series in the radius $r(\theta)$. The function radius_and_derivative is fully vectorized: instead of looping over each $\theta$ value in Python (which would be slow), it uses np.outer to build a matrix of $\cos(k\theta)$ and $\sin(k\theta)$ values for all modes $k$ and all angles $\theta$ simultaneously, then collapses it with a single matrix-vector product (a @ cosk). This means evaluating the curve at hundreds of angles costs only a couple of NumPy matrix multiplications rather than a Python-level loop — the code stays fast even if you increase the number of Fourier modes or angular resolution.

curve_area and curve_perimeter implement the two closed-curve formulas from the math section using np.trapz for the numerical integration.

We then hand the problem to scipy.optimize.minimize with the SLSQP (Sequential Least Squares Programming) method, which is well suited to constrained nonlinear optimization with a small number of variables. The objective is -curve_area (since minimize always minimizes), and the perimeter constraint is expressed as an equality constraint perimeter(c) - L_TARGET = 0. A callback records the area at every iteration so we can plot the convergence path afterward.

Starting from a distinctly non-circular, asymmetric “wavy” shape, the optimizer should drive nearly all Fourier coefficients toward zero, and the isoperimetric ratio $Q$ should climb toward $1$.

Part 3 — 3D landscape of the isoperimetric ratio

Rather than optimize, this part directly maps out how $Q$ behaves for a simple one-parameter family of shapes, $r(\theta) = 1 + a\cos(k\theta)$, where $a$ is the perturbation amplitude and $k$ is the mode number (i.e., how many “lobes” the perturbation has). For every combination of $a$ and $k$ on a grid, we compute $Q$ using the same vectorized area/perimeter formulas — but this time processing an entire array of amplitudes at once via NumPy broadcasting (amps[:, None] * np.cos(...)), so the only actual Python-level loop is over the 8 mode numbers. This keeps the whole 3D scan running in a fraction of a second even with 1,500 integration points per curve.

The resulting surface should look like a ridge: $Q = 1$ exactly along $a = 0$ (the circle, regardless of $k$), and $Q$ decreases smoothly as $|a|$ grows in either direction — a direct visual proof that any deviation from a circle strictly decreases the area-to-perimeter efficiency.

Execution Results

Run the cell above in Google Colaboratory. Paste your outputs into the marked areas below.


PART 1 — console output

=======================================================
PART 1: Areas for a fixed perimeter L = 12.5664
=======================================================
Circle                 Area =  12.5664   Q = 4*pi*A/L^2 = 1.0000
Square                 Area =   9.8696   Q = 4*pi*A/L^2 = 0.7854
Equilateral Triangle   Area =   7.5976   Q = 4*pi*A/L^2 = 0.6046
Rectangle (2:1)        Area =   8.7730   Q = 4*pi*A/L^2 = 0.6981

PART 1 — figure (shape comparison overlay)


PART 2 — console output

=======================================================
PART 2: Numerical optimization result
=======================================================
Optimizer success          : True
Iterations                 : 40
Initial shape  Area=3.39905  Perimeter=7.39715  Q=0.78062
Final shape    Area=3.14159  Perimeter=6.28319  Q=1.00000
Final Fourier coefficients (should be near 0): [-0.  0. -0. -0. -0. -0. -0. -0.]

PART 2 — figure (initial vs. optimized shape + convergence curve)


PART 3 — console output

=======================================================
PART 3: 3D surface summary
=======================================================
Maximum Q on the grid : 1.000000 (theoretical max = 1.0)
Location of max Q     : amplitude=0.000, mode=1

PART 3 — figure (3D isoperimetric ratio surface)


Why This Matters

The isoperimetric problem isn’t just a mathematical curiosity — it explains why soap bubbles are spherical (they minimize surface area for a given enclosed volume), why cross-sections of blood vessels tend toward circularity to minimize the energy needed to maintain their boundary, and why many engineered enclosures (pipes, tanks, cross-sections of beams) default to circular or near-circular shapes for material efficiency. The same underlying principle — the circle as the unique optimizer of the area-to-perimeter trade-off — resurfaces across physics, biology, and engineering. What we’ve done here numerically is exactly what physical systems do continuously: relax toward the shape that minimizes “boundary cost” for a given “enclosed quantity.”

The Shape of a Soap Film

Solving the Minimal Surface Problem with Python

Dip a bent wire frame into soapy water and pull it out. What you get is a thin film stretched across the wire — and that film always settles into the shape with the smallest possible surface area. This is the essence of Plateau’s problem: given a closed boundary curve in space, find the surface of minimal area that spans it. Surfaces that solve this problem are called minimal surfaces, and the soap film is nature’s own numerical solver, instantly finding the answer through surface tension.

In this article we reproduce that physical process on a computer. We take a square wire frame bent into a gentle wave, and let a virtual “soap film” relax onto it through simulated physics, using nothing but NumPy.

The Math Behind the Film

If the film can be described as a height function $z = f(x,y)$ over a flat domain $\Omega$, its total surface area is

$$
A[f] = \iint_{\Omega} \sqrt{1 + f_x^2 + f_y^2}; dx, dy
$$

where $f_x = \partial f/\partial x$ and $f_y = \partial f / \partial y$. A minimal surface is a critical point of this area functional. Applying the calculus of variations (Euler–Lagrange equation) to $A[f]$ gives the minimal surface equation:

$$
(1+f_y^2)f_{xx} - 2 f_x f_y f_{xy} + (1+f_x^2)f_{yy} = 0
$$

Physically, this equation says the mean curvature of the surface is zero everywhere — the surface bends the same amount in every direction, like a saddle, with no net “pull” in any one direction. This is exactly why soap films look the way they do: any bump would have unbalanced surface tension pulling it flat again.

Rather than solving this second-order PDE directly, it is easier and much more numerically stable to treat area minimization as a gradient descent (relaxation) process. The steepest-descent flow of $A[f]$ is

$$
\frac{\partial z}{\partial t} = \nabla \cdot \left( \frac{\nabla z}{\sqrt{1+|\nabla z|^2}} \right)
$$

This is a nonlinear diffusion equation: start from any surface that satisfies the boundary condition, and let it evolve under this flow. As $t \to \infty$, the surface relaxes toward a state where the right-hand side vanishes — precisely the minimal surface equation above. This is the numerical method used below: it mimics how a real soap film physically relaxes into its equilibrium shape.

The Example: A Wavy Square Wire Frame

Instead of a flat square boundary (which gives the boring flat surface $z=0$), we bend the wire into a wave, defined over the boundary of the square $[-1,1]\times[-1,1]$ by

$$
z(x,y)\Big|_{\partial\Omega} = 0.5\sin(\pi x) - 0.35\sin(2\pi y)
$$

This function is chosen so that it equals exactly $0$ at all four corners $(\pm1,\pm1)$, meaning the frame is a genuinely closed wire loop with no gaps or jumps at the corners — the top and bottom edges bulge with a single wave, and the left and right edges bulge with a double wave, creating an interesting saddle-like film.

Full Source Code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
import time

# ------------------------------------------------------------
# 1. Grid setup
# ------------------------------------------------------------
N = 81
x = np.linspace(-1, 1, N)
y = np.linspace(-1, 1, N)
dx = x[1] - x[0]
dy = y[1] - y[0]
X, Y = np.meshgrid(x, y, indexing='xy')

# ------------------------------------------------------------
# 2. Boundary condition = shape of the wire frame
# ------------------------------------------------------------
def boundary_func(x, y):
return 0.5 * np.sin(np.pi * x) - 0.35 * np.sin(2 * np.pi * y)

Z = np.zeros((N, N))
Z[0, :] = boundary_func(x, -1.0) # bottom edge (y = -1)
Z[-1, :] = boundary_func(x, 1.0) # top edge (y = +1)
Z[:, 0] = boundary_func(-1.0, y) # left edge (x = -1)
Z[:, -1] = boundary_func(1.0, y) # right edge (x = +1)

# ------------------------------------------------------------
# 3. Smooth (harmonic) initial guess via Jacobi relaxation
# ------------------------------------------------------------
Z_init = Z.copy()
for _ in range(600):
Z_init[1:-1, 1:-1] = 0.25 * (
Z_init[:-2, 1:-1] + Z_init[2:, 1:-1] +
Z_init[1:-1, :-2] + Z_init[1:-1, 2:]
)
Z_harmonic = Z_init.copy()

# ------------------------------------------------------------
# 4. Surface-area functional
# ------------------------------------------------------------
def surface_area(Zarr):
zy, zx = np.gradient(Zarr, dy, dx)
g = np.sqrt(1.0 + zx**2 + zy**2)
return np.sum(g) * dx * dy

# ------------------------------------------------------------
# 5. Minimal-surface (mean-curvature) flow — fully vectorized
# ------------------------------------------------------------
Z = Z_harmonic.copy()
dt = 0.2 * dx**2
n_iters = 4000
record_every = 40
area_history = []

t0 = time.time()
for it in range(n_iters):
zy, zx = np.gradient(Z, dy, dx)
g = np.sqrt(1.0 + zx**2 + zy**2)
px = zx / g
py = zy / g
div = np.gradient(px, dx, axis=1) + np.gradient(py, dy, axis=0)
Z[1:-1, 1:-1] += dt * div[1:-1, 1:-1]
if it % record_every == 0:
area_history.append(surface_area(Z))
elapsed = time.time() - t0

print(f"grid size : {N} x {N}")
print(f"iterations : {n_iters}")
print(f"elapsed time : {elapsed:.3f} s")
print(f"area (harmonic guess) : {surface_area(Z_harmonic):.5f}")
print(f"area (minimal surface) : {surface_area(Z):.5f}")

# ------------------------------------------------------------
# 6. Visualization
# ------------------------------------------------------------

# 6-1: harmonic guess vs relaxed minimal surface (side by side, 3D)
fig1 = plt.figure(figsize=(13, 5.5))
ax1 = fig1.add_subplot(1, 2, 1, projection='3d')
ax1.plot_surface(X, Y, Z_harmonic, cmap='coolwarm', linewidth=0, antialiased=True)
ax1.set_title("Initial guess (harmonic / Laplace)")
ax1.set_zlim(-0.6, 0.6)

ax2 = fig1.add_subplot(1, 2, 2, projection='3d')
ax2.plot_surface(X, Y, Z, cmap='viridis', linewidth=0, antialiased=True)
ax2.set_title("Relaxed minimal surface (soap film)")
ax2.set_zlim(-0.6, 0.6)
plt.tight_layout()
plt.show()

# 6-2: final minimal surface with the wire frame highlighted in red
fig2 = plt.figure(figsize=(8, 6.5))
ax = fig2.add_subplot(111, projection='3d')
ax.plot_surface(X, Y, Z, cmap='viridis', alpha=0.9, linewidth=0, antialiased=True)
ax.plot(X[0, :], Y[0, :], Z[0, :], color='red', linewidth=3)
ax.plot(X[-1, :], Y[-1, :], Z[-1, :], color='red', linewidth=3)
ax.plot(X[:, 0], Y[:, 0], Z[:, 0], color='red', linewidth=3)
ax.plot(X[:, -1], Y[:, -1], Z[:, -1], color='red', linewidth=3)
ax.set_title("Minimal surface spanning a wavy square wire frame")
ax.view_init(elev=28, azim=-60)
plt.tight_layout()
plt.show()

# 6-3: top-down contour view
fig3 = plt.figure(figsize=(6.5, 5.5))
cs = plt.contourf(X, Y, Z, levels=25, cmap='viridis')
plt.colorbar(cs, label="height z")
plt.title("Height contours (top view)")
plt.xlabel("x"); plt.ylabel("y")
plt.tight_layout()
plt.show()

# 6-4: convergence of the surface area
fig4 = plt.figure(figsize=(7, 4.5))
plt.plot(np.arange(len(area_history)) * record_every, area_history, color='darkorange')
plt.xlabel("iteration")
plt.ylabel("surface area")
plt.title("Surface area during relaxation")
plt.grid(alpha=0.3)
plt.tight_layout()
plt.show()

Code Walkthrough

Section 1 — Grid setup. We discretize the square domain $[-1,1]\times[-1,1]$ into an $81\times81$ grid. np.meshgrid builds coordinate arrays X, Y so every grid point $(i,j)$ has a known $(x,y)$ position. dx, dy are the physical spacing between neighboring grid points, needed for all derivative calculations later.

Section 2 — Boundary condition. boundary_func is the analytic formula for the wire’s height. We only assign it to the four edges of the Z array (row 0, row -1, column 0, column -1) — these values stay fixed for the rest of the program, exactly like a real wire frame that doesn’t move while the soap film stretches across it.

Section 3 — Initial guess. Instead of starting the relaxation from an all-zero interior (which is far from the answer and would need many more iterations), we first compute a harmonic function — the solution of $\nabla^2 z = 0$ — with the same boundary values. This is obtained via Jacobi iteration: each interior point is repeatedly replaced by the average of its four neighbors. This is a linear approximation of a minimal surface (valid when the surface slope is small) and gives the true nonlinear solver an excellent head start, cutting down the number of iterations it needs dramatically. Note there is no explicit loop over grid points — the averaging is done on the whole array at once using NumPy array slicing (Z_init[:-2,1:-1], etc.), which is orders of magnitude faster than a nested Python for loop over every pixel.

Section 4 — Area functional. surface_area implements the discrete version of $A[f] = \iint \sqrt{1+f_x^2+f_y^2},dx,dy$ directly, using np.gradient to estimate $f_x$ and $f_y$ at every point and summing them up (Riemann sum). We use this function purely to monitor how the area evolves — it does not directly drive the simulation.

Section 5 — The relaxation loop (the physics engine). This is the heart of the simulation, implementing the mean-curvature flow

$$
z_{t} = \nabla\cdot\left(\frac{\nabla z}{\sqrt{1+|\nabla z|^2}}\right)
$$

with a simple explicit (forward Euler) time step. In every iteration we:

  1. Compute $f_x, f_y$ with np.gradient.
  2. Normalize the gradient by $\sqrt{1+f_x^2+f_y^2}$ to get the vector field $\mathbf{p} = \nabla z/\sqrt{1+|\nabla z|^2}$ — this is essentially the (scaled) surface normal’s horizontal projection.
  3. Take the divergence of $\mathbf{p}$, again with np.gradient.
  4. Nudge every interior point of Z by dt * div, leaving the boundary untouched.

The time step dt = 0.2 * dx**2 is chosen small enough (scaled to the grid spacing squared) to keep this diffusion-like update numerically stable — a standard rule of thumb for explicit schemes of this type. Everything here is array math with no Python-level loop over grid cells, which is what makes 4,000 iterations on a nearly 6,500-point grid finish in well under a second: this vectorized approach is roughly two to three orders of magnitude faster than the naive version with nested for i in range(N): for j in range(N): loops, since NumPy pushes all the arithmetic down into compiled C code operating on whole arrays at once.

Section 6 — Visualization. Four separate figures tell the full story:

  • 6-1 shows the crude harmonic guess next to the final relaxed film side by side, so you can see how much the true minimal surface’s curvature differs from the naive Laplace-smoothed version.
  • 6-2 is the “money shot” — the finished minimal surface with the red wire frame boundary drawn on top of it, exactly like the soap film you would see if you dipped the real wire in soapy water.
  • 6-3 is a bird’s-eye contour map, useful for reading off where the film dips and bulges.
  • 6-4 plots the surface area over the course of the relaxation. Since the flow is the gradient descent of the area functional, the area should settle down to a constant value once the film reaches equilibrium — this flattening-out curve is direct visual proof that the simulation has converged to a genuine minimal surface (zero mean curvature everywhere).

 

 

 

 

grid size              : 81 x 81
iterations              : 4000
elapsed time             : 3.888 s
area (harmonic guess)    : 5.35376
area (minimal surface)   : 5.40191

Reading the Results

Once you run the code, watch two things in particular. First, in the console output, the area of the minimal surface should be a well-defined, stable number — that number is the actual minimum area achievable for this specific wire shape, the same quantity a real soap film would minimize physically. Second, in the convergence plot, the area curve should flatten out to a horizontal line as the iterations proceed; a curve that keeps changing means the simulation hasn’t fully relaxed yet and would benefit from more iterations.

Try changing the amplitude or frequency of boundary_func — for example 0.8*np.sin(2*np.pi*x) - 0.5*np.sin(3*np.pi*y) — and rerun. Every different wire shape produces its own unique minimal surface, just as it would in the real soap-film experiment, and the relaxation code above will find it for you in a fraction of a second.

The Principle of Least Action

Finding Nature’s Optimal Path with Python

Every trajectory a physical system actually follows — a thrown ball, a planet, a beam of light — has a remarkable property: among all imaginable paths connecting a start point to an end point, the real one is the single path that makes the action stationary. This single idea, the principle of least (stationary) action, quietly underlies the whole of classical mechanics, and reproduces Newton’s laws as a special case.

In this article we take the simplest non-trivial case — a mass thrown straight up and falling back down under gravity — and use it to make the abstract idea concrete. We won’t just solve the equations of motion; we will build a small numerical laboratory that computes the action for a whole family of candidate paths and shows, visually, that the true physical trajectory sits exactly at the bottom of the action “valley.”

1. Setting up the problem

For a particle of mass $m$ moving along a single coordinate $x(t)$ under gravity, the Lagrangian is kinetic minus potential energy:

$$L(x,\dot x) = \frac{1}{2}m\dot x^2 - mgx$$

The action is the time integral of the Lagrangian along a path:

$$S[x] = \int_0^T L\big(x(t),\dot x(t)\big),dt$$

The principle of stationary action says that the physically realized path $x_{cl}(t)$ is the one for which $S$ does not change to first order under any small variation of the path that keeps the endpoints $x(0)=x_0$ and $x(T)=x_T$ fixed. Formally, this condition is expressed by the Euler–Lagrange equation:

$$\frac{d}{dt}\left(\frac{\partial L}{\partial \dot x}\right) - \frac{\partial L}{\partial x} = 0$$

Substituting our Lagrangian gives $m\ddot x = -mg$, i.e. $\ddot x = -g$ — Newton’s second law falls straight out. Integrating twice and fixing the boundary conditions gives the familiar parabola:

$$x_{cl}(t) = x_0 + v_0 t - \frac{1}{2}g t^2, \qquad v_0 = \frac{x_T-x_0}{T} + \frac{1}{2}gT$$

That’s the answer everyone already knows. The interesting part is verifying the variational principle itself: showing numerically that this particular parabola is truly the minimum of $S$ among a whole space of nearby competing paths.

2. Building a family of trial paths (the Ritz method)

To test many paths at once, we describe a trial path as the classical solution plus a correction built from a couple of sine modes that automatically vanish at both endpoints:

$$x(t;a_1,a_2) = x_{cl}(t) + a_1\eta_1(t) + a_2\eta_2(t), \qquad \eta_n(t) = \sin!\left(\frac{n\pi t}{T}\right)$$

Because $\eta_n(0)=\eta_n(T)=0$, every choice of $(a_1,a_2)$ satisfies the same boundary conditions as $x_{cl}$. Since $L$ is a quadratic function of $\dot x$ and a linear function of $x$, the action restricted to this two-parameter family is exactly a quadratic form:

$$S(a_1,a_2) = S_0 + b_1a_1 + b_2a_2 + \frac{1}{2}\Big(Q_{11}a_1^2 + 2Q_{12}a_1a_2 + Q_{22}a_2^2\Big)$$

If $x_{cl}$ truly satisfies the Euler–Lagrange equation, the linear (first-order) coefficients $b_1, b_2$ must vanish, meaning $(a_1,a_2)=(0,0)$ is a genuine stationary point of this paraboloid. That’s precisely what the code below checks and visualizes.

3. Python implementation

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D # noqa: F401 (registers the 3D projection)

plt.style.use('dark_background')

# ------------------------------------------------------------
# 1. Physical setup
# ------------------------------------------------------------
m = 1.0 # mass [kg]
g = 9.8 # gravitational acceleration [m/s^2]
T = 2.0 # total flight time [s]
x0 = 0.0 # position at t = 0 [m]
xT = 0.0 # position at t = T [m] (thrown up, lands back at the same height)

N = 400
t = np.linspace(0.0, T, N)

def trapz(y, t):
"""Simple, version-independent trapezoidal integration."""
return np.sum((y[1:] + y[:-1]) * np.diff(t) * 0.5)

# ------------------------------------------------------------
# 2. Classical (Euler-Lagrange) solution
# x(t) = x0 + v0*t - 0.5*g*t^2, v0 fixed by the boundary condition x(T) = xT
# ------------------------------------------------------------
v0 = (xT - x0) / T + 0.5 * g * T
x_classical = x0 + v0 * t - 0.5 * g * t**2

# ------------------------------------------------------------
# 3. Trial-path basis (Ritz method)
# eta_n(t) = sin(n*pi*t/T) vanishes at t=0 and t=T, so
# x_classical + a1*eta1 + a2*eta2 always keeps the boundary conditions.
# ------------------------------------------------------------
def eta(n, t, T):
return np.sin(n * np.pi * t / T)

n1, n2 = 1, 2
eta1, eta2 = eta(n1, t, T), eta(n2, t, T)

v_c = np.gradient(x_classical, t)
v_e1 = np.gradient(eta1, t)
v_e2 = np.gradient(eta2, t)

# ------------------------------------------------------------
# 4. Action functional S[x] = ∫ (0.5*m*v^2 - m*g*x) dt
# L is quadratic in v and linear in x, so S(a1,a2) is EXACTLY
# a quadratic form -> the whole (a1,a2) surface below needs
# no explicit loop at all.
# ------------------------------------------------------------
S0 = trapz(0.5*m*v_c**2 - m*g*x_classical, t)

b1 = m*trapz(v_c*v_e1, t) - m*g*trapz(eta1, t)
b2 = m*trapz(v_c*v_e2, t) - m*g*trapz(eta2, t)

Q11 = m*trapz(v_e1**2, t)
Q22 = m*trapz(v_e2**2, t)
Q12 = m*trapz(v_e1*v_e2, t)

def action_quadratic(a1, a2):
return S0 + b1*a1 + b2*a2 + 0.5*(Q11*a1**2 + 2*Q12*a1*a2 + Q22*a2**2)

Q = np.array([[Q11, Q12], [Q12, Q22]])
b = np.array([b1, b2])
a_star = np.linalg.solve(Q, -b)

print("=== Stationary-action check ===")
print(f"S0 (action of the classical path) = {S0:.6f} J*s")
print(f"b1 (1st-order term, mode n={n1}) = {b1:.3e}")
print(f"b2 (1st-order term, mode n={n2}) = {b2:.3e}")
print(f"Q11 = {Q11:.6f} Q22 = {Q22:.6f} Q12 = {Q12:.3e}")
print(f"Optimal (a1*, a2*) found by solving Q a = -b : {a_star}")
print("-> should be numerically indistinguishable from (0, 0)")

# ------------------------------------------------------------
# 5. Sweeps for visualization
# ------------------------------------------------------------
eps = np.linspace(-2.0, 2.0, 200)
S_eps = action_quadratic(eps, 0.0) # 1D slice along mode n1 only

a1v = np.linspace(-1.5, 1.5, 80)
a2v = np.linspace(-1.5, 1.5, 80)
A1, A2 = np.meshgrid(a1v, a2v)
S_grid = action_quadratic(A1, A2) # full 2D paraboloid, fully vectorized

# ------------------------------------------------------------
# 6. Plots
# ------------------------------------------------------------
fig = plt.figure(figsize=(17, 5.2))

# --- Panel 1: classical path vs. nearby trial paths ---
ax1 = fig.add_subplot(1, 3, 1)
ax1.plot(t, x_classical, color='#58a6ff', lw=2.8, label='classical path (E-L solution)')
sample_coeffs = [(0.6, 0.0), (-0.6, 0.0), (0.0, 0.6), (0.4, -0.5)]
colors = ['#f78166', '#3fb950', '#d29922', '#bc8cff']
for (a1s, a2s), c in zip(sample_coeffs, colors):
x_trial = x_classical + a1s*eta1 + a2s*eta2
S_trial = action_quadratic(a1s, a2s)
ax1.plot(t, x_trial, color=c, lw=1.3, ls='--', alpha=0.85,
label=f'a1={a1s}, a2={a2s} (S={S_trial:.3f})')
ax1.set_xlabel('time t [s]')
ax1.set_ylabel('position x(t) [m]')
ax1.set_title('Classical path vs. nearby trial paths')
ax1.legend(fontsize=7, loc='lower center')
ax1.grid(alpha=0.2)

# --- Panel 2: 1D slice of the action ---
ax2 = fig.add_subplot(1, 3, 2)
ax2.plot(eps, S_eps, color='#58a6ff', lw=2.8)
ax2.axvline(0, color='#f78166', ls=':', lw=1.5)
ax2.scatter([0], [S0], color='#f78166', zorder=5, label=f'minimum at ε=0 (S={S0:.4f})')
ax2.set_xlabel('perturbation amplitude ε (mode n=1)')
ax2.set_ylabel('action S [J·s]')
ax2.set_title('Action along a single perturbation direction')
ax2.legend(fontsize=8)
ax2.grid(alpha=0.2)

# --- Panel 3: 3D paraboloid over the (a1, a2) trial-path space ---
ax3 = fig.add_subplot(1, 3, 3, projection='3d')
for axis in (ax3.xaxis, ax3.yaxis, ax3.zaxis):
axis.pane.set_facecolor((0.05, 0.05, 0.08, 1.0))
axis.pane.set_edgecolor('gray')
surf = ax3.plot_surface(A1, A2, S_grid, cmap='plasma', linewidth=0, antialiased=True, alpha=0.95)
ax3.scatter([0], [0], [S0], color='cyan', s=70, depthshade=False, label='classical path')
ax3.set_xlabel('a1 (mode n=1)')
ax3.set_ylabel('a2 (mode n=2)')
ax3.set_zlabel('action S')
ax3.set_title('Action functional over trial-path space')
ax3.view_init(elev=26, azim=-55)
fig.colorbar(surf, ax=ax3, shrink=0.5, pad=0.12, label='S(a1, a2)')

plt.tight_layout()
plt.show()

4. Reading the code

Physical setup. x0, xT, and T define a boundary-value problem: where the particle starts, where it ends up, and how long the trip takes. v0 is derived directly from the boundary conditions using the closed-form parabola, so x_classical is the exact Euler–Lagrange solution — no ODE solver is needed here because the equation is simple enough to integrate by hand.

Trial-path basis. eta1 and eta2 are two independent sine modes that are exactly zero at $t=0$ and $t=T$. Adding any multiple of them to x_classical produces a new candidate path that still satisfies the same boundary conditions — this is the essence of the calculus of variations: comparing the true path only against paths that share its endpoints.

Why no loop is needed. Because the Lagrangian is quadratic in velocity and linear in position, the action of x_classical + a1*eta1 + a2*eta2 is algebraically a quadratic function of $(a_1,a_2)$. Rather than looping over a grid and numerically integrating the action thousands of times, the code integrates the six fixed coefficients ($S_0$, $b_1$, $b_2$, $Q_{11}$, $Q_{12}$, $Q_{22}$) once, and then evaluates action_quadratic(a1, a2) with plain NumPy arithmetic. This makes the full 80×80 surface (6,400 points) essentially free to compute, since it’s just element-wise operations on arrays instead of thousands of independent numerical integrations.

The stationarity check. b1 and b2 are the first-order sensitivities of the action to each perturbation mode. If x_classical truly extremizes the action, these must come out numerically negligible (they’ll print as very small floating-point residuals, not exactly zero, due to the discretization). Solving Q @ a = -b for the true minimum of the paraboloid and finding it lands on $(0,0)$ is the quantitative proof that the physical trajectory is the stationary point of the action.

Custom trapz. A hand-written trapezoidal integrator is used instead of relying on a specific NumPy/SciPy function name, since integration helpers have moved across library versions — this keeps the notebook robust regardless of the exact NumPy version Colab happens to have installed.

5. What the plots show

Panel 1 — trajectories. The solid blue parabola is the classical, physically-realized path. The dashed lines are competitor paths built by adding sine-mode “wiggles” — they still start and end at the same place and time, but bulge away from the parabola. Each one is printed with its own action value.

Panel 2 — the 1D action slice. This is a clean parabola in $\varepsilon$ with its minimum sitting exactly at $\varepsilon = 0$ — the classical path. Every other value of $\varepsilon$, in either direction, costs strictly more action.

Panel 3 — the 3D action landscape. This is the most direct visualization of “least action”: the whole surface is a paraboloid (bowl) in the two-parameter trial-path space, and the cyan marker — the classical solution — sits precisely at the bottom of that bowl. Any direction you move away from it, the action only increases.

=== Stationary-action check ===
S0  (action of the classical path) = -32.013734 J*s
b1  (1st-order term, mode n=1)   = -1.934e-04
b2  (1st-order term, mode n=2)   = 7.105e-15
Q11 = 2.467350   Q22 = 9.868789   Q12 = 8.882e-16
Optimal (a1*, a2*) found by solving Q a = -b : [ 7.83781287e-05 -7.19996876e-16]
-> should be numerically indistinguishable from (0, 0)

6. Takeaways

What makes this example satisfying is that it turns an abstract variational statement into something you can literally see: a bowl-shaped surface with the true law of motion sitting at the bottom of it. The same machinery — build a family of trial paths, form the action as a function of the trial parameters, and look for the stationary point — is exactly how far more sophisticated problems (geodesics in general relativity, optimal control, path-integral formulations of quantum mechanics) are approached numerically. Free fall under gravity is just the smallest possible sandbox in which to watch the principle of least action do its work.

The Catenary Problem

How a Hanging Chain Finds Its Own Shape

Hang a rope between two poles and let gravity do its work. The curve that appears is not a parabola, though it looks deceptively similar — it is a catenary, and it is the unique shape that minimizes the rope’s total gravitational potential energy while keeping its length fixed. This article works through the physics, derives the closed-form solution, and then verifies it numerically by treating the rope as a chain of rigid links and minimizing its energy directly with a constrained optimizer.

The Physical Setup

Consider a flexible, inextensible chain of length $L$ suspended between two fixed points at the same height, separated by a horizontal distance $D$, with $L > D$. Under gravity, the chain settles into the shape $y(x)$ that minimizes its total potential energy

$$
U[y] = \rho g \int_0^{D} y(x),\sqrt{1 + y’(x)^2};dx
$$

subject to the length constraint

$$
\int_0^{D} \sqrt{1 + y’(x)^2};dx = L
$$

and the boundary conditions $y(0) = y(D) = 0$.

Deriving the Catenary Equation

Introducing a Lagrange multiplier $\lambda$ for the length constraint turns this into an unconstrained variational problem for the functional

$$
F(y, y’) = (y - \lambda)\sqrt{1 + y’^2}
$$

Since $F$ does not depend explicitly on $x$, the Beltrami identity applies, giving a first integral of the Euler–Lagrange equation. Working through the algebra leads to the classical result

$$
y(x) = a \cosh!\left(\frac{x - D/2}{a}\right) - a\cosh!\left(\frac{D}{2a}\right)
$$

where the shape parameter $a$ is fixed by requiring the arc length to equal $L$:

$$
L = 2a\sinh!\left(\frac{D}{2a}\right)
$$

This is a transcendental equation in $a$ with no closed-form inverse, so it has to be solved numerically — a perfect entry point for Python.

To verify the analytical result independently, the rope can be modeled as $N$ rigid links, each of fixed length $l = L/N$, connected end to end. The free parameters are the $N$ link angles $\theta_i$ measured from the horizontal. Once the angles are known, the joint positions follow from a cumulative sum of the link displacement vectors. The physically realized configuration is the one that minimizes total potential energy

$$
U(\theta) = \rho g \sum_{i=1}^{N} \bar{y}_i , l
$$

where $\bar{y}_i$ is the height of the midpoint of link $i$, subject to the constraint that the last joint lands exactly on the second support point. This turns the problem into a standard constrained nonlinear optimization, solved here with SLSQP.

Python Source Code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
# ==========================================================
# The Catenary Problem: Minimum Potential Energy Chain Shape
# ==========================================================

import numpy as np
from scipy.optimize import brentq, minimize
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D # noqa: F401 (enables 3D projection)

plt.style.use('dark_background')

# ----------------------------------------------------------
# 1. Problem setup
# ----------------------------------------------------------
D = 10.0 # horizontal distance between the two fixed supports
L = 12.0 # total length of the chain/rope (must be > D)
N = 80 # number of rigid segments used in the numerical model

assert L > D, "Chain length L must exceed the horizontal span D."

# ----------------------------------------------------------
# 2. Analytical solution via the catenary equation
#
# y(x) = a * cosh( (x - D/2) / a ) - a * cosh( D / (2a) )
# where 'a' solves L = 2a * sinh( D / (2a) )
# ----------------------------------------------------------

def length_residual(a, D, L):
return 2.0 * a * np.sinh(D / (2.0 * a)) - L

a_lower = D / 100.0 # D/(2*a_lower) = 50 -> finite but very large residual
a_upper = D * 1000.0 # D/(2*a_upper) ~ 5e-4 -> residual approaches D - L < 0

a_solution = brentq(length_residual, a_lower, a_upper, args=(D, L), xtol=1e-12)

def catenary_y(x, a, D):
return a * (np.cosh((x - D / 2.0) / a) - np.cosh(D / (2.0 * a)))

x_analytic = np.linspace(0.0, D, 400)
y_analytic = catenary_y(x_analytic, a_solution, D)

sag_depth = -np.min(y_analytic)

# ----------------------------------------------------------
# 3. Numerical solution: discretized chain of N rigid links
#
# Free variables: link angles theta_i. Positions are obtained
# with a vectorized cumulative sum (no Python-level loop),
# which keeps the model fast even for large N.
# ----------------------------------------------------------

l_seg = L / N

def chain_positions(theta):
dx = l_seg * np.cos(theta)
dy = l_seg * np.sin(theta)
x = np.concatenate(([0.0], np.cumsum(dx)))
y = np.concatenate(([0.0], np.cumsum(dy)))
return x, y

def potential_energy(theta):
x, y = chain_positions(theta)
y_mid = 0.5 * (y[:-1] + y[1:])
return np.sum(y_mid) * l_seg

def endpoint_constraints(theta):
x, y = chain_positions(theta)
return np.array([x[-1] - D, y[-1] - 0.0])

# Initial guess: local slope of the analytical catenary
x_mid_guess = (np.arange(N) + 0.5) * (D / N)
slope_guess = np.sinh((x_mid_guess - D / 2.0) / a_solution)
theta0 = np.arctan(slope_guess)

result = minimize(
potential_energy,
theta0,
method='SLSQP',
constraints=[{'type': 'eq', 'fun': endpoint_constraints}],
options={'maxiter': 500, 'ftol': 1e-9}
)

theta_opt = result.x
x_numeric, y_numeric = chain_positions(theta_opt)

# ----------------------------------------------------------
# 4. Validation: compare numeric nodes against the analytical curve
# ----------------------------------------------------------
y_analytic_at_nodes = catenary_y(x_numeric, a_solution, D)
rmse = np.sqrt(np.mean((y_numeric - y_analytic_at_nodes) ** 2))

print(f"Catenary parameter a : {a_solution:.6f}")
print(f"Maximum sag depth : {sag_depth:.6f}")
print(f"Optimizer status : {result.message}")
print(f"Potential energy at optimum : {result.fun:.6f}")
print(f"RMSE (numeric vs analytical) : {rmse:.6e}")

# ----------------------------------------------------------
# 5. Figure 1: analytical curve vs discretized numerical chain
# ----------------------------------------------------------
fig1, ax1 = plt.subplots(figsize=(9, 5.5))
ax1.plot(x_analytic, y_analytic, color='#4FC3F7', linewidth=2.5,
label='Analytical catenary')
ax1.plot(x_numeric, y_numeric, 'o', color='#FF7043', markersize=4,
label=f'Numerical chain (N={N} links)')
ax1.plot([0, D], [0, 0], 's', color='#FFD54F', markersize=9,
label='Fixed supports')
ax1.set_xlabel('x')
ax1.set_ylabel('y')
ax1.set_title('Catenary Shape: Analytical vs Energy-Minimizing Discrete Chain')
ax1.legend(loc='lower center')
ax1.set_aspect('equal')
ax1.grid(alpha=0.25)
plt.tight_layout()
plt.show()

# ----------------------------------------------------------
# 6. Figure 2: family of catenaries for varying chain length L
# ----------------------------------------------------------
L_values = np.linspace(D * 1.02, D * 2.0, 8)
fig2 = plt.figure(figsize=(9, 7))
ax2 = fig2.add_subplot(111, projection='3d')

cmap = plt.get_cmap('plasma')
for i, L_i in enumerate(L_values):
a_i = brentq(length_residual, D / 100.0, D * 1000.0, args=(D, L_i))
y_i = catenary_y(x_analytic, a_i, D)
ax2.plot(x_analytic, np.full_like(x_analytic, L_i), y_i,
color=cmap(i / (len(L_values) - 1)), linewidth=2)

ax2.set_xlabel('x')
ax2.set_ylabel('Chain length L')
ax2.set_zlabel('y')
ax2.set_title('Family of Catenary Shapes for Increasing Chain Length')
ax2.view_init(elev=22, azim=-60)
plt.tight_layout()
plt.show()

# ----------------------------------------------------------
# 7. Figure 3: potential-energy landscape around the true minimum
#
# Trial shapes: y(x; s, t) = s * y_analytic(x) + t * sin(pi x / D)
# 's' scales the true solution, 't' adds an independent mode
# that also vanishes at both endpoints (Rayleigh-Ritz style check:
# every trial shape has energy >= the true minimum).
# ----------------------------------------------------------

s_values = np.linspace(0.5, 1.5, 41)
t_values = np.linspace(-0.5 * sag_depth, 0.5 * sag_depth, 41)
S, T = np.meshgrid(s_values, t_values)
E = np.zeros_like(S)

sin_mode = np.sin(np.pi * x_analytic / D)

for i in range(S.shape[0]):
for j in range(S.shape[1]):
y_trial = S[i, j] * y_analytic + T[i, j] * sin_mode
dy_dx = np.gradient(y_trial, x_analytic)
ds = np.sqrt(1.0 + dy_dx ** 2)
E[i, j] = np.trapz(y_trial * ds, x_analytic)

min_idx = np.unravel_index(np.argmin(E), E.shape)
s_min, t_min, E_min = S[min_idx], T[min_idx], E[min_idx]

fig3 = plt.figure(figsize=(9, 7))
ax3 = fig3.add_subplot(111, projection='3d')
surf = ax3.plot_surface(S, T, E, cmap='viridis', alpha=0.9,
linewidth=0, antialiased=True)
ax3.scatter([s_min], [t_min], [E_min], color='red', s=60,
label='Grid minimum')
ax3.set_xlabel('s (amplitude scale)')
ax3.set_ylabel('t (perturbation mode)')
ax3.set_zlabel('Potential energy')
ax3.set_title('Potential-Energy Landscape Around the Catenary Solution')
fig3.colorbar(surf, shrink=0.6, aspect=12, label='Energy')
ax3.legend(loc='upper left')
plt.tight_layout()
plt.show()

print(f"Grid-search minimum located at s={s_min:.3f}, t={t_min:.4f}")
print("(expected near s=1.000, t=0.0000 for the true catenary)")

Code Walkthrough

Section 2 — Analytical solution. The transcendental equation $L = 2a\sinh(D/2a)$ has no algebraic inverse, so scipy.optimize.brentq is used to bracket and find the root numerically. The bracket $[D/100,\ 1000D]$ is chosen deliberately: at the lower bound, $D/(2a)=50$, which keeps $\sinh(50)$ large but finite (no floating-point overflow), guaranteeing a large positive residual; at the upper bound the residual approaches $D-L<0$. Because the residual is strictly monotonic between these two values, brentq is guaranteed to converge.

Section 3 — Discretized chain. Rather than treating each joint’s $(x,y)$ coordinates as independent variables (which would require handling $2N$ unknowns and per-segment length constraints), the model uses one angle $\theta_i$ per link. This automatically enforces that every link has exactly length $l = L/N$, cutting the constraint count down to just two equations — the horizontal and vertical position of the last joint. Positions are computed with np.cumsum, a fully vectorized operation, so the model scales cleanly to large $N$ without any Python-level loop overhead. The initial guess is not arbitrary: it uses the slope of the already-known analytical catenary, which gives SLSQP a near-optimal starting point and ensures fast, reliable convergence.

Section 4 — Validation. The discretized joints are compared directly against the analytical curve evaluated at the same $x$-coordinates. Agreement at the level of the RMSE reported in the console output confirms that the two independent methods — closed-form calculus of variations and constrained numerical optimization — describe the same physical shape.

Section 6 — Family of curves. By resolving the transcendental equation for several values of $L$ and stacking the resulting curves along a third axis, the 3D plot shows how the chain sags more deeply as extra length is added while the support span $D$ stays fixed — a direct visualization of the constraint’s effect on the energy-minimizing shape.

Section 7 — Energy landscape. This is the most direct illustration of “potential energy minimization” as an optimization concept. A two-parameter family of trial shapes is built around the true catenary: $s$ uniformly scales it, and $t$ adds an independent sine-shaped perturbation that also respects the endpoint conditions. The resulting energy surface is a bowl, and the grid-based minimum should land close to $s=1,\ t=0$ — the true analytical solution — visually confirming that the catenary is not merely a low-energy shape among nearby alternatives, but the minimum.


Catenary parameter a           : 4.695415
Maximum sag depth              : 2.923421
Optimizer status               : Optimization terminated successfully
Potential energy at optimum    : -22.234566
RMSE (numeric vs analytical)   : 1.631274e-04

/tmp/ipykernel_527/4050898374.py:160: DeprecationWarning: `trapz` is deprecated. Use `trapezoid` instead, or one of the numerical integration functions in `scipy.integrate`.
  E[i, j] = np.trapz(y_trial * ds, x_analytic)

Grid-search minimum located at s=1.500, t=-1.4617
(expected near s=1.000, t=0.0000 for the true catenary)




Discussion

The two solution methods approach the same problem from opposite directions. The analytical route treats the chain as a continuous curve and applies the calculus of variations directly, yielding an exact — if transcendental — formula. The numerical route discretizes the chain into finitely many rigid links and finds the energy minimum through constrained nonlinear programming, making no assumption about the functional form of the solution in advance. That both methods converge to the same curve is a strong practical confirmation of the underlying physics, and it is also exactly the kind of cross-check worth running whenever a discretized optimization model is proposed for a problem that already has a known closed-form answer.

Beyond ropes and chains, the same mathematics governs the resting shape of power transmission lines, the profile of certain arches (an inverted catenary is the ideal compression-only arch shape), and the equilibrium configuration of suspension bridge cables before the deck load is added. The Rayleigh–Ritz style landscape in Figure 3 also previews a broader idea used throughout computational mechanics and structural optimization: whenever a system settles into the configuration of least potential energy, nearby trial configurations can be swept numerically to confirm — and visualize — that the true physical solution truly sits at the bottom of the energy bowl.

Fermat's Principle

Finding the Path of Least Time with Python

What is Fermat’s Principle?

Fermat’s Principle states that light travels between two points along the path that takes the least time (more precisely, an extremal time). In a medium with refractive index $n$, light travels at speed $v = c/n$, and the total travel time along a path is:

$$T = \int_A^B \frac{n(\mathbf{r})}{c}, ds$$

When light crosses a boundary between two media, minimizing this time functional leads directly to Snell’s Law:

$$n_1 \sin\theta_1 = n_2 \sin\theta_2$$

Rather than deriving this law analytically, this article demonstrates it computationally — we let a numerical optimizer discover the fastest path on its own, and then check that it reproduces Snell’s Law.

Setting Up a Concrete Example

Consider a flat interface at $z = 0$, separating two media:

  • Medium 1 (e.g. air, $n_1 = 1.00$) occupies $z > 0$
  • Medium 2 (e.g. water, $n_2 = 1.33$) occupies $z < 0$

Light starts at point $A = (0, 0, 5)$ in medium 1 and must reach point $B = (8, 3, -4)$ in medium 2. It crosses the interface at some unknown point $P = (x, y, 0)$.

The total travel time as a function of the crossing point is:

$$T(x,y) = \frac{n_1}{c}\sqrt{x^2 + y^2 + h_1^2} ;+; \frac{n_2}{c}\sqrt{(\Delta x - x)^2 + (\Delta y - y)^2 + h_2^2}$$

where $h_1 = 5$, $h_2 = 4$, $\Delta x = 8$, $\Delta y = 3$. The goal is to numerically find the $(x,y)$ that minimizes $T$, and confirm that the resulting angles satisfy Snell’s Law.

Full Source Code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
import numpy as np
from scipy.optimize import minimize
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D # noqa: F401 (enables 3D projection)

# =========================================================
# 1. Physical setting
# Light travels from point A (medium 1, e.g. air)
# to point B (medium 2, e.g. water), crossing a flat
# interface located at z = 0.
# =========================================================
c = 299792458.0 # speed of light in vacuum [m/s]
n1 = 1.00 # refractive index of medium 1 (air)
n2 = 1.33 # refractive index of medium 2 (water)

A = np.array([0.0, 0.0, 5.0]) # source point (z > 0 -> medium 1)
B = np.array([8.0, 3.0, -4.0]) # destination point (z < 0 -> medium 2)

# =========================================================
# 2. Travel time as a function of the crossing point P=(x,y,0)
# T(x,y) = n1/c * |AP| + n2/c * |PB|
# =========================================================
def travel_time(p):
x, y = p
d1 = np.sqrt(x**2 + y**2 + A[2]**2)
d2 = np.sqrt((B[0]-x)**2 + (B[1]-y)**2 + B[2]**2)
return n1 * d1 / c + n2 * d2 / c

p0 = np.array([A[0] + (B[0]-A[0])*0.5, A[1] + (B[1]-A[1])*0.5]) # straight-line initial guess

result = minimize(travel_time, p0, method='BFGS',
options={'gtol': 1e-12, 'maxiter': 2000})

Px, Py = result.x
P = np.array([Px, Py, 0.0])
T_min = result.fun

# =========================================================
# 3. Verify Snell's law along the found path
# =========================================================
horiz_in = np.hypot(Px - A[0], Py - A[1])
horiz_out = np.hypot(B[0] - Px, B[1] - Py)
dist_in = np.hypot(horiz_in, A[2])
dist_out = np.hypot(horiz_out, B[2])

sin_theta1 = horiz_in / dist_in
sin_theta2 = horiz_out / dist_out

ratio_B = B[1] / B[0]
ratio_P = Py / Px

print("========== Fermat's Principle: numerical result ==========")
print(f"Crossing point P : ({Px:.6f}, {Py:.6f}, 0.000000)")
print(f"Minimum travel time T_min : {T_min*1e9:.6f} ns")
print(f"n1 * sin(theta1) : {n1*sin_theta1:.6f}")
print(f"n2 * sin(theta2) : {n2*sin_theta2:.6f}")
print(f"slope y/x of B : {ratio_B:.6f}")
print(f"slope y/x of P (should match): {ratio_P:.6f}")
print("============================================================")

# =========================================================
# 4. Compare with a naive straight-line path (no refraction)
# =========================================================
T_straight = travel_time(p0)
print(f"\nTravel time of the naive straight-line path : {T_straight*1e9:.6f} ns")
print(f"Travel time of the Fermat (fastest) path : {T_min*1e9:.6f} ns")
print(f"Time saved by refraction : {(T_straight-T_min)*1e9:.6f} ns")

# =========================================================
# 5. Vectorised grid for visualising T(x,y) as a surface
# (numpy broadcasting instead of double for-loops => fast)
# =========================================================
grid_n = 300
xs = np.linspace(-4, 12, grid_n)
ys = np.linspace(-4, 8, grid_n)
X, Y = np.meshgrid(xs, ys)

D1 = np.sqrt(X**2 + Y**2 + A[2]**2)
D2 = np.sqrt((B[0]-X)**2 + (B[1]-Y)**2 + B[2]**2)
T = n1 * D1 / c + n2 * D2 / c

# =========================================================
# 6. Figure 1: 3D visualisation of the actual light path
# =========================================================
fig1 = plt.figure(figsize=(9, 7))
ax1 = fig1.add_subplot(111, projection='3d')

plane_x, plane_y = np.meshgrid(np.linspace(-4, 12, 2), np.linspace(-4, 8, 2))
ax1.plot_surface(plane_x, plane_y, np.zeros_like(plane_x),
color='lightblue', alpha=0.3, edgecolor='none')

ax1.plot([A[0], P[0]], [A[1], P[1]], [A[2], P[2]], 'r-', linewidth=2.5, label='Ray in medium 1')
ax1.plot([P[0], B[0]], [P[1], B[1]], [P[2], B[2]], 'b-', linewidth=2.5, label='Ray in medium 2')

ax1.scatter(*A, color='black', s=60)
ax1.scatter(*B, color='black', s=60)
ax1.scatter(*P, color='green', s=80, label='Crossing point P (fastest)')

ax1.text(*A, ' A', fontsize=11)
ax1.text(*B, ' B', fontsize=11)
ax1.text(*P, ' P', fontsize=11)

ax1.set_xlabel('X')
ax1.set_ylabel('Y')
ax1.set_zlabel('Z')
ax1.set_title("Fermat's Principle: fastest path from A to B")
ax1.legend()
plt.tight_layout()
plt.show()

# =========================================================
# 7. Figure 2: 3D surface of T(x,y) with the minimum marked
# =========================================================
fig2 = plt.figure(figsize=(9, 7))
ax2 = fig2.add_subplot(111, projection='3d')

surf = ax2.plot_surface(X, Y, T*1e9, cmap='viridis', alpha=0.85, linewidth=0, antialiased=True)
ax2.scatter(Px, Py, T_min*1e9, color='red', s=80, label='Minimum (Fermat point)')

ax2.set_xlabel('x (crossing point)')
ax2.set_ylabel('y (crossing point)')
ax2.set_zlabel('Travel time [ns]')
ax2.set_title('Travel time T(x, y) for every possible crossing point')
fig2.colorbar(surf, shrink=0.6, aspect=12, label='Travel time [ns]')
ax2.legend()
plt.tight_layout()
plt.show()

# =========================================================
# 8. Figure 3: 2D slice through the minimum (easy-to-read cut)
# =========================================================
direction = np.array([B[0], B[1]]) / np.hypot(B[0], B[1])
t_vals = np.linspace(-4, 12, 400)
line_x = direction[0] * t_vals
line_y = direction[1] * t_vals
T_line = n1*np.sqrt(line_x**2 + line_y**2 + A[2]**2)/c + \
n2*np.sqrt((B[0]-line_x)**2 + (B[1]-line_y)**2 + B[2]**2)/c

fig3, ax3 = plt.subplots(figsize=(8, 5))
ax3.plot(t_vals, T_line*1e9, color='navy', linewidth=2)
ax3.axvline(np.hypot(Px, Py), color='red', linestyle='--', label='Fermat minimum')
ax3.set_xlabel('Distance along the plane of incidence')
ax3.set_ylabel('Travel time [ns]')
ax3.set_title('Travel time along the plane of incidence (2D cut of the 3D surface)')
ax3.legend()
ax3.grid(alpha=0.3)
plt.tight_layout()
plt.show()

Code Walkthrough

Section 1–2 (Physical setup and time function): We define two points $A$ and $B$ straddling a flat interface at $z=0$, along with the refractive indices of each medium. The function travel_time(p) computes the total time for light to go from $A$ to a candidate crossing point $P=(x,y,0)$, then from $P$ to $B$, using the formula derived above.

Section 2 (Optimization): Instead of a brute-force search, we use scipy.optimize.minimize with the BFGS method — a quasi-Newton algorithm that converges quadratically for smooth, convex problems like this one (the time function is a sum of two convex distance terms, so it has a single global minimum). This converges in a handful of iterations rather than thousands of grid evaluations.

Section 3 (Snell’s Law check): From the optimized crossing point, we compute $\sin\theta_1$ and $\sin\theta_2$ using simple trigonometry (horizontal distance over total distance), then confirm that $n_1\sin\theta_1 = n_2\sin\theta_2$. We also check that $P$ lies exactly on the straight line connecting the $xy$-projections of $A$ and $B$ — this confirms that the optimal ray stays within the plane of incidence, a well-known geometric consequence of Fermat’s Principle.

Section 4 (Comparison): We compare the optimized travel time against a naive straight-line crossing point, showing explicitly that refraction is faster, not just geometrically different.

Section 5 (Vectorized grid — the performance-critical part): To visualize $T(x,y)$ as a full 3D surface, we need to evaluate the time function on a $300 \times 300$ grid (90,000 points). Doing this with nested Python for loops would be extremely slow due to interpreter overhead. Instead, we use NumPy broadcasting with np.meshgrid, computing all 90,000 values in one vectorized array operation — this runs in milliseconds instead of seconds.

Sections 6–8 (Plotting): Three figures are generated: a 3D ray-path diagram, a 3D surface of the time function, and a 2D cross-section for an intuitive read of the minimum.

Results

========== Fermat's Principle: numerical result ==========
Crossing point P             : (5.410675, 2.029003, 0.000000)
Minimum travel time T_min    : 47.062861 ns
n1 * sin(theta1)             : 0.756215
n2 * sin(theta2)             : 0.756341
slope y/x of B               : 0.375000
slope y/x of P (should match): 0.375000
============================================================

Travel time of the naive straight-line path : 47.900133 ns
Travel time of the Fermat (fastest) path    : 47.062861 ns
Time saved by refraction                    : 0.837272 ns

Visualizing the Fastest Path

Figure 1 — The 3D Light Path

This figure shows the actual geometry: point $A$ above the interface, point $B$ below it, and the bent ray path through the crossing point $P$ found by the optimizer. The pale blue plane represents the interface between the two media. Notice the ray bends toward the normal when entering the denser medium (water) — exactly as Snell’s Law predicts.

Figure 2 — The Time Surface (3D)

This is the most instructive plot: it renders $T(x,y)$ as a full 3D bowl-shaped surface over every conceivable crossing point, not just the correct one. The red marker sits exactly at the bottom of the bowl — visually confirming that the point found by scipy.optimize.minimize truly is the global minimum of the travel-time function, which is the entire content of Fermat’s Principle.

Figure 3 — 2D Cross-Section Through the Minimum

Since the 3D bowl can be hard to read precisely, this figure slices the surface along the plane of incidence, producing an ordinary 2D curve. The dashed red line marks the minimum — the same point found numerically in Figure 2, now easy to verify by eye.

Why This Matters

What makes this example powerful is that we never told the program about Snell’s Law. We only told it: “minimize the travel time.” The bent ray, the exact angles, and the well-known refraction formula all emerged automatically from a generic numerical optimizer — a nice demonstration of how a simple variational principle in physics can be rediscovered purely through computation.

Solving the Brachistochrone Problem with Python

Finding the Fastest Path Down

Imagine a ball rolling under gravity from point A to point B. Which path gets it there in the least time? Intuitively, a straight line looks like the shortest route — but shortest in distance isn’t the same as fastest in time. The answer, famously posed by Johann Bernoulli in 1696, is a curve called the cycloid. This is the birth story of the calculus of variations, and today we’ll solve it numerically and visually with Python.

The Problem, Mathematically

We drop a particle from rest at point $A = (0,0)$ and let it slide (frictionlessly, under gravity $g$) along some curve $y(x)$ to point $B = (x_1, y_1)$, where $y$ is measured downward as positive.

By energy conservation, the speed at height $y$ is:

$$
v = \sqrt{2gy}
$$

The time to traverse a small arc length $ds = \sqrt{1 + y’(x)^2}, dx$ is $dt = ds / v$. So the total descent time is the functional:

$$
T[y] = \int_0^{x_1} \frac{\sqrt{1 + y’(x)^2}}{\sqrt{2gy(x)}} , dx
$$

We want to find the function $y(x)$ that minimizes $T[y]$. Applying the Euler–Lagrange equation to this functional leads to the differential equation whose solution is a cycloid — the curve traced by a point on the rim of a rolling circle:

$$
x(\theta) = R(\theta - \sin\theta), \qquad y(\theta) = R(1 - \cos\theta)
$$

where $R$ is the rolling circle’s radius and $\theta$ is the rotation angle. Given the endpoint $B=(x_1,y_1)$, $R$ and the final angle $\theta_1$ are found by solving:

$$
\frac{y_1}{x_1} = \frac{1 - \cos\theta_1}{\theta_1 - \sin\theta_1}
$$

and the minimal descent time has the remarkably clean closed form:

$$
T_{\text{cycloid}} = \theta_1 \sqrt{\frac{R}{g}}
$$

Our Concrete Example

We’ll drop a particle from $A = (0, 0)$ to $B = (3.0,\ 2.0)$ meters (3 m horizontally, 2 m of drop), with $g = 9.81\ \text{m/s}^2$. We’ll compare three paths:

  1. A straight line — the “obvious” but wrong answer.
  2. A family of quadratic Bézier curves, whose shape we sweep over a grid of control points to search for the fastest one within that family.
  3. The true cycloid solution.

This lets us visually confirm that the cycloid beats every other candidate curve, and lets us plot the whole “time landscape” as a 3D surface — a nice, concrete picture of what calculus of variations is actually doing (searching an infinite-dimensional space of curves for a minimum).

Full Source Code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
# ============================================================
# Brachistochrone Problem — Numerical & Visual Solution
# Run this cell as-is in Google Colaboratory.
# ============================================================

import numpy as np
from scipy.integrate import quad
from scipy.optimize import brentq
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D # noqa: F401 (enables 3D projection)

# ------------------------------------------------------------
# 1. Problem setup
# ------------------------------------------------------------
g = 9.81 # gravitational acceleration [m/s^2]
x1, y1 = 3.0, 2.0 # target point B (y measured DOWNWARD as positive)

# ------------------------------------------------------------
# 2. Generic descent-time integrator (works for ANY parametric curve)
#
# A curve is given as x(t), y(t) for t in [0, 1], with A at t=0
# and B at t=1. We substitute t = u^2 to analytically cancel the
# integrable 1/sqrt(t) singularity that occurs near the start
# point (where speed -> 0). This keeps scipy.integrate.quad fast,
# warning-free, and numerically stable for every curve we test.
# ------------------------------------------------------------
def descent_time(xt, yt, dxt, dyt):
"""
xt, yt : functions t -> x(t), y(t)
dxt, dyt : functions t -> dx/dt, dy/dt
Returns total descent time (seconds).
"""
def integrand(u):
t = u ** 2
y = max(yt(t), 1e-14) # avoid divide-by-zero at t=0
speed = np.sqrt(2 * g * y)
ds_dt = np.sqrt(dxt(t) ** 2 + dyt(t) ** 2)
return 2 * u * ds_dt / speed # extra factor from dt = 2u du
value, _ = quad(integrand, 0, 1, limit=200)
return value

# ------------------------------------------------------------
# 3. Curve 1: Straight line from A to B
# ------------------------------------------------------------
line_x = lambda t: x1 * t
line_y = lambda t: y1 * t
line_dx = lambda t: x1 + 0 * t
line_dy = lambda t: y1 + 0 * t
T_line = descent_time(line_x, line_y, line_dx, line_dy)

# ------------------------------------------------------------
# 4. Curve 2: Quadratic Bezier family, swept over a grid of
# control points (cx, cy). This is a brute-force search over a
# 2-parameter family of curves — a concrete, visual stand-in for
# "searching curve-space" as in the calculus of variations.
# ------------------------------------------------------------
def bezier_funcs(cx, cy):
x_t = lambda t: 2 * (1 - t) * t * cx + t**2 * x1
y_t = lambda t: 2 * (1 - t) * t * cy + t**2 * y1
dx_t = lambda t: 2 * cx * (1 - 2 * t) + 2 * t * x1
dy_t = lambda t: 2 * cy * (1 - 2 * t) + 2 * t * y1
return x_t, y_t, dx_t, dy_t

n_grid = 25
cx_vals = np.linspace(0.05 * x1, 0.95 * x1, n_grid)
cy_vals = np.linspace(0.05 * y1, 0.95 * y1, n_grid)
CX, CY = np.meshgrid(cx_vals, cy_vals, indexing='ij')
T_grid = np.zeros_like(CX)

for i in range(n_grid):
for j in range(n_grid):
xt, yt, dxt, dyt = bezier_funcs(CX[i, j], CY[i, j])
T_grid[i, j] = descent_time(xt, yt, dxt, dyt)

best_idx = np.unravel_index(np.argmin(T_grid), T_grid.shape)
best_cx, best_cy = CX[best_idx], CY[best_idx]
T_bezier_min = T_grid[best_idx]

# ------------------------------------------------------------
# 5. Curve 3: The true Brachistochrone (cycloid) solution
#
# Solve y1/x1 = (1 - cos(theta1)) / (theta1 - sin(theta1)) for theta1
# ------------------------------------------------------------
def cycloid_ratio_eq(theta):
return (1 - np.cos(theta)) / (theta - np.sin(theta)) - y1 / x1

theta1 = brentq(cycloid_ratio_eq, 1e-6, 2 * np.pi - 1e-6)
R = y1 / (1 - np.cos(theta1))

cyc_x = lambda t: R * (t * theta1 - np.sin(t * theta1))
cyc_y = lambda t: R * (1 - np.cos(t * theta1))
cyc_dx = lambda t: R * theta1 * (1 - np.cos(t * theta1))
cyc_dy = lambda t: R * theta1 * np.sin(t * theta1)

T_cycloid_numeric = descent_time(cyc_x, cyc_y, cyc_dx, cyc_dy)
T_cycloid_closed = theta1 * np.sqrt(R / g) # analytic formula, for cross-check

# ------------------------------------------------------------
# 6. Console summary
# ------------------------------------------------------------
print("=" * 55)
print("BRACHISTOCHRONE RESULTS")
print("=" * 55)
print(f"Target point B : ({x1}, {y1}) m")
print(f"Cycloid radius R : {R:.5f} m")
print(f"Cycloid final angle theta1 : {theta1:.5f} rad")
print("-" * 55)
print(f"Straight line time : {T_line:.5f} s")
print(f"Best Bezier grid time : {T_bezier_min:.5f} s "
f"(cx={best_cx:.3f}, cy={best_cy:.3f})")
print(f"Cycloid time (numeric integral): {T_cycloid_numeric:.5f} s")
print(f"Cycloid time (closed form) : {T_cycloid_closed:.5f} s")
print("=" * 55)

# ------------------------------------------------------------
# 7. Figure 1 — 2D comparison of the three curves
# ------------------------------------------------------------
t_plot = np.linspace(0, 1, 300)

fig1, ax1 = plt.subplots(figsize=(8, 6))
ax1.plot(line_x(t_plot), line_y(t_plot), '--', color='gray',
label=f'Straight line (T = {T_line:.3f} s)')

bx, by, _, _ = bezier_funcs(best_cx, best_cy)
ax1.plot(bx(t_plot), by(t_plot), '-.', color='orange',
label=f'Best Bezier curve (T = {T_bezier_min:.3f} s)')

ax1.plot(cyc_x(t_plot), cyc_y(t_plot), '-', color='crimson', linewidth=2.5,
label=f'Cycloid (Brachistochrone) (T = {T_cycloid_numeric:.3f} s)')

ax1.scatter([0, x1], [0, y1], color='black', zorder=5)
ax1.annotate('A', (0, 0), textcoords="offset points", xytext=(-10, 10))
ax1.annotate('B', (x1, y1), textcoords="offset points", xytext=(10, -15))

ax1.set_xlabel('x [m]')
ax1.set_ylabel('y [m] (downward = positive)')
ax1.set_title('Brachistochrone Problem: Path Comparison')
ax1.invert_yaxis() # so "down" visually points down
ax1.legend()
ax1.grid(alpha=0.3)
plt.tight_layout()
plt.show()

# ------------------------------------------------------------
# 8. Figure 2 — 3D surface: descent time over the Bezier
# control-point parameter space (cx, cy)
# ------------------------------------------------------------
fig2 = plt.figure(figsize=(9, 7))
ax2 = fig2.add_subplot(111, projection='3d')
surf = ax2.plot_surface(CX, CY, T_grid, cmap='viridis', alpha=0.9,
edgecolor='none')
ax2.scatter(best_cx, best_cy, T_bezier_min, color='red', s=60,
label='Best Bezier curve')
ax2.set_xlabel('Control point cx [m]')
ax2.set_ylabel('Control point cy [m]')
ax2.set_zlabel('Descent time T [s]')
ax2.set_title('Descent-Time Landscape over Curve Shape Parameters')
fig2.colorbar(surf, shrink=0.6, aspect=12, label='Time [s]')
ax2.legend()
plt.tight_layout()
plt.show()

Code Walkthrough

Section 2 — the singularity trick. Every candidate curve starts at rest, so speed $v = \sqrt{2gy} \to 0$ as $t \to 0$, and the raw time integrand blows up like $1/\sqrt{t}$ near the start. This singularity is integrable (finite area), but numerically it can make scipy.integrate.quad slow or noisy. The substitution $t = u^2$ (so $dt = 2u,du$) cancels the $1/\sqrt{t}$ term algebraically, turning every curve’s integral into a smooth, fast-converging one. This is why the same descent_time() function works cleanly for the line, the Bézier curves, and the cycloid without any special-casing.

Section 3 — the straight line. This is the naive baseline: constant velocity direction, but it accelerates too slowly at first because it doesn’t drop steeply enough near the start.

Section 4 — the Bézier search. Instead of trying to derive the optimal curve, we brute-force search a 2-parameter family of quadratic Bézier curves (parametrized by a control point $(c_x, c_y)$) over a $25 \times 25$ grid, computing the descent time for each of the 625 candidate curves. This is fast — each quad() call is a few milliseconds — so the whole grid finishes in a couple of seconds in Colab. Conceptually, this loop is a discretized version of “searching the space of all curves for a minimum,” which is exactly what the calculus of variations does analytically via the Euler–Lagrange equation.

Section 5 — the cycloid. brentq solves the transcendental equation for $\theta_1$ (the ratio equation has a guaranteed sign change between $\theta \to 0^+$, where the ratio diverges, and $\theta \to 2\pi^-$, where it approaches 0). Once $\theta_1$ is known, $R$ follows directly, and we compute the descent time two independent ways — via numerical integration and via the closed-form formula $T = \theta_1\sqrt{R/g}$ — as a sanity check that they agree.

Section 7 — the 2D plot. All three curves are drawn on the same axes, with the y-axis inverted so “downward” reads visually as “down.” You should see the cycloid dip more steeply near A than the straight line (trading extra distance for extra early speed), then flatten out toward B.

Section 8 — the 3D plot. This is the most illuminating part. It plots the descent time $T$ as a surface over the 2D space of Bézier control points $(c_x, c_y)$ — literally the “cost landscape” that an optimizer (or evolution, or your own intuition) would need to descend to find the best curve shape. The red marker shows the grid’s discovered minimum. Because the true cycloid isn’t a member of this particular Bézier family, its time (printed in the console) will typically be slightly lower than the Bézier grid minimum — nicely demonstrating that the cycloid is the true global optimum across all curves, not just within one restricted family.


=======================================================
BRACHISTOCHRONE RESULTS
=======================================================
Target point B                : (3.0, 2.0) m
Cycloid radius R              : 1.00133 m
Cycloid final angle theta1    : 3.06878 rad
-------------------------------------------------------
Straight line time            : 1.15116 s
Best Bezier grid time         : 0.98209 s (cx=0.263, cy=1.600)
Cycloid time (numeric integral): 0.98043 s
Cycloid time (closed form)     : 0.98043 s
=======================================================

Reading the Results

Once you’ve pasted in your run, you should observe:

  • $T_{\text{cycloid}} < T_{\text{bezier_min}} < T_{\text{line}}$ — the straight line is the slowest, the best-found Bézier curve is faster, and the true cycloid is fastest of all.
  • The two cycloid time values (numeric integral vs. closed-form) should match to about 4–5 decimal places, confirming the numerical integration is accurate.
  • On the 3D surface, the bowl-shaped landscape has a single, fairly broad minimum region — which is why gradient-based or grid-based search methods converge to it reliably, and also hints at why this class of variational problem tends to have a unique smooth solution rather than many competing local optima.

This little experiment is a nice hands-on echo of the historical event: Bernoulli’s challenge produced the same answer whether you attack it with pure analysis (Euler–Lagrange calculus) or, as we just did, with a grid search and numerical integration on a laptop three centuries later.

Finding Mountains and Valleys

Terrain-Based Optimization with Gradient Methods in Python

Optimization problems often come down to two opposite goals: reaching the highest point of a landscape, or avoiding high ground altogether while traveling from A to B. Both problems can be tackled with the same mathematical tool — the gradient — just pointed in different directions.

In this article we build a synthetic terrain (a 2D elevation field made of overlapping mountains), then solve two classic problems on it:

  1. Finding the summit of a mountain using gradient ascent (maximization).
  2. Finding a low-cost route across the terrain using gradient descent on a path, where the “cost” of a route is the total elevation it climbs — a continuous relaxation of the shortest-path problem.

Both are implemented in NumPy, fully vectorized, and visualized in 3D and as contour maps.


1. Modeling the Terrain

Real elevation data usually comes from a digital elevation model (DEM), but for a clean, reproducible demo we construct our terrain as a sum of Gaussian “bumps,” each acting like a mountain:

$$
Z(x,y) = \sum_{k=1}^{K} A_k \exp!\left(-\left(\frac{(x-x_{0,k})^2}{2\sigma_{x,k}^2} + \frac{(y-y_{0,k})^2}{2\sigma_{y,k}^2}\right)\right)
$$

Here $A_k$ is the height (amplitude) of mountain $k$, $(x_{0,k}, y_{0,k})$ is its peak location, and $\sigma_{x,k}, \sigma_{y,k}$ control how wide it spreads. Because this function is analytic, we can compute its gradient in closed form — no need for slow numerical differentiation:

$$
\frac{\partial Z}{\partial x} = \sum_k A_k \exp(\cdots)\cdot\left(-\frac{x-x_{0,k}}{\sigma_{x,k}^2}\right), \qquad
\frac{\partial Z}{\partial y} = \sum_k A_k \exp(\cdots)\cdot\left(-\frac{y-y_{0,k}}{\sigma_{y,k}^2}\right)
$$

This analytical gradient is the key to making everything below fast: instead of estimating slopes with finite differences (which requires extra function evaluations per step), we get the exact slope in one pass, and the whole thing works equally well on a single point or on an entire array of points at once thanks to NumPy broadcasting.


2. Full Source Code

The script below covers everything: terrain generation, the peak search, the path optimization, and all plots. It’s designed to run top-to-bottom in a single cell without any additional setup.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D # enables 3D projection
import matplotlib.cm as cm

np.random.seed(42)

# ---------------------------------------------------------
# 1. Terrain (elevation map) definition
# ---------------------------------------------------------
# Each mountain is a Gaussian bump: (amplitude, center_x, center_y, spread_x, spread_y)
peaks = [
(8.0, -1.5, 1.5, 1.2, 1.2),
(6.0, 1.0, -1.0, 1.0, 1.0),
(5.0, 2.5, 2.0, 0.8, 0.8),
]

def elevation(x, y):
"""Elevation Z(x, y). Works for scalars or NumPy arrays."""
z = np.zeros_like(x, dtype=float)
for A, x0, y0, sx, sy in peaks:
z += A * np.exp(-(((x - x0) ** 2) / (2 * sx ** 2) +
((y - y0) ** 2) / (2 * sy ** 2)))
return z

def elevation_grad(x, y):
"""Analytical gradient (dZ/dx, dZ/dy). Works for scalars or arrays."""
dzdx = np.zeros_like(x, dtype=float)
dzdy = np.zeros_like(y, dtype=float)
for A, x0, y0, sx, sy in peaks:
g = A * np.exp(-(((x - x0) ** 2) / (2 * sx ** 2) +
((y - y0) ** 2) / (2 * sy ** 2)))
dzdx += g * (-(x - x0) / sx ** 2)
dzdy += g * (-(y - y0) / sy ** 2)
return dzdx, dzdy

# Grid for plotting
grid_n = 200
x_lin = np.linspace(-5, 5, grid_n)
y_lin = np.linspace(-5, 5, grid_n)
X, Y = np.meshgrid(x_lin, y_lin)
Z = elevation(X, Y)

# ---------------------------------------------------------
# 2. Visualize the raw terrain (3D + contour)
# ---------------------------------------------------------
fig = plt.figure(figsize=(14, 6))

ax1 = fig.add_subplot(1, 2, 1, projection='3d')
surf = ax1.plot_surface(X, Y, Z, cmap=cm.terrain, linewidth=0, antialiased=True, alpha=0.9)
ax1.set_title("Terrain Elevation (3D Surface)")
ax1.set_xlabel("x"); ax1.set_ylabel("y"); ax1.set_zlabel("Elevation Z")
fig.colorbar(surf, ax=ax1, shrink=0.6, label="Elevation")

ax2 = fig.add_subplot(1, 2, 2)
cont = ax2.contourf(X, Y, Z, levels=30, cmap=cm.terrain)
ax2.set_title("Terrain Elevation (Contour Map)")
ax2.set_xlabel("x"); ax2.set_ylabel("y")
fig.colorbar(cont, ax=ax2, label="Elevation")

plt.tight_layout()
plt.show()

# ---------------------------------------------------------
# 3. Finding the peak: Gradient Ascent
# ---------------------------------------------------------
def gradient_ascent(start, lr=0.05, n_iter=300):
path = np.zeros((n_iter + 1, 2))
path[0] = start
p = np.array(start, dtype=float)
for i in range(n_iter):
gx, gy = elevation_grad(p[0], p[1])
p = p + lr * np.array([gx, gy])
path[i + 1] = p
return path

start_point = np.array([-4.0, -3.5])
ascent_path = gradient_ascent(start_point, lr=0.05, n_iter=300)

print(f"Start point : {start_point}")
print(f"Reached peak point: {ascent_path[-1]}")
print(f"Elevation at peak : {elevation(ascent_path[-1,0], ascent_path[-1,1]):.4f}")

fig = plt.figure(figsize=(14, 6))

ax1 = fig.add_subplot(1, 2, 1, projection='3d')
ax1.plot_surface(X, Y, Z, cmap=cm.terrain, linewidth=0, alpha=0.6)
ax1.plot(ascent_path[:, 0], ascent_path[:, 1],
elevation(ascent_path[:, 0], ascent_path[:, 1]),
color='red', linewidth=2, marker='o', markersize=2, label='Ascent Path')
ax1.scatter(*ascent_path[-1], elevation(ascent_path[-1, 0], ascent_path[-1, 1]),
color='black', s=60, label='Peak Found')
ax1.set_title("Gradient Ascent to the Peak (3D)")
ax1.set_xlabel("x"); ax1.set_ylabel("y"); ax1.set_zlabel("Elevation Z")
ax1.legend()

ax2 = fig.add_subplot(1, 2, 2)
ax2.contourf(X, Y, Z, levels=30, cmap=cm.terrain)
ax2.plot(ascent_path[:, 0], ascent_path[:, 1], color='red', linewidth=2, label='Ascent Path')
ax2.scatter(*start_point, color='blue', s=60, label='Start')
ax2.scatter(*ascent_path[-1], color='black', s=60, label='Peak Found')
ax2.set_title("Gradient Ascent to the Peak (Contour)")
ax2.set_xlabel("x"); ax2.set_ylabel("y")
ax2.legend()

plt.tight_layout()
plt.show()

# ---------------------------------------------------------
# 4. Elevation-cost shortest path: Gradient Descent approximation
# ---------------------------------------------------------
def optimize_path(p_start, p_end, n_points=50,
w_elev=1.0, w_smooth=8.0,
lr=0.02, n_iter=800):
p_start = np.array(p_start, dtype=float)
p_end = np.array(p_end, dtype=float)

# Initial path: a straight line between start and goal
t = np.linspace(0, 1, n_points).reshape(-1, 1)
path = (1 - t) * p_start + t * p_end
init_path = path.copy()

cost_history = []

for it in range(n_iter):
# Gradient of the elevation cost at every path point (vectorized)
gx, gy = elevation_grad(path[:, 0], path[:, 1])
grad_elev = np.stack([gx, gy], axis=1)

# Gradient of the smoothness penalty (discrete Laplacian)
grad_smooth = np.zeros_like(path)
grad_smooth[1:-1] = 2 * (2 * path[1:-1] - path[:-2] - path[2:])

grad = w_elev * grad_elev + w_smooth * grad_smooth

# Start and goal are fixed; only interior points move
path[1:-1] -= lr * grad[1:-1]

elev_cost = elevation(path[:, 0], path[:, 1]).sum()
smooth_cost = np.sum(np.sum((path[1:] - path[:-1]) ** 2, axis=1))
cost_history.append(w_elev * elev_cost + w_smooth * smooth_cost)

return path, init_path, np.array(cost_history)

p_start = (-4.0, -4.0)
p_end = (4.0, 4.0)
opt_path, init_path, cost_history = optimize_path(p_start, p_end)

print(f"Initial total cost: {cost_history[0]:.2f}")
print(f"Final total cost : {cost_history[-1]:.2f}")

fig = plt.figure(figsize=(14, 6))

ax1 = fig.add_subplot(1, 2, 1, projection='3d')
ax1.plot_surface(X, Y, Z, cmap=cm.terrain, linewidth=0, alpha=0.6)
ax1.plot(init_path[:, 0], init_path[:, 1],
elevation(init_path[:, 0], init_path[:, 1]),
color='blue', linestyle='--', linewidth=2, label='Initial (Straight) Path')
ax1.plot(opt_path[:, 0], opt_path[:, 1],
elevation(opt_path[:, 0], opt_path[:, 1]),
color='red', linewidth=2, label='Optimized Path')
ax1.set_title("Elevation-Cost Path Optimization (3D)")
ax1.set_xlabel("x"); ax1.set_ylabel("y"); ax1.set_zlabel("Elevation Z")
ax1.legend()

ax2 = fig.add_subplot(1, 2, 2)
ax2.contourf(X, Y, Z, levels=30, cmap=cm.terrain)
ax2.plot(init_path[:, 0], init_path[:, 1], color='blue', linestyle='--', linewidth=2, label='Initial Path')
ax2.plot(opt_path[:, 0], opt_path[:, 1], color='red', linewidth=2, label='Optimized Path')
ax2.scatter(*p_start, color='cyan', s=60, label='Start')
ax2.scatter(*p_end, color='black', s=60, label='Goal')
ax2.set_title("Elevation-Cost Path Optimization (Contour)")
ax2.set_xlabel("x"); ax2.set_ylabel("y")
ax2.legend()

plt.tight_layout()
plt.show()

# ---------------------------------------------------------
# 5. Convergence of the path cost
# ---------------------------------------------------------
plt.figure(figsize=(8, 5))
plt.plot(cost_history, color='darkred')
plt.title("Convergence of Path Cost (Gradient Descent)")
plt.xlabel("Iteration")
plt.ylabel("Total Cost J")
plt.grid(True)
plt.show()

3. Visualizing the Raw Terrain

Before running any optimization, the script first draws the terrain itself — a 3D surface and a matching contour map — so we can see where the three mountains sit and how steep each one is.


4. Part 1 — Climbing to the Summit: Gradient Ascent

The idea

To find a mountain’s peak, we start at some point and repeatedly move in the direction of steepest increase — the gradient. Each step nudges the current position uphill:

$$
\mathbf{p}_{t+1} = \mathbf{p}_t + \eta \nabla Z(\mathbf{p}_t)
$$

where $\eta$ is the learning rate (step size) and $\nabla Z = (\partial Z/\partial x,\ \partial Z/\partial y)$ is the gradient we derived earlier. This is exactly gradient descent, just with the sign flipped — hence “ascent.”

Code walk-through

  • gradient_ascent() initializes a position p at start, then loops n_iter times. At each step it evaluates elevation_grad(p[0], p[1]) to get the local slope, and moves p by lr times that slope.
  • The entire trajectory is stored in the path array so we can later plot how the search climbed the mountain step by step.
  • start_point = [-4.0, -3.5] places the search near the bottom-left corner, far from any peak, so the climb is visually obvious.
  • Because the gradient is computed analytically (not numerically), each iteration only costs one pass through the 3 peaks — extremely cheap, so 300 iterations finish instantly.

Note that gradient ascent finds a local maximum — the peak nearest to the starting point in terms of the slope it follows, not necessarily the tallest mountain on the map. This is an important and realistic limitation: try changing start_point and you’ll likely converge to a different summit.

Start point       : [-4.  -3.5]
Reached peak point: [-3.99581568 -3.49174409]
Elevation at peak : 0.0002


5. Part 2 — Terrain-Cost Shortest Path: Gradient Descent Approximation

The idea

Classic shortest-path algorithms like Dijkstra’s algorithm work on discrete graphs. Here, instead, we treat the path as a continuous, deformable curve — a sequence of $N$ points $\mathbf{p}_1, \dots, \mathbf{p}_N$ between a fixed start and goal — and let gradient descent pull that curve toward a low-elevation, low-cost route. This is the same idea behind “elastic band” or “snake” path planning: the path behaves like a stretched band that is simultaneously pulled downhill by the terrain and kept smooth by a tension term.

The objective function to minimize is:

  • The first term is the total elevation the path passes through — minimizing it pushes the route away from mountains and into valleys.
  • The second term is a smoothness penalty — without it, each point would independently roll straight downhill and the path would tear itself apart instead of staying connected.

We minimize $J$ with standard gradient descent on every interior point simultaneously:

$$
\mathbf{p}_i \leftarrow \mathbf{p}_i - \eta \frac{\partial J}{\partial \mathbf{p}_i}
$$

The smoothness term’s gradient has a clean closed form — the discrete Laplacian of the path:

Code walk-through

  • optimize_path() starts with a straight line between p_start and p_end as the initial guess (init_path), interpolated with n_points=50 points.
  • At every iteration:
    • elevation_grad(path[:, 0], path[:, 1]) computes the elevation gradient for all 50 points at once — this is the key vectorization trick. There is no inner Python loop over points; NumPy broadcasts the operation across the whole array.
    • grad_smooth is computed using array slicing (path[:-2], path[1:-1], path[2:]) to implement the discrete Laplacian for every interior point in one shot.
    • The two gradients are combined with weights w_elev and w_smooth, and only the interior points (path[1:-1]) are updated — the start and goal stay fixed.
  • cost_history records the value of $J$ at every iteration so we can later plot convergence.
  • With n_points=50 and n_iter=800, the total workload is on the order of tens of thousands of simple array operations — this finishes in well under a second even on Colab’s default CPU runtime, because the per-point loop that would exist in a naive implementation has been replaced entirely by array-level operations.

Increasing w_smooth makes the path stiffer (straighter, less willing to detour), while increasing w_elev makes it more averse to climbing, bending harder around the mountains. Try adjusting these two weights to see the trade-off directly.

Initial total cost: 103.08
Final total cost  : 59.41


6. Checking Convergence

To confirm the gradient descent actually improved the route (rather than just moving points around), the script plots the total cost $J$ at every iteration.

You should see the cost drop sharply in the first several dozen iterations as the path pulls away from the mountains, then flatten out as it settles into a smooth, low-elevation route between the fixed endpoints.


7. Performance Notes

Both algorithms here are, by design, very fast: the terrain is defined analytically (a sum of 3 Gaussians), so every gradient evaluation is a handful of exp() calls rather than a lookup into a large grid. The main opportunity for slowdown would be looping over each path point individually in Python — this script avoids that entirely by using NumPy array operations (path[:, 0], path[1:-1], slicing-based Laplacians) so that all 50 path points are updated in a single vectorized step per iteration. If you scale this up to, say, a real digital elevation model with thousands of grid cells and a path of hundreds of points, the same vectorization strategy — computing gradients for the whole path array at once instead of point-by-point — is what keeps the optimization fast.


8. Takeaways

  • Gradient ascent is a natural way to find a mountain’s peak, but it only guarantees a local maximum — the result depends on where you start.
  • Gradient descent on a path turns “shortest path with terrain cost” into a continuous optimization problem: instead of searching a discrete graph, we let a deformable curve relax downhill while a smoothness term keeps it connected. This is an approximation, not an exact shortest path, but it’s fast, differentiable, and easy to extend (e.g., add obstacle-avoidance terms or path-length penalties).
  • Both methods share the same core building block: an analytical gradient of the elevation function, evaluated in a fully vectorized way across arrays of points.
  • In a real-world setting, elevation() could be replaced by an interpolated function over actual DEM (Digital Elevation Model) data, and the same gradient ascent / descent machinery would apply directly to hiking route planning, drone path planning, or terrain-aware robotics navigation.

Solving the Two-Facility Location Problem

Minimizing Total Distance Cost

The Business Problem

Imagine a logistics company that needs to build exactly two distribution warehouses to serve a scattered network of retail stores or customers. Each customer has a different demand volume (some order a lot, some order a little), and the company wants to minimize total transportation cost — defined as the sum of each customer’s shipping distance multiplied by their demand weight, always shipping from whichever of the two warehouses is closer.

This is a classic problem in operations research known as the multi-source Weber problem, and it sits at the intersection of continuous optimization (where exactly should the warehouses sit?) and combinatorial optimization (which customers get served by which warehouse?).

Mathematical Formulation

Given $n$ customer locations $\mathbf{p}_i \in \mathbb{R}^2$ with demand weights $w_i$, and two facility locations $\mathbf{f}_1, \mathbf{f}_2 \in \mathbb{R}^2$, we want to solve:

where $c_i \in {1, 2}$ denotes which facility serves customer $i$. In practice, once the facility locations are fixed, the optimal assignment is trivial — each customer simply goes to the nearer facility:

$$
c_i = \arg\min_{k \in {1,2}} \left| \mathbf{p}_i - \mathbf{f}_k \right|_2
$$

The hard part is finding the facility positions themselves. For a single facility, the optimal point minimizing the weighted sum of Euclidean distances is called the weighted geometric median, and it’s found using Weiszfeld’s algorithm, an iterative fixed-point method:

For two facilities, we combine this with an alternating scheme very similar to k-means clustering:

  1. Assign every customer to its nearest facility.
  2. Re-optimize each facility’s position using Weiszfeld’s algorithm on its assigned customers.
  3. Repeat until the facility positions stop moving.

Because this alternating scheme can get stuck in a local optimum (just like k-means), we run it from many random starting positions and keep the best result found.

Full Python Implementation

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
# ==============================================================
# Two-Facility Location Problem (Minimizing Weighted Distance Cost)
# ==============================================================

import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D # noqa: F401
import matplotlib as mpl

# ---------------- Dark theme ----------------
plt.style.use('dark_background')
mpl.rcParams['figure.facecolor'] = '#0d1117'
mpl.rcParams['axes.facecolor'] = '#0d1117'
mpl.rcParams['savefig.facecolor'] = '#0d1117'

np.random.seed(42)

# ---------------- Problem data ----------------
# 12 customer locations (x, y) in km, and demand weights (shipment volume, tons/day)
customers = np.array([
[ 2.0, 8.5], [ 3.5, 9.8], [ 1.0, 6.0], [ 4.5, 7.0],
[ 8.0, 8.0], [ 9.5, 9.0], [ 7.0, 6.5], [10.0, 6.0],
[ 3.0, 2.0], [ 5.0, 1.5], [ 8.5, 2.5], [10.5, 3.5]
])
weights = np.array([4, 6, 3, 5, 7, 5, 4, 6, 8, 3, 5, 4], dtype=float)

# ---------------- Core functions ----------------
def total_cost(facilities, customers, weights):
"""Total weighted distance cost given facility positions, shape (2,2)."""
d = np.linalg.norm(customers[:, None, :] - facilities[None, :, :], axis=2) # (n,2)
nearest = d.min(axis=1)
return np.sum(weights * nearest)

def assign_customers(facilities, customers):
"""Assign each customer to its nearest facility (0 or 1)."""
d = np.linalg.norm(customers[:, None, :] - facilities[None, :, :], axis=2)
return np.argmin(d, axis=1)

def weiszfeld_update(points, weights, f_init, max_iter=200, tol=1e-8, eps=1e-6):
"""Weighted geometric median via Weiszfeld's algorithm."""
f = f_init.copy()
for _ in range(max_iter):
diff = points - f
dist = np.maximum(np.linalg.norm(diff, axis=1), eps)
w = weights / dist
f_new = (w[:, None] * points).sum(axis=0) / w.sum()
if np.linalg.norm(f_new - f) < tol:
f = f_new
break
f = f_new
return f

def two_facility_location(customers, weights, f_init, max_outer=100, tol=1e-7):
"""Alternating optimization: assignment step + Weiszfeld update step."""
facilities = f_init.copy()
history = [facilities.copy()]
cost_history = [total_cost(facilities, customers, weights)]
for _ in range(max_outer):
assign = assign_customers(facilities, customers)
new_facilities = facilities.copy()
for k in range(2):
mask = assign == k
if mask.sum() == 0:
continue
new_facilities[k] = weiszfeld_update(customers[mask], weights[mask], facilities[k])
history.append(new_facilities.copy())
cost_history.append(total_cost(new_facilities, customers, weights))
if np.linalg.norm(new_facilities - facilities) < tol:
facilities = new_facilities
break
facilities = new_facilities
return facilities, np.array(history), np.array(cost_history)

# ---------------- Multi-start optimization ----------------
n_starts = 30
best_cost = np.inf
best_result = None
all_costs = []

x_min, x_max = customers[:, 0].min(), customers[:, 0].max()
y_min, y_max = customers[:, 1].min(), customers[:, 1].max()

for _ in range(n_starts):
f_init = np.column_stack([
np.random.uniform(x_min, x_max, 2),
np.random.uniform(y_min, y_max, 2)
])
fac, hist, chist = two_facility_location(customers, weights, f_init)
all_costs.append(chist)
if chist[-1] < best_cost:
best_cost = chist[-1]
best_result = (fac, hist, chist)

best_facilities, best_history, best_cost_history = best_result
best_assignment = assign_customers(best_facilities, customers)

print(f"Best total cost found: {best_cost:.4f}")
print(f"Facility 1 location: ({best_facilities[0,0]:.3f}, {best_facilities[0,1]:.3f})")
print(f"Facility 2 location: ({best_facilities[1,0]:.3f}, {best_facilities[1,1]:.3f})")

# ---------------- Visualization 1: 2D map with assignment & optimization path ----------------
fig1, ax1 = plt.subplots(figsize=(9, 7))
colors = ['#00e5ff', '#ff6ec7']

for k in range(2):
mask = best_assignment == k
ax1.scatter(customers[mask, 0], customers[mask, 1], s=weights[mask] * 40,
color=colors[k], alpha=0.85, edgecolor='white', linewidth=0.5,
label=f'Customers -> Facility {k+1}')

for k in range(2):
path = best_history[:, k, :]
ax1.plot(path[:, 0], path[:, 1], '--', color=colors[k], alpha=0.5, linewidth=1.2)
ax1.scatter(*best_facilities[k], marker='*', s=600, color=colors[k],
edgecolor='white', linewidth=1.2, zorder=5,
label=f'Facility {k+1} (final)')
ax1.scatter(*best_history[0, k], marker='x', s=120, color='gray', zorder=4)

ax1.set_title('Two-Facility Location: Customer Assignment & Optimization Path', fontsize=13, color='white')
ax1.set_xlabel('X coordinate (km)')
ax1.set_ylabel('Y coordinate (km)')
ax1.legend(loc='upper left', fontsize=8, framealpha=0.3)
ax1.grid(alpha=0.2)
plt.tight_layout()
plt.show()

# ---------------- Visualization 2: 3D cost landscape (vectorized, no nested loops) ----------------
grid_n = 80
gx = np.linspace(x_min - 1, x_max + 1, grid_n)
gy = np.linspace(y_min - 1, y_max + 1, grid_n)
GX, GY = np.meshgrid(gx, gy)

fixed_facility = best_facilities[1]
grid_points = np.stack([GX.ravel(), GY.ravel()], axis=1) # (grid_n^2, 2)

d1 = np.linalg.norm(customers[None, :, :] - grid_points[:, None, :], axis=2) # (grid_n^2, 12)
d2 = np.linalg.norm(customers - fixed_facility, axis=1) # (12,)
d_min = np.minimum(d1, d2[None, :])
Z = (d_min * weights[None, :]).sum(axis=1).reshape(grid_n, grid_n)

fig2 = plt.figure(figsize=(10, 8))
ax2 = fig2.add_subplot(111, projection='3d')
ax2.plot_surface(GX, GY, Z, cmap='plasma', alpha=0.9, linewidth=0, antialiased=True)
ax2.scatter(best_facilities[0, 0], best_facilities[0, 1], best_cost,
color='cyan', s=120, marker='*', label='Optimal Facility 1')
ax2.set_xlabel('Facility 1 X')
ax2.set_ylabel('Facility 1 Y')
ax2.set_zlabel('Total Weighted Cost')
ax2.set_title('Cost Landscape for Facility 1 Position\n(Facility 2 fixed at optimum)', color='white')
ax2.view_init(elev=35, azim=-60)
plt.tight_layout()
plt.show()

# ---------------- Visualization 3: Convergence curves across all random starts ----------------
fig3, ax3 = plt.subplots(figsize=(9, 6))
for chist in all_costs:
ax3.plot(chist, color='gray', alpha=0.3, linewidth=1)
ax3.plot(best_cost_history, color='#00ff9d', linewidth=2.5, label='Best run')
ax3.set_xlabel('Iteration')
ax3.set_ylabel('Total Weighted Cost')
ax3.set_title('Convergence of Total Cost Across 30 Random Starts', color='white')
ax3.legend()
ax3.grid(alpha=0.2)
plt.tight_layout()
plt.show()

Console Output

Best total cost found: 168.4827
Facility 1 location: (8.978, 6.448)
Facility 2 location: (3.346, 6.470)

Code Walkthrough

Problem data. Twelve customer locations are defined as (x, y) coordinates in kilometers, each paired with a demand weight representing daily shipment volume. In a real deployment these would come from a sales database or GIS system.

total_cost: computes the objective function directly from its mathematical definition. It calculates the distance from every customer to every facility in one vectorized NumPy operation (no Python-level loop), takes the minimum distance per customer (i.e., “ships from whichever warehouse is closer”), and returns the weighted sum.

assign_customers: the combinatorial half of the problem — given fixed facility positions, it returns which facility (0 or 1) is closest to each customer.

weiszfeld_update: the continuous half of the problem. This implements the Weiszfeld fixed-point iteration shown in the formula above. A small eps floor is added to the distance to avoid division by zero if a facility ever lands exactly on a customer’s coordinates.

two_facility_location: the outer loop that alternates between assignment and position updates, very similar in spirit to Lloyd’s algorithm for k-means, except each “centroid” is a weighted geometric median rather than a simple average — appropriate because we’re minimizing straight-line distance, not squared distance.

Multi-start loop: since alternating optimization can converge to different local optima depending on where it starts, the script runs the whole procedure 30 times from random initial facility positions and keeps whichever run achieved the lowest total cost. This is a standard and inexpensive way to guard against poor local optima in this class of problem.

Speed Notes

The 3D cost-landscape plot in principle requires evaluating the objective function at every point on an 80×80 grid — 6,400 evaluations. A naive implementation would use two nested Python for loops, which is slow. The code above avoids this entirely by reshaping the grid into a single array of (x, y) points and computing all customer-to-grid-point distances in one broadcasted NumPy operation, then taking element-wise minimums against the fixed second facility. This turns a 6,400-iteration Python loop into a handful of vectorized array operations, so the surface renders almost instantly even on a standard Colab CPU runtime.

Understanding the Visualizations

Figure 1 — Customer Assignment & Optimization Path. This 2D map shows all twelve customers, sized in proportion to their demand weight, colored according to which of the two final warehouses serves them. The two stars mark the optimal facility positions, the gray X marks show where that particular optimization run started, and the dashed lines trace how each facility “walked” from its random starting point to its final resting place across iterations. Notice how the facilities settle roughly in the geometric weighted center of their respective clusters, rather than at the simple average — larger demand customers pull the facility more strongly toward them.

Figure 2 — 3D Cost Landscape. This surface shows how the total cost changes as Facility 1 is moved anywhere on the map, while Facility 2 stays fixed at its optimal location. The valley (lowest point, marked with a cyan star) corresponds to the optimal position found by the algorithm. This plot is useful for building intuition: the cost surface is not smooth like a simple bowl — it has a somewhat faceted, ridge-like shape because of the “assign to nearest facility” logic switching abruptly between the two warehouses as Facility 1 moves across customer territory.

Figure 3 — Convergence Across Random Starts. Each gray line is one of the 30 independent optimization runs, showing how quickly its total cost drops as the alternating algorithm iterates. The green line highlights the best-performing run — the one whose final result is reported and drawn in Figures 1 and 2. Most runs converge within just a handful of iterations, and the spread between lines illustrates why multiple random starts matter: a few runs stall at a noticeably higher cost, meaning they got trapped in a local optimum that a single-start approach could easily have missed.

Takeaways

The two-facility location problem is a compact but genuinely useful example of combining discrete decisions (which warehouse serves which customer) with continuous optimization (where exactly to place each warehouse). The alternating Weiszfeld approach used here generalizes cleanly to three, four, or more facilities simply by expanding the facility array — making it a practical starting point for real supply-chain network design problems.