Choosing the Optimization Algorithm

f you do not specify a method, scipy will automatically choose one for you based on whether your problem has bounds or constraints (usually defaulting to BFGS, L-BFGS-B, or SLSQP). However, explicitly choosing your algorithm allows you to match the solver to your specific mathematical terrain.

Sample Code

from scipy.optimize import minimize # 1. Define your objective function def objective(x): return x[0]**2 + x[1]**2 # 2. Choose your algorithm via the 'method' parameter result = minimize(objective, x0=[1.0, 1.0], method='BFGS') print(result.success) print(result.x)

Constrained & Bounded Algorithms

  • 'SLSQP' (Sequential Least Squares Programming): The industry standard for general constrained problems [1]. Fast, precise, and supports both bounds and arbitrary equality/inequality equations [1].
  • 'trust-constr': A modern, high-precision trust-region algorithm designed for large-scale, highly non-linear constrained problems.
  • 'L-BFGS-B': A low-memory variant of BFGS designed strictly for box bounds (min/max limits on individual variables) [1]. It does not support complex equations linking variables together
  • 'TNC' (Truncated Newton Conjugate-Gradient): Another excellent solver tailored for large-scale problems with simple box bounds
  • 'COBYLA' (Constrained Optimization By Linear Approximations): A unique constrained solver that does not require derivatives [1]. Perfect if your constraints are choppy, noisy, or non-differentiable

Unconstrained Gradient-Based Algorithms

  • 'BFGS' (Broyden-Fletcher-Goldfarb-Shanno): The most popular, highly efficient quasi-Newton method [1]. It uses the gradient to approximate the curvature (Hessian) of your function
  • 'CG' (Conjugate Gradient): Extremely memory-efficient [1]. Best choice for unconstrained problems with thousands of variables where calculating a full Hessian matrix would crash your RAM
  • 'Newton-CG': A Newton-CG method that requires you to provide the exact Hessian matrix (or a function computing its product).
  • 'dogleg' / 'trust-ncg' / 'trust-krylov' / 'trust-exact': Advanced trust-region algorithms that excel when moving through highly curved, narrow valleys, but they require explicit Hessian information.

Derivative-Free (Noisy) Algorithms

  • 'Nelder-Mead': The classic "downhill simplex" method [1]. It creates a geometric shape (a simplex) and flips/stretches it across your function landscape to crawl its way downhill
  • 'Powell': A modification of the conjugate direction method that optimizes along one vector direction at a time without ever needing a gradient

Contents