Historical and philosophical contexts of the calculus of variations

What is a variational principle?

A variational principle is a mathematical or physical law expressed in terms of maximising or minimising a certain quantity. The simplest example was known to ancient Egyptian builders and surveyers: a taut rope stretched between two points takes the shape that minimises its length. This is in fact a straight line on the plane, although Ptolemy (2nd century AD) observed that one must correct for 'deviations from a straight course'. (Which is not to say that he had the concepts of geodesics or Riemannian metrics.) Note, however, that these two statements have different statuses: the variational statement 'a taut rope stretched between two points takes the shape that minimises its length' by itself makes no assertions about straight or curved lines.

These are other examples of variational statements:

• The geodesic problem: The shortest length between two points on a sphere can be found by stretching a taut rope around the sphere through those two points.
• The catenary problem: A hanging chain takes a shape that minimises its potential energy.
• The isoperimetric problem (Dido's problem): For a given perimeter, the area enclosed by that perimeter is maximised by taking as round and convex a shape as possible --- namely, a circle.
• Bubble shape: A pressurised container, such as a soap film or balloon, takes a shape that minimises its surface energy. For bubbles, that's a sphere.
• Temperature distribution: If the surface of a body is kept at a time-independent temperature, then the temperature distribution within the body takes minimal gradient. In particular, if the surface is at constant temperature, then the temperature is constant throughout the body In modern notation, if $T$ solves $\nabla^2T = 0$ then $T$ minimises $\iiint |\nabla T|^2 \mathrm{d}V$, which is solved by constant $T$ if the boundary conditions are homogeneous.

What is the calculus of variations?

To convert these variational statements into mathematical statements (usually differential equations), the calculus of variations was gradually developed throughout the 17th and 18th centuries. Leibniz, Huygens and Johann Bernoulli solved the catenary problem in 1691, while Newton solved the brachistochrone problem and the minimal resistance problem possibly using variational methods ¹; but it was Euler and his student Lagrange who presented the general calculus of variations as it is usually taught today. In the late 1890s, Weierstrass placed the calculus of variations onto the theoretical foundations of analysis; further work by the likes of Hilbert and Noether in the 20th century saw the topic become part of a wider interest into boundary value problems, motivated by developments in quantum mechanics.

Using the calculus of variations, it can be shown that a variational statement such as 'a hanging chain minimises its potential energy' is mathematically equivalent to reasoning 'from first principles', using force-balance arguments on infinitesimal line elements. (Likewise with the surface shape of a balloon.) But although the two formulations predict the same results, they are dissimilar in their starting points. The force-balance approach looks at individual elements of the chain and constructs a differential equation describing their shape. On the other hand, the variational approach makes a statement about the overall shape of the chain, as a whole: in particular, it makes no postulates about the existence of infinitesimal elements. Such variational descriptions ('the solution minimises a particular quantity') are attractive in their simplicity, even though they do not give the solutions themselves without the heavy machinery of the calculus of variations.

Optics and Fermat's principle

A particular variational principle with a long history comes from optics. In his study of geometric optics, Euclid made the assertion that light travels in straight lines, based on the observation that one needs a direct line of sight to see an object. The path of light therefore obeys the shortest path principle, like a taut piece of string. Hero of Alexandria (c. 10--70 CE) studied the reflection of images off mirrors, making the observation that the angle of reflection is equal to the angle of incidence, and showed that this was also consistent with the shortest path principle.

Centuries later, in the 980s the Islamic mathematician Ibn Sahl described on an empirical basis the law of refraction (Snell's law) governing the angles at which light moving from one material to another is refracted. The indices of refraction of various materials were known to mediaeval Muslim scientists; it was not until the 1600s that European scientists such as Snellius claimed the discovery of this law of refraction.

The fact that refracting light does not travel in a straight line contradicts the shortest path principle. In a 1662 letter, Pierre de Fermat instead proposed a principle of minimal time, suggesting (on no experimental or theoretical basis) that the refractive index of a material is related to the finite speed of light in that material, with light being slower in denser materials. This was in stark contrast to the more prevalent theory of Descartes, which asserted that light travelled at infinite speed in vacuum and even faster in denser material: while this now appears to be self-contradictory, notions of infinity were not yet formalised at the time.

Neither Fermat's nor Descartes' assertions about the speed of light were justified at the time: experimental evidence about the speed of light was not available until 1676. This was the first criticism of Fermat's principle by the Cartesian school. The second and more fundamental criticism came from its inability to answer the question of why it should be true. The principle of minimal time is teleological: why does light 'choose' to take a time-minimising path, and how does it 'know' how to find the correct path in advance? Why should it 'choose' to minimise travel time and not some other quantity such as distance?

Claude Clerselier, a Cartesian critic of Fermat, wrote:

The principle which you take as the basis for your proof, namely that Nature always acts by using the simplest and shortest paths, is merely a moral, and not a physical one. It is not, and cannot be, the cause of any effect in Nature.

That said, it was not disputed that Fermat's principle made predictions that were consistent with Snell's law. Despite the empirical support, it was not accepted as an explanation of the experimental results, but merely as an elegant mathematical expression. A more mechanistic formulation of Fermat's principle and Snell's law comes from the modern formulation of light as a wave, which would not come for another two centuries.

Particle mechanics and the principle of least action

Fermat's principle was the first of two variational principles developed in the 17th and 18th centuries. The second was the principle of least action, or its earlier formulation as Maupertuis' principle. The principle of least action states that a conservative system (such as a particle in a potential) evolves in such a way that a certain quantity $S$, 'the action', is minimised. The action is the integral over time of a quantity $L$ called the Lagrangian. The discomfort of using a mathematically simple, yet apparently teleological, variational principle was felt even more strongly here.

Newton's Principia was published in in 1687. Newton's formulation of particle mechanics, including the law of motion $F = ma$ and the inverse square law for gravitation, gives a mathematical foundation for Kepler's (empirical) laws of planetary motion. Despite some initial controversy of their own, Newton's ideas had become accepted by the time of Maupertuis and Euler.

An important development came in the 1740s with the development of the principle of least action by Pierre Louis Maupertuis and Leonhard Euler. Maupertuis defined action $S$ as an 'amount of motion': for a single particle, action is momentum mv multiplied by the distance s travelled; for constant speed, $s = vt$, so the action is $S = mv^2t$. In the absence of a potential, this matches our modern definition of action, up to a factor of 2. (Maupertuis referred to the quantity $mv^2$ as the vis viva, or 'living force', of the particle.) Studying the velocities of two colliding bodies before and after collision, Maupertuis showed that the law of conservation of momentum (by now well-established) is equivalent to the statement that the final velocities are such that the action of this process is minimised.

Euler is generally credited with inventing the calculus of variations in an early form, applying it to studying particle trajectories. (The modern form was later developed by Lagrange, his student, in 1755.) Euler generalised Maupertuis' definition of action into the modern action integral, and included a new term for potential energy. He showed in 1744 that a particle subject to a central force (such as planetary motion) takes a path (calculated by Newton) that extremises this action, and vice-versa. Lagrange later showed more generally that the principle of least action is mathematically equivalent to Newton's laws.

But why is this a sensible definition of action? In fact, what is action?

Maupertuis' reasoning was that 'Nature is thrifty in all its actions', positing that action is a sort of 'effort'. He was happy to attribute the principle of least action as some sort of God trying to minimise the effort of motions in the universe. But how does one know to choose this definition of action and not some other? As for refraction, why does one minimise travel time and not distance? Maupertuis argues that one cannot know to begin with, but that the correct functional needs to be identified.

Fermat and Euler took a rather weaker view, and refuse to make any metaphysical interpretations about their variational principles. Fermat stated that his principle is 'a mathematical regularity from which the empirically correct law can be derived' (Sklar 2012): this is an aesthetic statement about the theory, but says nothing about its
origins.

Why do we find the principle of least action problematic?

The principle of least action is mathematically equivalent to Newton's laws of motion, and therefore have equivalent predictive power and status when compared against experiments. However, Newton's laws are specified as differential equations with initial values ('start in this state, and forward-march in time, with no memory about your past and no information about your future'). In contrast, the principle of least action is formulated as a boundary value problem ('get from A to B in time T, accumulating as little action as possible'), governed by the Euler–Lagrange equations. Despite their equivalence, most people are inherently more comfortable with the Newtonian formulation than with the Euler-Lagrange formulation, and there are two major reasons for this.

The first is the question: Given that we are at the initial position A, how can we know that we will be at B after time T? This can be resolved by realising that when we solve the Euler–Lagrange equations, we have not been told what the initial velocity is, and have the freedom to choose it such that the final position will be B. Thus, one can convert between an IVP and a BVP: this is the approach taken with the shooting method for solving BVP numerically.

The second reason is cultural: most of us are taught Newtonian physics before Lagrangian physics. This is paedagogically sensible: the Newtonian formulation requires far less mathematical machinery and can be taught even with a minimal amount of calculus. More technically, an IVP is guaranteed to have a unique solution, at least for a finite domain (according to the Picard–Lindelöf theorem); a similar guarantee cannot be made for a BVP. While this is not necessarily at the top of mind of most students, it is consistent with our everyday experience.

Acknowledgements

The above essay has been guided by Lawrence Sklar's book, Philosophy and the Foundations of Dynamics.

1. As with much of Newton’s other work, the publications used geometric arguments, and did not reveal his actual heuristics. ↩