In my last Differential Geometry class I have introduced my students to the Calculus of Variations. This is an important topic when one wants to minimize anything, especially when one wants to find geodesics. There is a very neat fact that I came across again and I wanted just to write it down.

In general for the reader interested in Differential Geometry I would like to refer you to the nicely written Differential Geometry notes by my friend Mario Micheli.
He is currently at UCLA. The notes are available here and here. The lectures with easy introduction to Calculus of Variations are 26, 27, 28.

I will be very brief in describing the nifty little thing. So the question is “find function q(t)\in C^1[a,b] with fixed points, i.e. q(a)=y_0 and q(b)=y_1, such that this q(t) has the graph of minimal length”. Everything boils down to minimizing the functional
J(q) = \int_a^b \sqrt{1+\dot{q}(t)}dt
with respect to function q(t).

So the interesting bit comes when one computes the gradient of this function. One gets (again, for details refer to the links above, or do the computation yourself. “It is an easy and helpful exercise, muahaha”):
\nabla J(q) = -\frac{\ddot{q}}{(1+\dot{q}^2)^{3/2}}.

If one imagines the simple gradient descent algorithm on any chose path between points (a,y_0) and (b,y_1), then one can see that the steepest descent will have to follow in the direction -\nabla J(q). That means that where function q(t) is concave, then \ddot{q}<0. Therefore the steepest descent will be flattening the concave part. Similarly, it would be flattening the convex part of the function, where the second derivative is positive. Also, the more concave (convex) the function is, the bigger the gradient, thus the function gets pushed to the straight line very quickly.
(As you have guessed straight line is the solution to this problem).

It is very good exercise to implement steepest descent for this problem. And then follow it with conjugate gradient and the some quasi-newton method. That’s how I learned these optimization techniques. Cheers.