Nelder-Mead Pattern Search
The Nelder-Mead algorithm is a pattern search method for use on unconstrained nonlinear models. It is also known as the downhill simplex method or the flexible polyhedron method. A simplex is simply a polytope of n+1 vertices in n-dimensions. So for example, it is a line segment on a line, a triangle on a plane, and a tetrahedron in three-dimensional space.
The following two animations are demonstrations of the Nelder-Mead method in action. The first animation is a simplex search over the well-known Rosenbrock banana-shaped objective function. Observe how the simplex moves, expands and shrinks until it converges upon the optimum point.