Q:

What's the distance between a ray and a sphere?

http://cdsmith.wordpress.com/2008/06/06/ray-tracing-in-haskell/
I don't quite understand the shapeDistance function when dealing with spheres.  What does it mean to calculate the distance between a ray and a sphere? and what is the formula?
if you don't know Haskell, the link says:
c = the center of a sphere
r = radius of a sphere
ori = origin of a ray
dir = direction of ray
. = dot product
p = ori + (((c - ori) . dir) * dir)
d_cp = magnitude(p - c)
d = magnitude(p - ori) - sqrt(r^2 - d_cp^2)

If (d_cp >= r) OR (((p - ori) . dir) <= 0) Then
return Nothing
Else
return d

A:

The page you link to doesn't mention "the distance between a ray and a sphere". It talks about "the distance to it along a given ray", and that's also what the code you quote calculates; that is, the distance from the origin of the ray along the ray up to the first point where the ray intersects the sphere. Here are the steps of the calculation, in mathematical notation and with comments:
$$\vec p=\vec o + (((\vec c -\vec o)\cdot \vec n)\vec n$$
$\vec c - \vec o$ is the vector from the origin $\vec o$ to the center $\vec c$ of the sphere, and $\vec n$ is a unit vector along the direction of the ray. Forming the dot product of a vector with a unit vector and multiplying by the unit vector yields the projection of the vector onto the direction of the unit vector. So $(((\vec c -\vec o)\cdot \vec n)\vec n$ is the projection of the vector from the origin to the centre onto the ray, and thus $\vec p$ is the point at which the ray is perpendicular to the direction to the centre.
$$d_{\mathrm cp}=|\vec p - \vec c|$$
This is the distance from $\vec p$ to the centre $\vec c$, i.e. the perpendicular distance from the centre of the sphere to the ray.
$$d=|\vec p - \vec o|-\sqrt{r^2-d_{\mathrm cp}^2}$$
This calculation is somewhat wasteful, since the first term is 
$$
\begin{align}
|\vec p - \vec o|
&=|(\vec o + (((\vec c -\vec o)\cdot \vec n)\vec n) - \vec o|
\\
&=|((\vec c -\vec o)\cdot \vec n)\vec n|
\\
&=(\vec c -\vec o)\cdot \vec n\;,
\end{align}
$$
which had already been calculated in the first step. The first term is the distance from the origin of the ray to $\vec p$. If $\vec p$ is outside the sphere, that is, if its distance from the centre of the sphere is greater than $r$, the argument of the square root in the second term is negative; in this case $d$ won't be used since the first condition in the if statement won't be fulfilled, and the value Nothing will be returned to indicate that the ray doesn't intersect the sphere. If, on the other hand, $\vec p$ is inside the sphere, that is, if its distance from the centre of the sphere is less than $r$, then the square root in the second term calculates the distance on the ray from the intersection with the sphere to $\vec p$; this is an applciation of the Pythagoraen theorem; the desired distance is one leg of a right triangle, the distance from $\vec p$ to the centre is the other, and the sphere's radius is the hypotenuse. This distance from the intersection with the sphere to $\vec p$ is then subtracted from the distance from the origin to $\vec p$, to obtain the distance from the origin to the intersection. Finally, the second condition in the if statement, 
$$(\vec p - \vec o)\cdot \vec n\le 0\;,$$
checks whether $\vec p$ lies on the ray (as opposed to the line containing the ray) by checking whether it lies in the positive direction from the origin along the unit vector. If so, the distance to the intersection is returned; if not, the ray (as opposed to the line) doesn't intersect the sphere, and Nothing is returned.
Note that this only works if the origin is outside the sphere; in this case $\vec p$ and the intersection are always on the same ray from the origin. If the origin lies inside the sphere, the intersection could be in the direction away from $\vec p$.

