Q:

What's the cause of Diesels' typical clatter noise?

I guess it's related to their basically running off knocking, but if I were to explain it to someone like this, it wouldn't convince me neither.
[Edit] From http://www.ford-trucks.com/forums/369700-what-exactly-causes-the-diesel-noise.html

The Loud knock is caused by the introduction of high pressure cold fuel into very high cylinder pressures.

Now, who's right?

A:

Two types of sounds
There are two sound types that emerge from a diesel engine.

Combustion Noise
Mechanical Noise

Combustion noise is created by the compression ignition process which compresses the air fuel mixture of the gas which creates higher temperatures upon increased compression until combustion occurs.
Mechanical noise is created primarily by piston slap where the piston rocks back and forth upon the wrist pin in the bore of the cylinder making a tapping noise that transfers through the solid case and emanates from the surface.
Citations
Diesel Engine System Design
Combustion Noise from a Running Diesel Engine Based on Transient Combustion Noise Generation Model

A:

The engine operates on high compression to ignite the fuel at very low RPMs which gives it it's distinctive sound. The lower RPM makes the ignition of each cylinder easier to hear and the compression ignition system gives it it's rattle like sound. This is as opposed to gasoline engines that operate by a spark/compression ignition system. The key to understanding this lies in understanding the diesel engine. See: the Wiki page for this.
In short, the distinctiveness of your typical diesel sound comes from a combination of it's properties of low RPMs and the high compression ignition system. The actual sounds are generated at combustion and the opening and closing of valves. You really cannot hear the injection of the fluids into the chambers. Also, as another poster first pointed out (DucatiKiller), there is sometimes a piston slap that make a audible sound but not always in newer engines. 
Piston Slap
The high pitch hiss noise you hear from some diesels as they accelerate comes from the turbo charger turbine spinning up and sucking in air at a high rate. Don't mistake the hiss I refer to with the air-brake system, as this is altogether something else. 
See:How a Turbo Charger works.

