Q:

Choosing a hash function for best performance

I need to compare many files (some could be large, some are small) over the network.
So I am planning to hash every file on every client and send only the hash value over the network.
The main goal here is performance. This implies minimal network traffic. Security is not the issue.
There should also be "zero" collisions since I don't want ever to mistakenly consider two different files as identical. Saying that, I know that theoretically there are always collisions, I just want the chance to ever practically meet them be absolutely negligible.
So my question is: Which .net hash function is best for this task?  
I was thinking to use a buffered reader and MD5CryptoServiceProvider (since CNG may not be available on all clients). 
Is there a way to get better performance than that? (perhaps using some external library?)

A:

I was in a similar situation where I needed a .NET hashing algorithm. I needed it for server response caching where speed was more important than security. When I got to this thread, I noticed speculation about performance differences in choice of algorithm and in 32-bit versus 64-bit execution. To bring some science into this debate, I have created some code to actually test some of the available algorithms. I decided to test the built-in MD5, SHA1, SHA256, and SHA512 algorithms. I also included a CRC32 implementation from force-net and a CRC64 implementation from DamienGKit. My results with a ~115MB file are as follows:

Running in 32-bit mode.
Warm-up phase:
CRC32: 296 MiB/s [9C54580A] in 390ms.
CRC64: 95 MiB/s  [636BCF1455BC885A] in 1212ms.
MD5: 191 MiB/s   [mm/JVFusWMKcT/P+IR4BjQ==] in 604ms.
SHA1: 165 MiB/s  [WSFkGbnYte5EXb7kgp1kqbi2...] in 699ms.
SHA256: 93 MiB/s [USKMHQmfMil8/KL/ASyE6rm/...] in 1240ms.
SHA512: 47 MiB/s [Cp9cazN7WsydTPn+k4Xu359M...] in 2464ms.
Final run:
CRC32: 279 MiB/s [9C54580A] in 414ms.
CRC64: 96 MiB/s  [636BCF1455BC885A] in 1203ms.
MD5: 197 MiB/s   [mm/JVFusWMKcT/P+IR4BjQ==] in 588ms.
SHA1: 164 MiB/s  [WSFkGbnYte5EXb7kgp1kqbi2...] in 707ms.
SHA256: 96 MiB/s [USKMHQmfMil8/KL/ASyE6rm/...] in 1200ms.
SHA512: 47 MiB/s [Cp9cazN7WsydTPn+k4Xu359M...] in 2441ms.

Running in 64-bit mode.
Warm-up phase:
CRC32: 310 MiB/s  [9C54580A] in 373ms.
CRC64: 117 MiB/s  [636BCF1455BC885A] in 986ms.
MD5: 198 MiB/s    [mm/JVFusWMKcT/P+IR4BjQ==] in 584ms.
SHA1: 184 MiB/s   [WSFkGbnYte5EXb7kgp1kqbi2...] in 627ms.
SHA256: 104 MiB/s [USKMHQmfMil8/KL/ASyE6rm/...] in 1112ms.
SHA512: 149 MiB/s [Cp9cazN7WsydTPn+k4Xu359M...] in 778ms.
Final run:
CRC32: 292 MiB/s  [9C54580A] in 396ms.
CRC64: 119 MiB/s  [636BCF1455BC885A] in 975ms.
MD5: 199 MiB/s    [mm/JVFusWMKcT/P+IR4BjQ==] in 582ms.
SHA1: 192 MiB/s   [WSFkGbnYte5EXb7kgp1kqbi2...] in 601ms.
SHA256: 106 MiB/s [USKMHQmfMil8/KL/ASyE6rm/...] in 1091ms.
SHA512: 157 MiB/s [Cp9cazN7WsydTPn+k4Xu359M...] in 738ms.

These results were obtained from a compiled Release-build ASP.NET project running .NET v4.5.2. Both the 32-bit and 64-bit results are from the same machine. In Visual Studio, I changed the mode via Tools > Options > Projects and Solutions > Web Projects > Use the 64 bit version of IIS Express, along with changing the Platform target of the project.
We can see that although the results fluctuate a bit run-to-run, CRC32 (by force-net) is the fastest, followed by Microsoft's MD5 and SHA1. Curiously, there is no performance benefit in choosing DamienGKit's CRC64 over the built-in MD5 or SHA1. 64-bit execution seems to help a lot with SHA512 but only modestly with the others.
To answer the OP's question, it would seem that the built-in MD5 or SHA1 may provide the best balance of collision-avoidance and performance.
My code is as follows:
Stopwatch timer = new Stopwatch();
Force.Crc32.Crc32Algorithm hasherCRC32 = new Force.Crc32.Crc32Algorithm();
System.Security.Cryptography.MD5Cng hasherMD5 = new System.Security.Cryptography.MD5Cng();
System.Security.Cryptography.SHA1Cng hasherSHA1 = new System.Security.Cryptography.SHA1Cng();
System.Security.Cryptography.SHA256Cng hasherSHA256 = new System.Security.Cryptography.SHA256Cng();
System.Security.Cryptography.SHA512Cng hasherSHA512 = new System.Security.Cryptography.SHA512Cng();
String result = "";
String rate = "";

Status.Text += "Running in " + ((IntPtr.Size == 8) ? "64" : "32") + "-bit mode.<br /><br />";

Status.Text += "Warm-up phase:<br />";

timer.Restart();
result = BitConverter.ToUInt32(hasherCRC32.ComputeHash(ImageUploader.FileBytes), 0).ToString("X8");
timer.Stop();
rate = ((double)ImageUploader.FileBytes.Length / timer.ElapsedMilliseconds / 1.024 / 1024).ToString("0");
Status.Text += "CRC32: " + rate + " MiB/s [" + result + "] in " + timer.ElapsedMilliseconds + "ms" + ".<br />";

timer.Restart();
result = DamienG.Security.Cryptography.Crc64Iso.Compute(ImageUploader.FileBytes).ToString("X16");
timer.Stop();
rate = ((double)ImageUploader.FileBytes.Length / timer.ElapsedMilliseconds / 1.024 / 1024).ToString("0");
Status.Text += "CRC64: " + rate + " MiB/s [" + result + "] in " + timer.ElapsedMilliseconds + "ms" + ".<br />";

timer.Restart();
result = Convert.ToBase64String(hasherMD5.ComputeHash(ImageUploader.FileBytes));
timer.Stop();
rate = ((double)ImageUploader.FileBytes.Length / timer.ElapsedMilliseconds / 1.024 / 1024).ToString("0");
Status.Text += "MD5: " + rate + " MiB/s [" + result + "] in " + timer.ElapsedMilliseconds + "ms" + ".<br />";

timer.Restart();
result = Convert.ToBase64String(hasherSHA1.ComputeHash(ImageUploader.FileBytes));
timer.Stop();
rate = ((double)ImageUploader.FileBytes.Length / timer.ElapsedMilliseconds / 1.024 / 1024).ToString("0");
Status.Text += "SHA1: " + rate + " MiB/s [" + result + "] in " + timer.ElapsedMilliseconds + "ms" + ".<br />";

timer.Restart();
result = Convert.ToBase64String(hasherSHA256.ComputeHash(ImageUploader.FileBytes));
timer.Stop();
rate = ((double)ImageUploader.FileBytes.Length / timer.ElapsedMilliseconds / 1.024 / 1024).ToString("0");
Status.Text += "SHA256: " + rate + " MiB/s [" + result + "] in " + timer.ElapsedMilliseconds + "ms" + ".<br />";

timer.Restart();
result = Convert.ToBase64String(hasherSHA512.ComputeHash(ImageUploader.FileBytes));
timer.Stop();
rate = ((double)ImageUploader.FileBytes.Length / timer.ElapsedMilliseconds / 1.024 / 1024).ToString("0");
Status.Text += "SHA512: " + rate + " MiB/s [" + result + "] in " + timer.ElapsedMilliseconds + "ms" + ".<br />";

Status.Text += "<br />Final run:<br />";

timer.Restart();
result = BitConverter.ToUInt32(hasherCRC32.ComputeHash(ImageUploader.FileBytes), 0).ToString("X8");
timer.Stop();
rate = ((double)ImageUploader.FileBytes.Length / timer.ElapsedMilliseconds / 1.024 / 1024).ToString("0");
Status.Text += "CRC32: " + rate + " MiB/s [" + result + "] in " + timer.ElapsedMilliseconds + "ms" + ".<br />";

timer.Restart();
result = DamienG.Security.Cryptography.Crc64Iso.Compute(ImageUploader.FileBytes).ToString("X16");
timer.Stop();
rate = ((double)ImageUploader.FileBytes.Length / timer.ElapsedMilliseconds / 1.024 / 1024).ToString("0");
Status.Text += "CRC64: " + rate + " MiB/s [" + result + "] in " + timer.ElapsedMilliseconds + "ms" + ".<br />";

timer.Restart();
result = Convert.ToBase64String(hasherMD5.ComputeHash(ImageUploader.FileBytes));
timer.Stop();
rate = ((double)ImageUploader.FileBytes.Length / timer.ElapsedMilliseconds / 1.024 / 1024).ToString("0");
Status.Text += "MD5: " + rate + " MiB/s [" + result + "] in " + timer.ElapsedMilliseconds + "ms" + ".<br />";

timer.Restart();
result = Convert.ToBase64String(hasherSHA1.ComputeHash(ImageUploader.FileBytes));
timer.Stop();
rate = ((double)ImageUploader.FileBytes.Length / timer.ElapsedMilliseconds / 1.024 / 1024).ToString("0");
Status.Text += "SHA1: " + rate + " MiB/s [" + result + "] in " + timer.ElapsedMilliseconds + "ms" + ".<br />";

timer.Restart();
result = Convert.ToBase64String(hasherSHA256.ComputeHash(ImageUploader.FileBytes));
timer.Stop();
rate = ((double)ImageUploader.FileBytes.Length / timer.ElapsedMilliseconds / 1.024 / 1024).ToString("0");
Status.Text += "SHA256: " + rate + " MiB/s [" + result + "] in " + timer.ElapsedMilliseconds + "ms" + ".<br />";

timer.Restart();
result = Convert.ToBase64String(hasherSHA512.ComputeHash(ImageUploader.FileBytes));
timer.Stop();
rate = ((double)ImageUploader.FileBytes.Length / timer.ElapsedMilliseconds / 1.024 / 1024).ToString("0");
Status.Text += "SHA512: " + rate + " MiB/s [" + result + "] in " + timer.ElapsedMilliseconds + "ms" + ".<br />";

A:

It depends on the number of files you have.
The chance of a collision P(collision) = c/2^N (in a perfect hash function), where c is your number of messages (files) and N is the number of bits in your collision algorithm.
As real-world hash functions aren't perfect so you have two options: optimize for speed and optimize for collision avoidance.
In the first case you will want to use CRC32. CRC32 is very common but, depending on the number of files you have, might not be enough: you're guaranteed to have a collision at ~4,3 billion messages (32 effective bits), but in practice you might encounter your first collision at ~10 million messages. CRC32 has very fast implementations (SSE 4.2 even has a hardware instruction for it). CRC64 has a lot lower chance of a collision but is not widely used, hence if you want more collision avoidance than CRC32 you better look at cryptographic hash functions.
If you want to avoid collisions while sacrificing speed you will want cryptographic hash functions, of which MD5 (128 bits), SHA-1 (160 bits) and SHA-2 (usually SHA-256 or SHA-512) are the most widely used and have fast implementations. Very efficient hash collision finding algorithms for MD5 are available, but if you input random messages you'll get as close to the P(collision) = c/2^128 as you're ever going to get while still running in reasonable time.

A:

The Analysis of the Tests Results
I will further analyze performance of built-in PHP hashes, in addition to the analysis made earlier here by Michael (see the above post), because this topic is quite interesting and has unexpected outcomes.
The results are not that obvious, or even surprising. A simple algorithm - CRC32 or CRC16, is slower than the complex one - MD5. It seems that modern CPUs do not like particular old algorithms and execute them very slowly, at least when the algorithms are not implemented the new way, taking advantage of modern CPU architecture. CRC16 CCITT algorithm was relatively fast and efficient in good old days when there were 300 BPS dial-up modems. Now there are modern algorithm specially designed for new hardware which may work much faster on the same hardware than the old algorithms, which are inherently unfit to the new hardware, and even if you try to optimize them, they will anyway be relatively slow. For example, for algorithms where each byte depends on previous byte and you cannot take benefits of Out-of-order execution or 64-bit registers and process many bits in parallel.
You may see from other cryptographic libraries that confirm what we see in PHP -
 that CRC32 IEEE has almost the same throughout speed as MD5. Here is the link with the results of another library: https://www.cryptopp.com/benchmarks.html
OpenSSL shows similar results. At first glance, it may seem irrational, because the algorithm for CRC32 is much simpler and than for MD5, but the reality shows the opposite.
I just want to show how simple the CRC32 function is.
Here is a code that updates CRCR32 counter with the next incoming byte (Delphi):
   // Returns an updated CRC32
  function UpdateCrc32(CurByte: Byte; CurCrc: Cardinal): Cardinal; inline;
  begin
    UpdateCrc32 := Crc32Table[Byte(CurCrc xor CurByte)] xor (CurCrc shr 8);
  end;

Here is this code on Assembly:
@calc_crc32:
    xor    dl,[esi]
    mov    al,dl
    shr    edx,8
    xor    edx,dword ptr [edi+eax*4]
    inc    esi
    loop   @calc_crc32

You may also unroll this code, so you will get just 5 CPU instructions per byte:
    xor    dl,bl
    shr    rbx,8
    mov    al,dl
    shr    edx,8
    xor    edx,dword ptr [r8+rax*4]

You just need to load the rbx register with next 8 bytes of data and then repeat this code 8 times until you need to load next 8 bytes to the rbx 64-bit register.
Here is the routine that calculates CRC32 of the whole string:
    function CalcCRC32(const B; Size: NativeUINT;
    const 
      InitialValue: Cardinal = CRC32_INIT): Cardinal;
    var
      C: Cardinal;
      P: PAnsiChar;
      i: NativeUINT;
    begin
      C := InitialValue;
      if Size > 0 then
      begin
        P := @B;
        for i := 0 to Size - 1 do
          C := UpdateCrc32(Byte(P[i]), C);
      end;
      Result := C;
    end;

And here is how it is compiled into machine code by Delphi – not very optimal, but quite simple – just 11 assembly instructions for each byte, which, surprisingly, on Intel Core i5-6600 work little bit faster than the above assembler code even after loop unrollment. As you see, and all these instructions to implement the CRC32 IEEE are straightforward, with no loops or comparisons, there is just one comparison at the end of each byte. This is just a debugger output of a compiled Delphi code, not an assembly code written by a human.
    CRC32.pas.78: begin
                                    push esi
                                    push edi
    CRC32.pas.80: if Size > 0 then
                                    test edx,edx
                                    jbe $00500601
    CRC32.pas.82: P := @B;
                                    mov edi,eax
    CRC32.pas.83: for i := 0 to Size - 1 do
                                    mov eax,edx
                                    dec eax
                                    test eax,eax
                                    jb $00500601
                                    inc eax
                                    xor esi,esi
    CRC32.pas.84: C := UpdateCrc32(Byte(P[i]), C);
                                    movzx edx,[edi+esi]
                                    xor dl,cl
                                    movzx edx,dl
                                    mov edx,[edx*4+$517dec]
                                    shr ecx,$08
                                    xor edx,ecx
                                    mov ecx,edx
                                    inc esi
    CRC32.pas.83: for i := 0 to Size - 1 do
                                    dec eax
                                    jnz $005005e6
    CRC32.pas.86: Result := C;
                                    mov eax,ecx
    CRC32.pas.87: end;
                                    pop edi
                                    pop esi
                                    ret

Here is another variant of assembler code for CRC32, just 5 processor commands for each bytes, not 11, but it is essentially the same as the above assembler code, just uses different registers and avoids the "loop" command which is again on i5-6600 faster than the two different instructions. You can find the whole code at CRC32 assembler function called from C console app
             586
            .model flat, stdcall 
            .xmm
            .data
            .code
        CRC32 proc sizeOfFile:DWORD, file:DWORD
            push    esi
            push    ecx
            push    edx

            mov esi, file
            xor edx, edx
            or  eax, -1
            mov ecx, sizeOfFile

        CRC32_loop:
            mov dl, byte ptr [esi]
            xor dl, al
            shr eax, 8
            xor eax, dword ptr [crc32_table + 4*edx]
            inc esi
            dec ecx
            jnz CRC32_loop

            not eax

            pop edx
            pop ecx
            pop esi
            ret

Now compare it with MD5, with this highly-optimized assembler code by Peter Sawatzki:
; MD5_386.Asm   -  386 optimized helper routine for calculating
;                  MD Message-Digest values
; written 2/2/94 by
;
; Peter Sawatzki
; Buchenhof 3
; D58091 Hagen, Germany Fed Rep
;
; EMail: Peter@Sawatzki.de
; EMail: 100031.3002@compuserve.com
; WWW:   http://www.sawatzki.de
;
;
; original C Source was found in Dr. Dobbs Journal Sep 91
; MD5 algorithm from RSA Data Security, Inc.

.386
.MODEL FLAT
.CODE

R1 = ESi
R2 = EDi

FF Macro a,b,c,d,x,s,ac
; a:= ROL (a+x+ac + (b And c Or Not b And d), s) + b
  Add a, [EBp+(4*x)]
  Add a, ac
  Mov R1, b
  Not R1
  And R1, d
  Mov R2, c
  And R2, b
  Or  R1, R2
  Add a, R1
  Rol a, s
  Add a, b
EndM

GG Macro a,b,c,d,x,s,ac
; a:= ROL (a+x+ac + (b And d Or c And Not d), s) + b
  Add a, [EBp+(4*x)]
  Add a, ac
  Mov R1, d
  Not R1
  And R1, c
  Mov R2, d
  And R2, b
  Or  R1, R2
  Add a, R1
  Rol a, s
  Add a, b
EndM

HH Macro a,b,c,d,x,s,ac
; a:= ROL (a+x+ac + (b Xor c Xor d), s) + b
  Add a, [EBp+(4*x)]
  Add a, ac
  Mov R1, d
  Xor R1, c
  Xor R1, b
  Add a, R1
  Rol a, s
  Add a, b
EndM

II Macro a,b,c,d,x,s,ac
; a:= ROL (a+x+ac + (c Xor (b Or Not d)), s) + b
  Add a, [EBp+(4*x)]
  Add a, ac
  Mov R1, d
  Not R1
  Or  R1, b
  Xor R1, c
  Add a, R1
  Rol a, s
  Add a, b
EndM

Transform Proc
Public Transform
;Procedure Transform (Var Accu; Const Buf); Register;

; save registers that Delphi requires to be restored
  Push EBx
  Push ESi
  Push EDi
  Push EBp

  Mov EBp, EDx ; Buf -> EBp
  Push EAx     ; Accu -> Stack
  Mov EDx, [EAx+12]
  Mov ECx, [EAx+8]
  Mov EBx, [EAx+4]
  Mov EAx, [EAx]

  FF EAx,EBx,ECx,EDx,  0,  7, 0d76aa478h  ; 1
  FF EDx,EAx,EBx,ECx,  1, 12, 0e8c7b756h  ; 2
  FF ECx,EDx,EAx,EBx,  2, 17, 0242070dbh  ; 3
  FF EBx,ECx,EDx,EAx,  3, 22, 0c1bdceeeh  ; 4
  FF EAx,EBx,ECx,EDx,  4,  7, 0f57c0fafh  ; 5
  FF EDx,EAx,EBx,ECx,  5, 12, 04787c62ah  ; 6
  FF ECx,EDx,EAx,EBx,  6, 17, 0a8304613h  ; 7
  FF EBx,ECx,EDx,EAx,  7, 22, 0fd469501h  ; 8
  FF EAx,EBx,ECx,EDx,  8,  7, 0698098d8h  ; 9
  FF EDx,EAx,EBx,ECx,  9, 12, 08b44f7afh  ; 10
  FF ECx,EDx,EAx,EBx, 10, 17, 0ffff5bb1h  ; 11
  FF EBx,ECx,EDx,EAx, 11, 22, 0895cd7beh  ; 12
  FF EAx,EBx,ECx,EDx, 12,  7, 06b901122h  ; 13
  FF EDx,EAx,EBx,ECx, 13, 12, 0fd987193h  ; 14
  FF ECx,EDx,EAx,EBx, 14, 17, 0a679438eh  ; 15
  FF EBx,ECx,EDx,EAx, 15, 22, 049b40821h  ; 16

  GG EAx,EBx,ECx,EDx,  1,  5, 0f61e2562h  ; 17
  GG EDx,EAx,EBx,ECx,  6,  9, 0c040b340h  ; 18
  GG ECx,EDx,EAx,EBx, 11, 14, 0265e5a51h  ; 19
  GG EBx,ECx,EDx,EAx,  0, 20, 0e9b6c7aah  ; 20
  GG EAx,EBx,ECx,EDx,  5,  5, 0d62f105dh  ; 21
  GG EDx,EAx,EBx,ECx, 10,  9, 002441453h  ; 22
  GG ECx,EDx,EAx,EBx, 15, 14, 0d8a1e681h  ; 23
  GG EBx,ECx,EDx,EAx,  4, 20, 0e7d3fbc8h  ; 24
  GG EAx,EBx,ECx,EDx,  9,  5, 021e1cde6h  ; 25
  GG EDx,EAx,EBx,ECx, 14,  9, 0c33707d6h  ; 26
  GG ECx,EDx,EAx,EBx,  3, 14, 0f4d50d87h  ; 27
  GG EBx,ECx,EDx,EAx,  8, 20, 0455a14edh  ; 28
  GG EAx,EBx,ECx,EDx, 13,  5, 0a9e3e905h  ; 29
  GG EDx,EAx,EBx,ECx,  2,  9, 0fcefa3f8h  ; 30
  GG ECx,EDx,EAx,EBx,  7, 14, 0676f02d9h  ; 31
  GG EBx,ECx,EDx,EAx, 12, 20, 08d2a4c8ah  ; 32

  HH EAx,EBx,ECx,EDx,  5,  4, 0fffa3942h  ; 33
  HH EDx,EAx,EBx,ECx,  8, 11, 08771f681h  ; 34
  HH ECx,EDx,EAx,EBx, 11, 16, 06d9d6122h  ; 35
  HH EBx,ECx,EDx,EAx, 14, 23, 0fde5380ch  ; 36
  HH EAx,EBx,ECx,EDx,  1,  4, 0a4beea44h  ; 37
  HH EDx,EAx,EBx,ECx,  4, 11, 04bdecfa9h  ; 38
  HH ECx,EDx,EAx,EBx,  7, 16, 0f6bb4b60h  ; 39
  HH EBx,ECx,EDx,EAx, 10, 23, 0bebfbc70h  ; 40
  HH EAx,EBx,ECx,EDx, 13,  4, 0289b7ec6h  ; 41
  HH EDx,EAx,EBx,ECx,  0, 11, 0eaa127fah  ; 42
  HH ECx,EDx,EAx,EBx,  3, 16, 0d4ef3085h  ; 43
  HH EBx,ECx,EDx,EAx,  6, 23, 004881d05h  ; 44
  HH EAx,EBx,ECx,EDx,  9,  4, 0d9d4d039h  ; 45
  HH EDx,EAx,EBx,ECx, 12, 11, 0e6db99e5h  ; 46
  HH ECx,EDx,EAx,EBx, 15, 16, 01fa27cf8h  ; 47
  HH EBx,ECx,EDx,EAx,  2, 23, 0c4ac5665h  ; 48

  II EAx,EBx,ECx,EDx,  0,  6, 0f4292244h  ; 49
  II EDx,EAx,EBx,ECx,  7, 10, 0432aff97h  ; 50
  II ECx,EDx,EAx,EBx, 14, 15, 0ab9423a7h  ; 51
  II EBx,ECx,EDx,EAx,  5, 21, 0fc93a039h  ; 52
  II EAx,EBx,ECx,EDx, 12,  6, 0655b59c3h  ; 53
  II EDx,EAx,EBx,ECx,  3, 10, 08f0ccc92h  ; 54
  II ECx,EDx,EAx,EBx, 10, 15, 0ffeff47dh  ; 55
  II EBx,ECx,EDx,EAx,  1, 21, 085845dd1h  ; 56
  II EAx,EBx,ECx,EDx,  8,  6, 06fa87e4fh  ; 57
  II EDx,EAx,EBx,ECx, 15, 10, 0fe2ce6e0h  ; 58
  II ECx,EDx,EAx,EBx,  6, 15, 0a3014314h  ; 59
  II EBx,ECx,EDx,EAx, 13, 21, 04e0811a1h  ; 60
  II EAx,EBx,ECx,EDx,  4,  6, 0f7537e82h  ; 61
  II EDx,EAx,EBx,ECx, 11, 10, 0bd3af235h  ; 62
  II ECx,EDx,EAx,EBx,  2, 15, 02ad7d2bbh  ; 63
  II EBx,ECx,EDx,EAx,  9, 21, 0eb86d391h  ; 64

  Pop ESi            ; get Accu from stack
  Add [ESi],    EAx
  Add [ESi+4],  EBx
  Add [ESi+8],  ECx
  Add [ESi+12], EDx

; restore registers for Delphi
  Pop EBp
  Pop EDi
  Pop ESi
  Pop EBx

  Ret
  Transform EndP

  End

You can find 32-bit and 64-bit versions of this code at https://github.com/maximmasiutin/MD5_Transform-x64
The performance of this code under IA-32 or x86-64 is 4.94 CPU cycles per byte (on Skylake) of data to calculate MD5.
The above code processes at one call 64 bytes of incoming data. It is called from the main routine that does preparation steps:
    procedure CiphersMD5Update(var Context: TMD5Ctx; const ChkBuf; len:         UInt32);
    var
      BufPtr: ^Byte;
      Left: UInt32;
    begin
      If Context.Count[0] + UInt32(len) shl 3 < Context.Count[0] then
        Inc(Context.Count[1]);
      Inc(Context.Count[0], UInt32(len) shl 3);
      Inc(Context.Count[1], UInt32(len) shr 29);

      BufPtr := @ChkBuf;
      if Context.BLen > 0 then
      begin
        Left := 64 - Context.BLen;
        if Left > len then
          Left := len;
        Move(BufPtr^, Context.Buffer[Context.BLen], Left);
        Inc(Context.BLen, Left);
        Inc(BufPtr, Left);
        If Context.BLen < 64 then
          Exit;
        Transform(Context.State, @Context.Buffer);
        Context.BLen := 0;
        Dec(len, Left)
      end;
      while len >= 64 do
      begin
        Transform(Context.State, BufPtr);
        Inc(BufPtr, 64);
        Dec(len, 64)
      end;
      if len > 0 then
      begin
        Context.BLen := len;
        Move(BufPtr^, Context.Buffer[0], Context.BLen)
      end
    end;

And if your processor supports CRC32 opcodes (SSE 4.2), you can calculate checksums 10 times faster with this code:
      function crc32csse42(crc: cardinal; buf: Pointer; len: NativeUInt): cardinal;
      asm // ecx=crc, rdx=buf, r8=len
        .NOFRAME
        mov eax,ecx
        not eax
        test r8,r8;   jz @0
        test rdx,rdx; jz @0
 @7:    test rdx,7;   jz @8 // align to 8 bytes boundary
        crc32 eax,byte ptr [rdx]
        inc rdx
        dec r8;     jz @0
        test rdx,7; jnz @7
 @8:    mov rcx,r8
        shr r8,3
        jz @2
 @1:    crc32 eax,qword ptr [rdx] // calculate CRC of 8 bytes, aligned
        dec r8
        lea rdx,rdx+8
        jnz @1
 @2:    // less than 8 bytes remaining
        and rcx,7; jz @0
        cmp rcx,4; jb @4
        crc32 eax,dword ptr [rdx] // calculate CRC of 4 bytes
        sub rcx,4
        lea rdx,rdx+4
        jz @0
@4:     // less than 4 bytes remaining
        crc32 eax,byte ptr [rdx]
        dec rcx; jz @0
        crc32 eax,byte ptr [rdx+1]
        dec rcx; jz @0
        crc32 eax,byte ptr [rdx+2]
@0:     not eax
      end;

Please note that in my example I'm using a buffer of just 5KB, to fit in the processor's cache and to exclude the effect of slower RAM on the speed of the digest calculation.
There are fast implementations of CRC32 algorighm for modern processors that does not use CRC32 opcodes, but take benefit of Out-of-order execution, including speculative execution via register renaming. An example of such implementation is CRC32 Slicing-By-8. IA-32 or x86-64 assembler code has 1,20 CPU clock cycles (on Skylake) per byte of data. You can find such implementation at https://github.com/maximmasiutin/CRC32-Slicing-x64-Asm-Pas
In PHP, even in version 7, there seem to be no support for hardware acceleration of CRC32, albeit these instructions are supported on Intel and AMD processors since ages. Intel support CRC32 since November 2008 (Nehalem (microarchitecture)) and AMD seems to support it since 2013.
My Own Tests that Confirm Michael's Results
I have tested various PHP hash functions on different configurations: (1) AMD FX-8320 (released in 2012) under Ubuntu with PHP 5, and (2) Intel Core i5-6600 released in 2015 under Windows with PHP 7. I have also run OpenSSL test on this Intel Core i5-6600. Besides that, I run tests of the cryptographic routines that we use in our software “The Bat!” written in Delphi. Although the main software is written in Delphi, the cryptographic routines that we use are written on Assembler for Intel processor (32-bit or 64-bit) or in C.
I have found out that the our Delphi code shows very big speed differences between various hash functions and data sizes. This is in contrast to PHP, where to a certain degree and with rare exceptions all hash functions from simplest CRC32 to once-cryptographically-strong MD5 have almost the same throughoutput speed.
So, here are the measurements that I have done on AMD FX-8320, PHP5, Ubuntu. I made two test cases. First, I ran 5000 iterations to hash a message consisting of just 5 bytes. By this small message size I intended to test the duration of initialization/finalization steps of various algorithms, and how it affects overall performance. For some algorithms, like CRC32, three are virtually no finalization steps - the digest is always ready, after each byte. Cryptographically strong functions like SHA1 or MD5 or other have a finalization step that compresses larger context to a smaller final digest. Second, I run 5000 iterations to hash a message 5000 bytes long. Both of the messages were filled in advance with pseudo-random bytes (they were not re-filled after each iteration; they were only filled once, when the program started). 
Results of my PHP Hash Speed Test
I have modified your PHP code and made it now work for both PHP5 and PHP7, where are different functions to generate random data in different versions of PHP. I have just measured the time that was needed to hash 5000 iterations of 5-bytes messages and then 5000 iterations of 5000-bytes message. Here are the results:
        Legend:
        (1) 5b x 5000, AMD FX-8320, PHP5
        (2) 5000b x 5000, AMD FX-8320, PHP5

PHP hash              (1)            (2)
--------         ------------   ------------
md2              0.021267 sec   2.602651 sec
md4              0.002684 sec   0.035243 sec
md5              0.002570 sec   0.055548 sec
sha1             0.003346 sec   0.106432 sec
sha224           0.004945 sec   0.210954 sec
sha256           0.004735 sec   0.238030 sec
sha384           0.005848 sec   0.144015 sec
sha512           0.006085 sec   0.142884 sec
ripemd128        0.003385 sec   0.120959 sec
ripemd160        0.004164 sec   0.174045 sec
ripemd256        0.003487 sec   0.121477 sec
ripemd320        0.004206 sec   0.177473 sec
whirlpool        0.009713 sec   0.509682 sec
tiger128,3       0.003414 sec   0.059028 sec
tiger160,3       0.004354 sec   0.059335 sec
tiger192,3       0.003379 sec   0.058891 sec
tiger128,4       0.003514 sec   0.073468 sec
tiger160,4       0.003602 sec   0.072329 sec
tiger192,4       0.003507 sec   0.071856 sec
snefru           0.022101 sec   1.190888 sec
snefru256        0.021972 sec   1.217704 sec
gost             0.013961 sec   0.653600 sec
adler32          0.001459 sec   0.038849 sec
crc32            0.001429 sec   0.068742 sec
crc32b           0.001553 sec   0.063308 sec
fnv132           0.001431 sec   0.038256 sec
fnv164           0.001586 sec   0.060622 sec
joaat            0.001569 sec   0.062947 sec
haval128,3       0.006747 sec   0.174759 sec
haval160,3       0.005810 sec   0.166154 sec
haval192,3       0.006129 sec   0.168382 sec
haval224,3       0.005918 sec   0.166792 sec
haval256,3       0.006119 sec   0.173360 sec
haval128,4       0.007364 sec   0.233829 sec
haval160,4       0.007917 sec   0.240273 sec
haval192,4       0.007676 sec   0.245864 sec
haval224,4       0.007580 sec   0.245249 sec
haval256,4       0.007442 sec   0.241091 sec
haval128,5       0.008651 sec   0.281248 sec
haval160,5       0.009304 sec   0.278619 sec
haval192,5       0.008972 sec   0.281235 sec
haval224,5       0.008917 sec   0.274923 sec
haval256,5       0.008853 sec   0.282171 sec

I then run the same PHP script under Intel Core i5-6600, with 64-bit version of PHP7 under Windows 10. Here are the results:
        Legend:
        (1) 5b x 5000, Intel Core i5-6600, PHP7
        (2) 5000b x 5000, Intel Core i5-6600, PHP7

PHP hash           (1)            (2)
---------    ------------    ------------
md2          0.016131 sec    2.308100 sec
md4          0.001218 sec    0.040803 sec
md5          0.001284 sec    0.046208 sec
sha1         0.001499 sec    0.050259 sec
sha224       0.002683 sec    0.120510 sec
sha256       0.002297 sec    0.119602 sec
sha384       0.002792 sec    0.080670 sec
ripemd128    0.001984 sec    0.094280 sec
ripemd160    0.002514 sec    0.128295 sec
ripemd256    0.002015 sec    0.093887 sec
ripemd320    0.002748 sec    0.128955 sec
whirlpool    0.003402 sec    0.271102 sec
tiger128,3   0.001282 sec    0.038638 sec
tiger160,3   0.001305 sec    0.037155 sec
tiger192,3   0.001309 sec    0.037684 sec
tiger128,4   0.001618 sec    0.050690 sec
tiger160,4   0.001571 sec    0.049656 sec
tiger192,4   0.001711 sec    0.050682 sec
snefru       0.010949 sec    0.865108 sec
snefru256    0.011587 sec    0.867685 sec
gost         0.008968 sec    0.449647 sec
adler32      0.000588 sec    0.014345 sec
crc32        0.000609 sec    0.079202 sec
crc32b       0.000636 sec    0.074408 sec
fnv132       0.000570 sec    0.028157 sec
fnv164       0.000566 sec    0.028776 sec
joaat        0.000623 sec    0.042127 sec
haval128,3   0.002972 sec    0.084010 sec
haval160,3   0.002968 sec    0.083213 sec
haval192,3   0.002943 sec    0.082217 sec
haval224,3   0.002798 sec    0.084726 sec
haval256,3   0.002995 sec    0.082568 sec
haval128,4   0.003659 sec    0.112680 sec
haval160,4   0.003858 sec    0.111462 sec
haval192,4   0.003526 sec    0.112510 sec
haval224,4   0.003671 sec    0.111656 sec
haval256,4   0.003636 sec    0.111236 sec
haval128,5   0.004488 sec    0.140130 sec
haval160,5   0.005095 sec    0.137777 sec
haval192,5   0.004117 sec    0.140711 sec
haval224,5   0.004311 sec    0.139564 sec
haval256,5   0.004382 sec    0.138345 sec

As you see, to calculate CRC32 of messages in PHP, in almost all of my tests, it takes just about half of the time it takes to calculate MD5 of the same messages. The only exception was that on the test of 5000 messages of 5000 bytes  on Intel Core i5-6600 with PHP7, CRC32 even took longer than MD5(!). This strange result was always repeatable on my case. I could not find a plausible explanation for it.
Also, on PHP, there was almost no noticeable speed difference between MD5 and SHA1, except on Ubuntu with PHP5, where on the test of 5000 messages of 5000 bytes, MD5 was twice faster.
Results of my OpenSSL's Tests of Hash Performance
Here are the tests of OpenSSL on the Intel i5-660. They show data differently. They don’t show how much time it took do digest certain set of data, but vice versa: they show the volume of data that OpenSSL have managed to hash in 3 seconds. So, higher value is better:
Legend:
 (1) OpenSSL 1.1.0 on Intel Core i5-6600, number of 16-bytes messages processed in 3 seconds
 (2) OpenSSL 1.1.0 on Intel Core i5-6600, number of 8192-bytes messages processed in 3 seconds

Algorighm              (1)            (2)
---------         ---------      ----------
md4               50390.16k      817875.48k
md5              115875.35k      680700.59k
sha1             118158.30k      995986.09k
ripemd160         30308.79k      213224.11k
whirlpool         39605.02k      182072.66k

Again, there is almost no differences between md5 and sha1 which is strange 
and requires further investigation whether the MD5 and SHA-1 algorithm are inherently the same in terms of time consumption.
Results of our Delphi Hash Functions Performance
Here are the results of our Delphi library on Intel Core i5-6600 under Windows 10 64-bit, the code tested was a 32-bit Win32 application. 
      Legend:
        (1) Delphi, 5b x 5000 iterations
        (2) Delphi, 5000b x 5000 iterations

Algorighm                  (1)                     (2)
---------------      --------------         --------------
md2                  0.0381010 secs         5.8495807 secs
md5                  0.0005015 secs         0.0376252 secs
sha1                 0.0050118 secs         0.1830871 secs
crc32               >0.0000001 secs         0.0581535 secs
crc32c (intel hw)   >0.0000001 secs         0.0055349 secs

As you see, MD2 is also much much shower than the other hashes – the same outcome as with the PHP code, but MD5 is much faster than SHA-1, and overall it took less time in Delphi to do the same on the same machine as PHP
For example, PHP7 took 0.001284 sec to digest 5000 5-byte messages with MD5, 0.001499 sec with SHA1. As about 5000 bytes message – it took PHP7 0.046208 sec with MD5 and 0.050259 sec with SHA-1.
As about Delphi, it took 0.0005015 sec to digest 5000 5-byte messages with MD5 and 0.0050118 secs with SHA1. As about 5000 bytes message – it took Delphi 0.0376252 secs sec with MD5 and 0.1830871 secs with SHA-1. As you see, MD5 works much faster in Delphi, but SHA-1 is about the same. Also, Delphi is about 10 times faster on 5-bytes message, but as about 5000-bytes messages, it is about the same or even slower with SHA-1.
But when it comes to CRC32 and CRC32C, Delphi is unbeatable, from 10 to 1000 time faster than PHP.
Conclusion
PHP is inherently very slow on loops and small operations. So it doesn’t matter on PHP which hash function you call if you need to calculate a hash of a very small message. But if you need to digest a large message, the differences in algorithm speed began to show them: for example, MD2 performance is about ten times worse than for MD5. Anyway, there is absolutely no reason today to use MD2. There are no advantages for a PHP user of MD2 over MD5. As about MD5, being initially designed as a cryptographic hash function and then used by PGP in RFC-1991, now can no longer be used in Cryptography, but can be used as a checksum in trusted environments, for example for ETags or other means. This function is very fast when implemented properly (and on PHP it is not slow at least), and MD5 produces very compact digest comparing to other functions. Here is my PHP code that made these benchmarks. I have made it based on the original code sample by Michael (see above).
<?
 define (TRAILING_ZEROS, 6);
 $strlens = array(5, 30, 90, 1000, 5000);
 $hashes = hash_algos();

 function generate_bytes($len)
 {
   if (function_exists('random_bytes')) {$fn='random_bytes';$str = random_bytes($len);} else // for php 5
   if (function_exists('openssl_random_pseudo_bytes')) {$fn='openssl_random_pseudo_bytes';$str = openssl_random_pseudo_bytes($strlen);} else // for php 7
   {
        flush();
        ob_start () ;
        phpinfo () ;
        $str = str_pad(substr(ob_get_contents (), 0, $len), $len) ;
        ob_end_clean () ;
        $fn = 'phpinfo';
   }
   return array(0=>$str, 1=>$fn);
 }

 foreach ($strlens as $strlen)
 {

 $loops = 5000;
 echo "<h1>$loops iterations on $strlen bytes message</h1>".PHP_EOL;
 echo '<p>';
 $r = generate_bytes($strlen);
 $str = $r[0];
 $gotlen = strlen($str);
 while ($gotlen < $strlen)
 {
   // for some uncodumented reason, the  openssl_random_pseudo_bytes returned less bytes than needed
   $left = $strlen-$gotlen;
   echo "The ".$r[1]."() function returned $left byes less, trying again to get these remaining bytes only<br>";
   $r = generate_bytes($left);
   $str.= $r[0];
   $gotlen = strlen($str);
 };

 echo "Got the whole string of ".strlen($str)." bytes!";
 echo '</p>';
 echo PHP_EOL;
 echo "<pre>";

 foreach ($hashes as $hash)
 {
        $tss = microtime(true);
        for($i=0; $i<$loops; $i++)
        {
                $x = hash($hash, $str, true);
        }
        $tse = microtime(true);
        echo "\n".str_pad($hash, 15, ' ')."\t" . str_pad(round($tse-$tss, TRAILING_ZEROS), TRAILING_ZEROS+2, '0') . " sec \t" . bin2hex($x);
 }

 echo PHP_EOL."</pre>".PHP_EOL;
 flush();
 }
?>

Continuation of the post...

