

An Analysis of Linux Scalability to Many Cores [OSDI'10 PDF] - yarapavan
http://pdos.csail.mit.edu/papers/linux:osdi10.pdf

======
patrickgzill
Summary: they found bottlenecks in many applications and found that by
implementing various code changes to the kernel, about 3000 lines worth, they
could greatly improve performance on a 48-core Linux box.

For instance, per-core data structures to speed access to commonly used kernel
items.

They also introduce a concept they call "sloppy counters" which is a per-core,
reduced lock contention method of handling certain systemwide counters.

------
xtacy
As noted in the paper, something of relevance: Receive Packet Steering is a
recent patch from Google for the Linux Kernel addresses the issue of packet
processing by hashing packets to specific cores, while honouring data
locality.

<http://lwn.net/Articles/328339/>

Facebook also apparently made some changes to their Kernel to optimise the IO
path for memcached. Mark Zuckerberg presents some of the modifications in this
(slightly long) techtalk:
<http://www.facebook.com/video/video.php?v=631826881803>

------
jgrahamc
It's worth pointing out that people rarely get the gmake speedups that the
authors are seeing from gmake -j because most makefiles contain problems that
constrain parallelism and cause broken builds. For example, many have missing
dependency information, or reuse the same file over and again for intermediate
results.

Fixing that was part of the motivation behind Electric Cloud.

