Q:

Django persistent database connection

I'm using django with apache and mod_wsgi and PostgreSQL (all on same host), and I need to handle a lot of simple dynamic page requests (hundreds per second). I faced with problem that the bottleneck is that a django don't have persistent database connection and reconnects on each requests (that takes near 5ms). 
While doing a benchmark I got that with persistent connection I can handle near 500 r/s while without I get only 50 r/s.
Anyone have any advice? How to modify django to use persistent connection? Or speed up connection from python to DB
Thanks in advance.

A:

Django 1.6 has added persistent connections support (link to doc for latest stable Django ):

Persistent connections avoid the overhead of re-establishing a
connection to the database in each request. They’re controlled by the
CONN_MAX_AGE parameter which defines the maximum lifetime of a
connection. It can be set independently for each database.

A:

Try PgBouncer - a lightweight connection pooler for PostgreSQL.
Features:

Several levels of brutality when rotating connections:

Session pooling
Transaction pooling
Statement pooling

Low memory requirements (2k per connection by default).

A:

In Django trunk, edit django/db/__init__.py and comment out the line:
signals.request_finished.connect(close_connection)

This signal handler causes it to disconnect from the database after every request.  I don't know what all of the side-effects of doing this will be, but it doesn't make any sense to start a new connection after every request; it destroys performance, as you've noticed.
I'm using this now, but I havn't done a full set of tests to see if anything breaks.
I don't know why everyone thinks this needs a new backend or a special connection pooler or other complex solutions.  This seems very simple, though I don't doubt there are some obscure gotchas that made them do this in the first place--which should be dealt with more sensibly; 5ms overhead for every request is quite a lot for a high-performance service, as you've noticed.  (It takes me 150ms--I havn't figured out why yet.)
Edit: another necessary change is in django/middleware/transaction.py; remove the two transaction.is_dirty() tests and always call commit() or rollback().  Otherwise, it won't commit a transaction if it only read from the database, which will leave locks open that should be closed.

