Q:

Why SSE in rails hang on the connection?

I am reading this post which talks about SSE connections without blocking server threads The author is describing how to solve the blocking.
My confusion is If I am closing the stream on server side (with sse.close) and on client side (with source.close()) why you have a problem in the first place? Why the server hangs on the connection?

A:

Problem is not in closing the connection, it's about when you are doing it - this happens way later(dozens of seconds, may be even into hours) than for a regular request, which is usually handled below couple of seconds even for the heaviest ones.
For example, let's say (very simplified and approximated):

you have 10 threads/workers, no SSE
you serve each html page in under 0.1 seconds
a user will wait up to a 1 second for page to load before busting in tears and going away from your site
the user will then read the page for 9 seconds before requesting next one

this way each thread can serve 10 pages a second, all threads - 100 pages a second, since every user requests at most one per 10 seconds - you can handle about 1000 users using your app simultaneously.
Now adding SSE updates to these pages, step by step:

first user connects, gets html in 0.1 sec, then one thread is occupied with SSE request for 9 seconds before page reload, and after that reload it will again be locked by the same users' request
you have only 9 idle threads
second user connects, the same repeats

This way only maximum of 10 users can be handled by the same system, that's 100x less. And you cannot just increase threads to 1000, because they are not free (memory, scheduler overhead etc.).
The catch is that most of the time most such connections are not doing anything, just waiting for events, so they actually do not need a thread reserved for them. So it's logical to free the thread for other requests without closing the connection, that's what hijack does.
PS. This approach can be taken even further - client live updates connection can be kept open by a process other than rails server (non-ruby and more efficient) while still doing all the events logic in rails. For example with anycable backend for ActionCable you can keep thousands of concurrent connections easily

