Q:

Why do i need to publish my chrome extension to Chrome Web Store?

I built a web application which has a link to download a chrome extension .CRX file. 
Why do I need to upload same .CRX file in Chrome Web Store, paying $5 as developer fee?
Sometimes chrome is disabling a chrome extension which I added manually when I reopen Chrome browser window. Can someone tell me why this is happening?  

A:

Why do I need to upload same .crx file in chrome web store by paying $5 as developer fee?

Optimistic answer: malware control
I invite you to read the blog post where Google announced the change at Chromium blog:

Protecting Windows users from malicious extensions

and follow-ups: 

Protecting Chrome users from malicious extensions
Continuing to protect Chrome users from malicious extensions.

From personal experience, malicious extensions were a real problem with Chrome on Windows until that change. Programs were allowed to add any third-party code to Chrome, without much in terms of user consent, just through registry manipulation.
With extensions being served from a central source, Google is able to blacklist and remotely kill misbehaving extensions, which is not possible otherwise.
If a developer account is caught spreading malicious extensions, the account will be suspended. And the one-time fee is to deter people from dodging ToS violation account suspensions by simply creating new accounts. It also provides a degree of identity verification.
Pessimistic answer: terms & conditions enforcement
Not every extension is compatible with Developer Agreement and Program Policies, even if they don't do anything malicious. Example: extensions that offered YouTube video downloads.
Therefore, whether this is the main intent or just a nice bonus, enforcing CWS as the hosting platform enforces said Agreement.

For your particular case (a companion extension for a web app), you should look at Inline Installation guide. It provides better UX than simply a link to the extension.

Sometimes Chrome is disabling a chrome extension which I added manually when I reopen chrome browser window. Can someone tell me why this is happening?

If the extension is installed unpacked through Developer Mode, the mode must be kept enabled and a nag popup at each Chrome launch must be correctly answered. Otherwise, extensions will be disabled.
If an extension is installed manually through CRX, that's an unauthorized source. Developers are able to locally install the CRX they packed themselves, but that's only for testing purposes. Extended use of it is not supported as per Chrome policies, so it gets disabled eventually.

