Q:

Why normal acceleration doesn't bring a change in speed?

Suppose There's a particle of mass m moving with speed $v_{0}$, at certain moment a force starts acting on it (centripetal force) and it start uniform circular motion
Now every other reference states that centripetal force only brings about change only in direction
But plugging-in things in the equations paint a different picture
suppose  normal/centripetal acceleration = $A$
then $|v| = \sqrt{v_{0}^2+\frac{(A t)^2}{1}}$
we can certainly see that factor of $t$ will bring change in magnitude of velocity 
Secondly, why the velocity is always tangential to the circle, but how can we mathematically show such?
Intuitively for me if there's constant acceleration toward a certain then at some point of time, the body should 'slack' and fall toward the direction of acceleration 
Here' a picture for a reference:

A:

Let me try to answer your questions separately below.

then $|v| = \sqrt{v_{0}^2+\frac{(At)^2}{1}}$

The acceleration which you plug into this formula does not act in the same direction as the velocity in the formula does.
This formula looks like one of the four fundamental kinematic equations. They only work along one path (along one dimension), because they are scalar equations, not vector equations. Don't mix dimensions. (In other words, the acceleration (which is a gain in speed) must be a gain in the same speed as $v$ represents - and scalar-wise different directions/dimensions have different speeds).
From a comment:

the initial velocity v and the new velocity V due to centripetal acceleration are both mutually perepndicular vectors so adding them shall produce a new velocity vector with the above given magnitude ain't it?

The perpendicular acceleration creates a perpendicular velocity. We now have the original velocity and this new sideways velocity. Together they create a new net velocity, which is tilted inwards a tiny bit. Turned a tiny bit. 
If the sideways component is negligible (which it is since the acceleration only pulls that way for a negligibly short time), then the magnitude change is negligible. So no change in magnitude.
The turn is tiny, but in the next instant, the same thing happens again because the acceleration turns with the velocity. Do that many times and you see a combined large turn but no change in magnitude.

Secondly ,why the velocity is always tangential to the circle, but how can we mathematically show such? 

Mathematically, I would call the velocity vector the direction vector (not sure this is a correct English term, though. This is a direct translation from my native language). It is the direction vector because it always points in the direction of the change at the very point it is standing in.
The velocity is the change in position, metres per second. So whichever way it points is the way the position is added metres, so that's the way the particle moves. The velocity vector will thus always be tangential to a circular path, because this is the path travelled by the object - the position is changed to the next point on this path in every instant.

Intuitively for me if there's constant acceleration toward a certain then at some point of time, the body should 'slack' and fall toward the direction of acceleration

Sure, and it definitely also would. But the acceleration turns along with the direction turning. In other words, your intuition is correct but for just a moment. Then you have a new situation and a new direction for your object to fall.
Think of a satellite in orbit around the planet. It is held in orbit by gravity which is such an acceleration that always points inwards.

If the satellite was just placed in some distance from Earth by a space station, it would fall straight down and crash.
If it was being thrown sideways, then it would still fall down and crash but it would fall with a curve because it now had sideways speed as well. 
Now imagine throwing it so hard that it misses Earth! It falls and falls and falls, but the sideways speed is large enough for it to fall beside Earth rather than crashing.

This is essentially what happens in circular motion: The object (satellite) falls towards the centre (Earth) but misses all the time.

A:

Why normal acceleration doesn't bring a change in speed?

It is, on my view, more fruitful to ask "what is the acceleration vector of an object with uniform (constant speed) circular motion?"
Such an object, moving in the x-y plane has coordinates:
$$x(t) = R\cos(\omega t + \phi)$$
$$y(t) = R\sin(\omega t + \phi)$$
where $R$ (the radius of the circular path) and $\omega$ (the angular speed of the object) are constants.  The velocity vector of the object is then
$$\mathbf{v}(t) = -\omega R\sin(\omega t + \phi)\mathbf{\hat{x}} + \omega R\cos(\omega t + \phi)\mathbf{\hat{y}}$$
Clearly, the speed (magnitude of the velocity vector) is constant and equal to $|\mathbf{v}| = \omega R$.
Now, calculate the acceleration vector (do this yourself) and find that (1) it is non-zero and constant in magnitude and, (2) it is perpendicular (normal) to the velocity always.

