Q:

What is the derivative of $\mathrm{trace}((S^T S)^{-2})$ w.r.t. $S$

I would like to compute the derivative of $\mathrm{trace}((S^T S)^{-2})$ w.r.t. $S$.
I know that $\frac{\partial \mathrm{trace}((S^T S)^{-1})}{\partial S} = -2S(S^T S)^{-2}$ but I have a higher order in my expression.
Any help would be really appreciated.

A:

Let's use definition
$$
\begin{align*}
&Tr[((S^T+\Delta^T)(S+\Delta))^{-2}] - Tr[(S^TS)^{-2}] \\
&=Tr[(S^TS+S^T\Delta+\Delta^TS+\Delta^T\Delta)^{-2} - (S^TS)^{-2}]\\
&=Tr[((S^TS)^{-1} - (S^TS)^{-1}(S^T\Delta+\Delta^TS)(S^TS)^{-1} + \mathcal{O}\Big(||\Delta||^2\Big))^2- (S^TS)^{-2}]\\
&=Tr[(S^TS)^{-2}-(S^TS)^{-2}(S^T\Delta+\Delta^TS)(S^TS)^{-1}-(S^TS)^{-1}(S^T\Delta+\Delta^TS)(S^TS)^{-2} - (S^TS)^{-2} + \mathcal{O}\Big(||\Delta||^2\Big)]\\
&=-4Tr[(S^TS)^{-3}S^T\Delta] + \mathcal{O}\Big(||\Delta||^2\Big)\\
&=\langle-4S(S^TS)^{-3},\Delta\rangle + \mathcal{O}\Big(||\Delta||^2\Big)
\end{align*}
$$

