Q:

Minimisation of the $\epsilon$-insensitive loss function

Let $X$ be a random variable admitting first and second moment. We define the function $\mathcal{R}_Q(x)$ as follows:
$$ \mathcal{R}_Q(x) = \mathbb{E}\left[\mathcal{L}_Q(X,x)\right] = \mathbb{E}\left[(X-x)^2\right]$$
where $\mathcal{L}_Q(\cdot,\cdot)$ is the quadratic loss function. We know that in order to minimise $\mathcal{R}_Q(\cdot)$, we need:
$$ x = \mathbb{E}[X] $$
Indeed:
$$ \begin{align}
\mathbb{E}\left[(X-x)^2\right] &= \mathbb{E}\left[(X-\mathbb{E}[X]+\mathbb{E}[X]-x)^2\right]
\\[6pt]
&=\mathbb{E}\left[(X-\mathbb{E}[X])^2\right]+2\mathbb{E}\left[X-\mathbb{E}[X]\right]\left(\mathbb{E}[X]-x\right)+\left(\mathbb{E}[X]-x\right)^2
\\[6pt]
&=\mathbb{E}\left[(X-\mathbb{E}[X])^2\right]+\left(\mathbb{E}[X]-x\right)^2
\\[6pt]
&=\mathbb{V}\left[X\right]+\left(\mathbb{E}[X]-x\right)^2
\end{align} $$
which is minimised when $x = \mathbb{E}[X]$.
Now, we define the $\epsilon$-insensitive loss $\mathcal{L}_{\epsilon}(\cdot,\cdot)$ as follows:
$$ \mathcal{L}_{\epsilon}(X,x) =\max\left(0,|X-x|-\epsilon\right)=\mathbb{I}_{|X-x|\geq\epsilon}\left(|X-x|-\epsilon\right) $$
We define the function $\mathcal{R}_{\epsilon}(x)$ as follows:
$$ \mathcal{R}_{\epsilon}(x) = \mathbb{E}\left[\mathcal{L}_{\epsilon}(X,x)\right] = \mathbb{E}\left[\max\left(0,|X-x|-\epsilon\right)\right]$$
I am wondering if it is possible to derive an abstract result similar to the one for the quadratic loss, but for the $\epsilon$-insensitive loss. The problem is that the $\max$ function makes the problem much harder and I don't really know how to proceed.
Any ideas on whether it is possible to derive a similar result? 

A:

Assuming $X$ is an absolutely continuous random variable with pdf $f$ and CDF $F$. Then
$$ \begin{align} \mathcal{R}_{\epsilon}(x) 
& = \int_{-\infty}^{+\infty} \max(0, |u - x|-\epsilon)f(u)du \\
& = \int_{-\infty}^{x - \epsilon} (x - u - \epsilon)f(u)du 
+ \int_{x+\epsilon}^{+\infty} (u - x - \epsilon)f(u)du \\
\end{align}$$
Differtiating with respect to $x$ (assuming all the regularity conditions are met), we have
$$ \begin{align} \mathcal{R}'_{\epsilon}(x) 
& = \int_{-\infty}^{x - \epsilon} f(u)du 
- \int_{x+\epsilon}^{+\infty} f(u)du \\
& = F(x-\epsilon) - [1 - F(x+\epsilon)] \\
& = F(x - \epsilon) + F(x+\epsilon)-1
\end{align}$$
By the properties of CDF, we know that 
$$\lim_{x\to-\infty}\mathcal{R}'_{\epsilon}(x)  = -1, 
\lim_{x\to\infty}\mathcal{R}'_{\epsilon}(x)  = 1$$
it is increasing from $-1$ to $1$. And since $F$ is continuous, by intermediate value theorem the exist a value $x_0$ such that $\mathcal{R}'_{\epsilon}(x) = 0$, and we conclude that such value is the minimum point of $\mathcal{R}_{\epsilon}(x)$.
Therefore in general you need to solve the equation
$$ F(x - \epsilon) + F(x+\epsilon) = 1$$
for the minimum point.

