English: For Regression problems the Mean Square Error (MSE) is most frequently used.

However the MSE is very sensitive to outliers, and a differentiable approximation to the Mean Absolute Error (MAE) is desired. In particular if the training data for regression comes from a disctribution with a ”heavy tail”, the presence of many outliers will lead to large prediction errors if MSE is used during training. The MAE weights data equally and is robust to the presence of outliers, however its use is limited since it is not differentiable at the origin. Another problem with the MAE loss is that it has a large derivative (±1) close to the origin leading to oscillations about the minimum during gradient descent. ANN training loss functions have to be differentiable, since the Backpopagation algorithm requires the loss function to be differentiable. Thus differentiable and computationally cheap alternatives to the MSE loss like the Huber loss and loss have been explored in the past.

From the figure it is clear that the SMAE loss proposed in https://arxiv.org/abs/2303.09935 is less affected by outliers than other loss functions. The robustness of the SMAE loss can be attributed to its close approximation to the ideal MAE loss compared to other loss functions.