A remarkable parallel between artificial and biological learning systems is that both feature an elementary nonlinearity, the firing threshold: the linear response of a unit (an artificial or a biological neuron) is fed through a firing rate nonlinearity, which clips the response of the unit under a certain threshold. We discuss insights obtained from neuroscience why this humble nonlinearity is effective in supporting cutting-edge performance in applications ranging from scientific image analysis to multi-billion dollar commercial applications. We approach this question from an appealing idea that is phrased in neuroscience as representational untangling: the idea that high dimensional signals that are hopelessly nonlinearly entangled become linearly separable through computational processes. Signatures of such computations can be identified in the visual cortex where complex image content such as the identity of faces can be linearly decoded irrespective of nuisance variables, such as pose, lighting, or orientation. It remains a burning question, however, what elementary computations can contribute to representational untangling. We point out that the  basic but ubiquitous form of local nonlinearity the firing threshold can achieve untangling under nuisance variable uncertainty. We argue that the efficiency of this nonlinearity in achieving easily readable codes lies in its capability to balance between two computational goals: preservation of information and sparsification of neural responses. We show through recordings from visual cortical neurons that the threshold of biological neurons is in a range that is optimal for representational untangling.