Question-48

Which of the following statements is/are correct about the rectified linear unit (ReLU) activation function defined as \(\text{ReLU}( x) =\max( x,0)\), where \(x\in \mathbb{R}\)?

• ReLU is continuous everywhere

• ReLU is differentiable everywhere

• ReLU is not differentiable at \(x=0\)

• \(\text{ReLU}( x) =\text{ReLU}( ax) ,\)for all \(a\in \mathbb{R}\)

NoteAnswer

• ReLU is continuous everywhere

• ReLU is differentiable everywhere

• ReLU is not differentiable at \(x=0\)

• \(\text{ReLU}( x) =\text{ReLU}( ax) ,\)for all \(a\in \mathbb{R}\)

NoteSolution

ReLU is piece-wise linear. It is continuous everywhere but not differentiable at \(x=0\). The last option is clearly incorrect.