激活函数(Activation functions)

Sigmoid:$$\quad a=\frac{1}{1 + e^{-x}}$$

Tanh: $$a=\frac{e^{x}-e^{-x}}{e^{x}+e^{-x}}$$

Relu: $$a=max(a,0.0)$$