MishBackward#
MishBackward operation computes gradient for Mish.
\[\dst & = \diffdst * \frac{e^{\src} * \omega}{\delta^{2}}\]
where
\[\begin{split}\omega & = e^{3\src} + 4 * e^{2\src} + e^{\src} * (4 * \src + 6) + 4 * (\src + 1) \\
\delta & = e^{2\src} + 2 * e^{\src} + 2\end{split}\]
Operation Attributes#
MishBackward operation does not support any attribute.
Execution Arguments#
The inputs and outputs must be provided according to the below index order when constructing an operation.
Inputs#
Index |
Argument Name |
Required or Optional |
|---|---|---|
0 |
|
Required |
1 |
|
Required |
Outputs#
Index |
Argument Name |
Required or Optional |
|---|---|---|
0 |
|
Required |
Supported Data Types#
MishBackward operation supports the following data type combinations.
Src |
Diff_dst |
Diff_src |
|---|---|---|
f32 |
f32 |
f32 |
f16 |
f16 |
f16 |
bf16 |
bf16 |
bf16 |