mail  mail@stemandmusic.in

    

call  +91-9818088802

Gradient Vector, Jacobian Matrix and Hessian Matrix

1. Gradient or Gradient Vector of any Multi-Variable Function is the Vector of all Partial Derivatives of the Function with respect to all its Variables. For example, given a Function \(F\) of \(N\) Variables \(x_1, x_2, \cdots, x_n\), the Gradient of Function \(F\) is given as
   
   \(grad\hspace{2mm}F = \begin{bmatrix}{\Large \frac{\partial F}{\partial x_1}} \\ {\Large \frac{\partial F}{\partial x_2}} \\ \vdots \\ {\Large \frac{\partial F}{\partial x_n}}\end{bmatrix}\)
2. Given a set of \(K\) Multi-Variable Functions \(F_1, F_2, F_3, \cdots, F_k\) dependent on same set of \(N\) Variables \(x_1, x_2, x_3, \cdots, x_n\), the Jacobian Matrix \(J\) is given by the Partial Derivatives of the Functions as follows
   
   \(J = \begin{bmatrix}{\Large \frac{\partial F_1}{\partial x_1}} & {\Large \frac{\partial F_1}{\partial x_2}} & \cdots & {\Large \frac{\partial F_1}{\partial x_n}} \\ {\Large \frac{\partial F_2}{\partial x_1}} & {\Large \frac{\partial F_2}{\partial x_2}} & \cdots & {\Large \frac{\partial F_2}{\partial x_n}} \\ \vdots & \vdots & \ddots & \vdots \\ {\Large \frac{\partial F_k}{\partial x_1}} & {\Large \frac{\partial F_k}{\partial x_2}} & \cdots & {\Large \frac{\partial F_k}{\partial x_n}} \end{bmatrix}=\begin{bmatrix}(grad\hspace{2mm}F_1)^T \\ (grad\hspace{2mm}F_2)^T \\ \vdots \\ (grad\hspace{2mm}F_k)^T\end{bmatrix} \)
   
   The Rows of any Jacobian Matrix are an composed of Transpose of the Gradients of contributing Functions.
   
   The Transpose of Jacobian Matrix \(J\) given above is given as follows
   
   \(J^T = \begin{bmatrix}{\Large \frac{\partial F_1}{\partial x_1}} & {\Large \frac{\partial F_2}{\partial x_1}} & \cdots & {\Large \frac{\partial F_k}{\partial x_1}} \\ {\Large \frac{\partial F_1}{\partial x_2}} & {\Large \frac{\partial F_2}{\partial x_2}} & \cdots & {\Large \frac{\partial F_k}{\partial x_2}} \\ \vdots & \vdots & \ddots & \vdots \\ {\Large \frac{\partial F_1}{\partial x_n}} & {\Large \frac{\partial F_2}{\partial x_n}} & \cdots & {\Large \frac{\partial F_k}{\partial x_n}} \end{bmatrix}=\begin{bmatrix}grad\hspace{2mm}F_1 & grad\hspace{2mm}F_2 & \cdots & grad\hspace{2mm}F_k\end{bmatrix} \)
   
   The Columns of Transpose of any Jacobian Matrix are an composed of Gradients of contributing Functions.
3. A Hessian Matrix is a Square Matrix of Second-Order Partial Derivatives of a Multi-Variable Scalar Function. For example, given a Function \(F\) of \(N\) Variables \(x_1, x_2, x_3, \cdots, x_n\), the \(N \times N\) Hessian Matrix for the function is given as
   
   \(H_F = \begin{bmatrix}{\Large \frac{{\partial}^2 F}{{\partial x_1}^2}} & {\Large \frac{{\partial}^2 F}{\partial x_1\partial x_2}} & {\Large \frac{{\partial}^2 F}{\partial x_1\partial x_3}} & \cdots & {\Large \frac{{\partial}^2 F}{\partial x_1\partial x_n}} \\ {\Large \frac{{\partial}^2 F}{\partial x_2\partial x_1}} & {\Large \frac{{\partial}^2 F}{{\partial x_2}^2}} & {\Large \frac{{\partial}^2 F}{\partial x_2\partial x_3}} & \cdots & {\Large \frac{{\partial}^2 F}{\partial x_2\partial x_n}} \\ {\Large \frac{{\partial}^2 F}{\partial x_3\partial x_1}} & {\Large \frac{{\partial}^2 F}{\partial x_3\partial x_2}} & {\Large \frac{{\partial}^2 F}{{\partial x_3}^2}} & \cdots & {\Large \frac{{\partial}^2 F}{\partial x_3\partial x_n}} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ {\Large \frac{{\partial}^2 F}{\partial x_n\partial x_1}} & {\Large \frac{{\partial}^2 F}{\partial x_n\partial x_2}} & {\Large \frac{{\partial}^2 F}{\partial x_n\partial x_3}} & \cdots & {\Large \frac{{\partial}^2 F}{{\partial x_n}^2}} \end{bmatrix}\)