**Normalizing Flows and Its Friends (Part 1)**¶

Here is some pre-requisite knowledge required before we move to Normalizing Flows.

**1. Jacobian Matrix**¶

Jacobian is a just matrix containing all partial derivatives between each input and output. It looks scary, but in fact, it is just a fancy name for a partial derivative matrix.

**For example, if**

z = \begin{bmatrix} z_{1} \\ z_{2} \end{bmatrix}

x = \begin{bmatrix} x_{1} \\ x_{2} \end{bmatrix}

**and**

z = f(x)

x = f^{-1}(z)

**then,**

J_{f} = \begin{bmatrix} \frac{\partial x_{1}}{\partial z_{1}} & \frac{\partial x_{1}}{\partial z_{2}} \\
\frac{\partial x_{2}}{\partial z_{1}} & \frac{\partial x_{2}}{\partial z_{2}} \end{bmatrix}

J_{f^{-1}} = \begin{bmatrix} \frac{\partial z_{1}}{\partial x_{1}} & \frac{\partial z_{1}}{\partial x_{2}} \\
\frac{\partial z_{2}}{\partial x_{1}} & \frac{\partial z_{2}}{\partial x_{2}} \end{bmatrix}

J_{f} J_{f^{-1}} = I

**2. Determinant**¶

Determinant is a scalar value, which describes the area/volume enclosed by all the vectors in a matrix.

**For example, if**

A = \begin{bmatrix} a & b \\ c & d \end{bmatrix}

then,

det(A) = ad - bc

The most important thing is that:

det(A) = {\frac{1}{A^{-1}}}

det(J_{f}) = {\frac{1}{J_{f^{-1}}}}

In a 2D space, it looks like this:

**3. Change of Variable Theorem**¶

p(x') \mid det {\begin{bmatrix} \Delta x_{11} & \Delta x_{21} \\
\Delta x_{12} & \Delta x_{22} \end{bmatrix}} \mid = \pi(z') \Delta {z_{1}} \Delta {z_{2}}

and this becomes to

p(x') \mid det(J_{f}) \mid = \pi(z')

p(x') = \pi(z') \mid {det(J_{f^{-1}})} \mid