Tensor Product of Matrices

1. Tensor Product is a Special Type of Kronecker Product carried out between a Co-Vector (Row Matrix) and Some Other Tensor whose Rank is Greater than or Equal to 1.
2. Tensor Product Always increases the Rank of the Tensor that the Co-Vector is operating upon.
3. Tensor Product of 2 Vectors/Column Matrices is also called Outer Product of Vectors.
4. Let's consider 2 Column Matrices (or Vectors), a \(M \times 1\) Matrix \(A\) and a \(N \times 1\) Matrix \(B\) given as follows
   
   \(A=\begin{bmatrix}a_{1} \\ a_{2} \\ \vdots \\ a_m\end{bmatrix}\hspace{.5cm}B=\begin{bmatrix}b_{1} \\ b_{2} \\ \vdots \\ b_n\end{bmatrix}\)
   
   The Tensor Product of Vector \(A\) with Vector \(B\) is given as
   
   \(A \otimes_{T} B = B^T \otimes A = A \otimes B^T = \begin{bmatrix}b_{1} & b_{2} & \cdots & b_n\end{bmatrix} \otimes \begin{bmatrix}a_{1} \\ a_{2} \\ \vdots \\ a_m\end{bmatrix} = \begin{bmatrix}a_{1} \\ a_{2} \\ \vdots \\ a_m\end{bmatrix} \otimes \begin{bmatrix}b_{1} & b_{2} & \cdots & b_n\end{bmatrix} = \begin{bmatrix}a_1 \cdot b_1 & a_1 \cdot b_2 & \cdots & a_1 \cdot b_n\\a_2 \cdot b_1 & a_2 \cdot b_2 & \cdots & a_2 \cdot b_n \\ \vdots & \vdots & \ddots & \vdots\\a_m \cdot b_1 & a_m \cdot b_2 & \cdots & a_m \cdot b_n \end{bmatrix}\)
   
   The Tensor Product of Vector \(B\) with Vector \(A\) is given as
   
   \(B \otimes_{T} A = A^T \otimes B= B \otimes A^T = \begin{bmatrix}a_{1} & a_{2} & \cdots & a_m\end{bmatrix} \otimes \begin{bmatrix}b_{1} \\ b_{2} \\ \vdots \\ b_n\end{bmatrix} = \begin{bmatrix}b_{1} \\ b_{2} \\ \vdots \\ b_n\end{bmatrix} \otimes \begin{bmatrix}a_{1} & a_{2} & \cdots & a_m\end{bmatrix} = \begin{bmatrix}a_1 \cdot b_1 & a_2 \cdot b_1 & \cdots & a_m \cdot b_1\\a_1 \cdot b_2 & a_2 \cdot b_2 & \cdots & a_m \cdot b_2 \\ \vdots & \vdots & \ddots & \vdots\\a_1 \cdot b_n & a_2 \cdot b_n & \cdots & a_m \cdot b_n \end{bmatrix}\)
   
   Please note that The Tensor Products of Vector \(A\) with Vector \(B\) and Vector \(B\) with Vector \(A\) are Transposes of each other. That is,
   
   \(A \otimes_{T} B= B^T \otimes A=A \otimes B^T = [A^T \otimes B]^T = [B \otimes A^T]^T = [B \otimes_{T} A]^T\)
5. Following example calculates the Tensor Products of Vectors \(A\) and \(B\) as given below
   
   \(A=\begin{bmatrix}2 \\ -3\end{bmatrix}\hspace{.5cm}B=\begin{bmatrix}7 \\-1 \\ 4 \end{bmatrix}\)
   
   \(A \otimes_{T} B=\begin{bmatrix}7 & -1 & 4 \end{bmatrix}\otimes\begin{bmatrix}2 \\ -3\end{bmatrix}=\begin{bmatrix}2 \\ -3\end{bmatrix}\otimes\begin{bmatrix}7 & -1 & 4 \end{bmatrix}=\begin{bmatrix}14 & -2 & 8 \\ -21 & 3 & -12\end{bmatrix}\)
   
   \(B \otimes_{T} A=\begin{bmatrix}2 & -3\end{bmatrix} \otimes \begin{bmatrix}7 \\ -1 \\ 4 \end{bmatrix}=\begin{bmatrix}7 \\ -1 \\ 4 \end{bmatrix}\otimes\begin{bmatrix}2 & -3\end{bmatrix}=\begin{bmatrix}14 & -21 \\ -2 & 3 \\ 8 & -12\end{bmatrix}\)
   
   
6. Tensor Products between any 2 Vectors can be Calculated Using Matrix Multiplication/Dot Product/Inner Product as given in the following
   
   \(AB^T= A \otimes_{T} B = A \otimes B^T = B^T \otimes A\)
   
   \(BA^T= B \otimes_{T} A = B \otimes A^T = A^T \otimes B\)
   
   You can use the Tensor Product of Vectors Calculator to calculate Tensor Product between any 2 Vectors.
7. Kronecker Product and Tensor Product between any 2 Vectors \(A\) and \(B\) are related as follows
   
   \(A \otimes B = Vec(B \otimes_{T} A)\)
   
   \(B \otimes A = Vec(A \otimes_{T} B)\)
8. Following example shows/proves the Relation between Kronecker Product and Tensor Products between 2 Vectors \(A\) and \(B\) as given below
   
   \(A=\begin{bmatrix}2 \\ -3\end{bmatrix}\hspace{.5cm}B=\begin{bmatrix}7 \\-1 \\ 4 \end{bmatrix}\)
   
   \(A \otimes B=\begin{bmatrix}7\\-1\\4\end{bmatrix} \otimes \begin{bmatrix}2\\-3\end{bmatrix}=\begin{bmatrix}14\\-2\\8\\-21\\3\\-12\end{bmatrix}\)
   
   \(B \otimes_{T} A=\begin{bmatrix}2&-3\end{bmatrix} \otimes \begin{bmatrix}7\\-1\\4\end{bmatrix}=\begin{bmatrix}14&-21\\-2&3\\8&-12\end{bmatrix}\)
   
   \(\Rightarrow Vec(B \otimes_{T} A)=\begin{bmatrix}14\\-2\\8\\-21\\3\\-12\end{bmatrix}=A \otimes B\)
   
   \(B \otimes A=\begin{bmatrix}7\\-1\\4\end{bmatrix}\otimes \begin{bmatrix}2\\-3\end{bmatrix} =\begin{bmatrix}14\\-21\\-2\\3\\8\\-12\end{bmatrix}\)
   
   \(A \otimes_{T} B=\begin{bmatrix}7&-1&4\end{bmatrix} \otimes \begin{bmatrix}2\\-3\end{bmatrix} =\begin{bmatrix}14&-2&8\\-21&3&-12\end{bmatrix}\)
   
   \(\Rightarrow Vec(A \otimes_{T} B)=\begin{bmatrix}14\\-21\\-2\\3\\8\\-12\end{bmatrix}=B \otimes A\)
   
   
9. Tensor Products between any 2 Vectors having Same Number of Elements Result in Special Kind of Square Matrices known as Dyads.
10. Tensor Product of 3 Vectors \(A\), \(B\) and \(C\) is calculated using Kronecker Product as given below
    
    \(A \otimes_{T} B \otimes_{T} C = C^T \otimes ( B^T \otimes A)\)
    
    Likewize if \(N\) Vectors are given as \(A_1\), \(A_1\), \(A_1\), ..., \(A_N\) then the Tensor Product between these Vectors is calculated as following
    
    \(A_1 \otimes_{T} A_2 \otimes_{T} A_3 \otimes_{T} ... A_N= {A_N}^T \otimes ( {A_{N-1}}^T \otimes ... {A_3}^T \otimes ({A_2}^T \otimes A_1))\)
    
    
11. Tensor Product between a Vector (Column Matrix) \(A\) and a Matrix \(B\) is calculated using Kronecker Product as given below
    
    \(A \otimes_{T} B = B \otimes_{T} A = A^T \otimes B \)