Eigen-Values and Eigen-Vectors of a Square Matrix

1. Generally, Pre-Multiplying any \(N \times N\) Square Matrix with an \(N \times 1\) Vector (Column Matrix) can Scale the Vector, Change the Direction/Orientation of the Vector or Both.
   
   The Eigen-Vectors of any \(N \times N\) Square Matrix are List of All \(N\times 1\) Non-NULL Vectors which when Pre-Multiplied with the Matrix only Scales the Vectors without changing the Line Direction/Orientation of the Vectors. The Value/Factor by which any Eigen-Vector gets Scaled is the corresponding Eigen-Value for the Eigen-Vector of the Matrix.
2. The following explains the Steps for Finding Eigen-Values and Eigen-Vectors of any Square Matrix
   
   Let \(A\) be a \(N\times N\) Square Matrix and \(X\) be one of \(N \times 1\) Eigen-Vectors of the Matrix \(A\) given as follows
   
   \(A=\begin{bmatrix} a_{11} & a_{12} & ... & a_{1n}\\ a_{21} & a_{22} & ... & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\a_{n1} & a_{n2} & ... & a_{nn}\end{bmatrix}\hspace{.5cm},X=\begin{bmatrix}x_1\\x_2\\\vdots\\x_n\end{bmatrix}\)
   
   Let \(\lambda\) be the Factor/Value by which the Matrix \(A\) Scales the Vector \(X\). Therefore , as per definition of Eigen-Vectors and Eigen-Values we have
   
   \(AX=\lambda X\)   ...(1)
   
   The above equation can also be written as
   
   \(AX=\lambda IX\)   ...(where \(I\) is the \(N \times N \) Identity Matrix)
   
   \(\Rightarrow AX-\lambda IX=0\)
   
   \(\Rightarrow (A-\lambda I)X=0\)   ...(2)
   
   \(\Rightarrow \Bigg (\begin{bmatrix} a_{11} & a_{12} & ... & a_{1n}\\ a_{21} & a_{22} & ... & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\a_{n1} & a_{n2} & ... & a_{nn}\end{bmatrix}-\lambda \begin{bmatrix} 1 & 0 & ... & 0\\ 0 & 1 & ... & 0 \\ \vdots & \vdots & \ddots & \vdots \\0 & 0 & ... & 1\end{bmatrix}\Bigg )\begin{bmatrix}x_1\\x_2\\\vdots\\x_n\end{bmatrix}=0\)
   
   \(\Rightarrow \Bigg (\begin{bmatrix} a_{11} & a_{12} & ... & a_{1n}\\ a_{21} & a_{22} & ... & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\a_{n1} & a_{n2} & ... & a_{nn}\end{bmatrix}- \begin{bmatrix} \lambda & 0 & ... & 0\\ 0 & \lambda & ... & 0 \\ \vdots & \vdots & \ddots & \vdots \\0 & 0 & ... & \lambda\end{bmatrix}\Bigg )\begin{bmatrix}x_1\\x_2\\\vdots\\x_n\end{bmatrix}=0\)
   
   \(\Rightarrow \begin{bmatrix} a_{11}-\lambda & a_{12} & ... & a_{1n}\\ a_{21} & a_{22}-\lambda & ... & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\a_{n1} & a_{n2} & ... & a_{nn}-\lambda\end{bmatrix}\begin{bmatrix}x_1\\x_2\\\vdots\\x_n\end{bmatrix}=0\)   ...(3)
   
   The Eigen-Vectors of Matrix \(A\) can be found out by Finding Non-Trivial Solutions of System of Homogeneous Linear Equations given in equation (3) above.
   
   For this, the values of \(\lambda\) can be found as follows
   
   Let \(B=\begin{bmatrix} a_{11}-\lambda & a_{12} & ... & a_{1n}\\ a_{21} & a_{22}-\lambda & ... & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\a_{n1} & a_{n2} & ... & a_{nn}-\lambda\end{bmatrix}\)
   
   For equation (3) to have Non Trivial Solutions the Matrix \(B\) must be Singular i.e. Determinant of Matrix \(B\) must be 0. That is
   
   \(|B|=\begin{vmatrix} a_{11}-\lambda & a_{12} & ... & a_{1n}\\ a_{21} & a_{22}-\lambda & ... & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\a_{n1} & a_{n2} & ... & a_{nn}-\lambda\end{vmatrix}=0\)   ...(4)
   
   Expanding the Determinant in the equation (4) above we get the Characteristic Polynomial Equation for Matrix \(A\).
   
   \({\lambda}^N\hspace{.2cm}+\hspace{.2cm}(-1)^{1} T_1 {\lambda}^{N-1}\hspace{.2cm}+\hspace{.2cm}(-1)^{2} T_2 {\lambda}^{N-2}\hspace{.2cm}+\hspace{.2cm}(-1)^{3} T_3 {\lambda}^{N-3}\hspace{.2cm}+\hspace{.2cm}\cdots\hspace{.2cm} +\hspace{.2cm}(-1)^{N-1} T_{N-1}x\hspace{.2cm}+\hspace{.2cm}(-1)^{N}D =0\)   ...(5)
   
   where \(T_1, T_2, T_3, ..., T_{N-1}\) are the Traces of K^th Principal Minors of Matrix \(A\) and \(D\) is the Determinant Value of Matrix \(A\). The Roots of the Polynomial Equation as given by equation (5) give the values of \(\lambda\) and hence the Eigen Values of the Matrix \(A\).
   
   Putting each of these Eigen Values of the Matrix \(A\) in equation (3) and solving for the Non-Trivial Solutions give us the different Eigen-Vectors of Matrix \(A\).
3. Following are some of the Properties of Eigen-Values of a Matrix
   1. A \(N \times N\) Matrix has \(N\) Eigen-Values.
   2. Any \(N \times N\) Matrix and it's Transpose have Same Eigen-Values.
   3. The Product of all Eigen-Values of a Matrix is Same as its Determinant Value.
   4. The Sum of all Eigen-Values of a Matrix is Same as the Trace of the Matrix.
   5. For any Non-Singular Matrix \(A\) having Eigen-Values \(\lambda_1, \lambda_2, ... , \lambda_n\), the Eigen-Values of \(A^{-1}\) are \(\frac{1}{\lambda_1}, \frac{1}{\lambda_2}, ... ,\frac{1}{\lambda_n}\).
   6. For any Matrix \(A\) having Eigen-Values \(\lambda_1, \lambda_2, ... , \lambda_n\), the Eigen-Values of \(cA\) are \(c\lambda_1, c\lambda_2, ... , c\lambda_n\), where \(c\) is any Non-Zero Numerical Constant.
   7. For any Matrix \(A\) having Eigen-Values \(\lambda_1, \lambda_2, ... , \lambda_n\), the Eigen-Values of \(A^k\) are \({\lambda_1}^k, {\lambda_2}^k, ... , {\lambda_n}^k\), where \(k\) is any Non-Zero Integer.
   8. The Eigen-Values of all Hermitian Matrices are Real.
   9. The Eigen-Values of all Skew-Symmetric Matrices / Anti-Hermitian Matrices are either 0 or Pure Imaginary.
   10. The Eigen-Values of all Orthogonal / Unitary Matrices are either Real Values \(\pm\hspace{1mm}1\) or Complex Numbers having Absolute Value \(1\).
   11. The Number of times any Eigen-Value gets repeated for a Matrix is called the Algebraic Multiplicity of the Eigen-Value. The Number of Linearly Independent Eigen-Vectors corresponding to any Eigen-Value of a Matrix is called the Geometric Multiplicity of the Eigen-Value. Algebraic Multiplicity of any Eigen-Value is Always Lesser Than or Equal To its Geometric Multiplicity.
4. Following are some of the Properties of Eigen-Vectors of a Matrix
   1. For any Non-Singular Matrix \(A\), the Eigen-Vectors of \(A^{-1}\) are Same as Eigen-Vectors of \(A\).
   2. Eigen-Vectors corresponding to Different Eigen-Values are Linearly Independent. Eigen-Vectors corresponding to Same Eigen-Value May or May not be Linearly Independent.
   3. If \(X\) is an Eigen-Vector of Matrix \(A\), then so are \(cX\), where \(c\) is any Non-Zero Numerical Constant.
   4. Eigen-Vectors of any Orthogonal/Unitary Matrix , are Orthogonal/Unitary.