Linear Algebra

Linear Operators and Matrices

$$A(\mathrm{\Sigma }{a}_i|v_i⟩)=\mathrm{\Sigma }{a}_{i}A|v_i⟩$$

Now, see a linear operator is just a matrix. Suppose $A:V\to W$ and $|v_1⟩,|v_2⟩,...,|v_m⟩$ are basis of $V$ and $|w_1⟩,|w_2⟩,...,|w_n⟩$ is a basis of $W$ then,

$$A|v_j⟩={\mathrm{\Sigma }}_i{A}_{ij}|w_i⟩$$

Inner Products

Ok so imagine an operation $(\mathrm{\_},\mathrm{\_}):V\times V\to \mathbb{C}$ such that the following shit holds ok?

1. $(|v⟩,{\mathrm{\Sigma }}_i{\lambda }_i|w_i⟩)={\mathrm{\Sigma }}_i{\lambda }_i(|v⟩,|w_i⟩)$
2. $(|v⟩,|w⟩)=(|w⟩,|v⟩{)}^{\ast }$
3. $(|v⟩,|v⟩)\ge 0$ and $=0$ iff $|v⟩$

In finite dimensions, inner product space i.e., vector spaces equipped with inner prducts for all $|v⟩\in$ vector space $=$ Hilbert Space

Consider $|i⟩\mathrm{\&}|j⟩$ to be orthonormal basis, we have —

$$⟨v|w⟩=({\mathrm{\Sigma }}_i{v}_i|i⟩,{\mathrm{\Sigma }}_j{w}_j|j⟩)={\mathrm{\Sigma }}_i{\mathrm{\Sigma }}_j{v}_i^{\ast }{w}_j{\delta }_{ij}={\mathrm{\Sigma }}_i{v}_i^{\ast }{w}_i=|v{⟩}^{†}|w⟩$$

Norm of a vector

$$||v||=\sqrt{⟨v|v⟩}$$

We can say that $|v⟩$ is normalized iff $||v||=1$.

A set of $|a_i⟩$ vectors is orthonormal if $⟨a_i|a_j⟩={\delta }_{ij}$ i.e., $\mathrm{\forall }i\ne j⟨a_i|a_j⟩=0$ and $⟨a_i|a_j⟩=1\mathrm{\forall }i=j$.

Gram Schmidt: for orthonormal basis

$$|v_{k+1}⟩=\frac{|w_{k+1}⟩-{\mathrm{\Sigma }}_{i=1}^k⟨v_i|w_{k+1}⟩|v_i⟩}{|||w_{k+1}⟩-{\mathrm{\Sigma }}_{i=1}^k⟨v_i|w_{k+1}⟩|v_i⟩||},|v_1⟩=|w_1⟩/|||w_1⟩||$$

Outer Product

$$|w⟩⟨v|(|v^{\prime }⟩)=|w⟩|v{⟩}^{†}|v^{\prime }⟩=|w⟩⟨v|v^{\prime }⟩=⟨v|v^{\prime }⟩|w⟩$$

• From this notion we obtain the completeness relation, ${\mathrm{\Sigma }}_i|i⟩⟨i|=I$.
• $A=I_{w}A{I}_v={\mathrm{\Sigma }}_{ij}|w_j⟩⟨w_j|A|v_i⟩⟨v_i|={\mathrm{\Sigma }}_{ij}⟨w_j|A|v_i⟩|w_j⟩⟨v_i|$
• Cauchy Schwarz: $⟨v|v⟩⟨w|w⟩\ge ⟨v|w⟩⟨w|v⟩=|⟨v|w⟩{|}^2$

Hilbert Space

A Hilbert Space $\mathcal{H}$ is complete which means that every Cauchy sequence of vectors admits in the space itself. Under this hypothesis there exist Hilbert bases also known as complete orthonormal systems of vectors in $\mathcal{H}$.

For any orthonormal basis of $\mathcal{H}$, we have the following.

$$\begin{gathered} \text{Orthonormality}\equiv ⟨{\psi }_i|{\psi }_j⟩={\delta }_{ij} \\ \text{Completeness}\equiv \sum _i|{\psi }_i⟩⟨{\psi }_i|=I \end{gathered}$$

Eigenvectors and Eigenvalues

Under a given linear transformation $A$, $A|v⟩=\lambda |v⟩$ where $\mathrm{\exists }|v⟩$ s.t. they do not get shifted off their span.

All such vectors are referred as eigenvectors and $(A-\lambda I)|v⟩=0⟹det|A-\lambda I|=0$ gives all possible eigenvalue.

If all ${\lambda }_i\ge 0$, it is positive semidefinite and if they are $>0$, it is positive definite.

Eigenspace

It is the space of all vectors with a given eigenvalue $\lambda$. When an eigenspace is more than one dimensional, we call it degenerate.

Adjoints and Hermitian

Suppose $A:V\to V$ then $\mathrm{\exists }{A}^{†}:V\to V$ such that $\mathrm{\forall }|v⟩,|w⟩\in V$ we have,

$$(|v⟩,A|w⟩)=(A^{†}|v⟩,|w⟩)$$

This operator is called as the adjoint or Hermitian conjugate of the operator $A$.

$$(|v⟩,A|w⟩)=⟨v|A|w⟩={\overrightarrow{v}}^{†}A\overrightarrow{w}=(A^{†}\overrightarrow{v}{)}^{†}\overrightarrow{w}=(A^{†}|v⟩,|w⟩)$$

• We have, $(AB{)}^{†}=B^{†}{A}^{†}$
• $|v{⟩}^{†}=⟨v|$

Some defintions

• Normal matrices: $A{A}^{†}=A^{†}A$
• Hermitian matrices: $A^{†}=A$
• Unitary matrices: $A{A}^{†}=I$
• A normal matrix is Hermitian if and only if it has real eigenvalues.
• If $⟨x|A|x⟩\ge 0,\mathrm{\forall }|x⟩$ then $A$ is positive semi-definite and has positive eigenvalues.

Some properties

• If a Hermitian matrix has positive eigenvalues then it is positive semi-definite.
• If $M=A{A}^{†}$ then it is both Hermitian and positive semi-definite.
• All positive semi-definite operators are Hermitian, by definition.

Spectral Decomposition

Definition: A linear operator is diagonalizable if and only if it is normal.

Some notes and derivation regarding the above:

$A\overrightarrow{v}=\lambda \overrightarrow{v}={\mathrm{\Sigma }}_i{\lambda }_{ij}\overrightarrow{q_i}$ where $q_i$'s are linearly independent eigenvalues of $A$.

$AQ=Q\mathrm{\Lambda }$ where $Q=\left[\begin{array}{cccc}{q}_1& q_2& \dots & q_n\end{array}\right]⟹A=Q\mathrm{\Lambda }{Q}^{-1}$

$$A=IAI=(P+Q)A(P+Q)=PAP+QAP+PAQ+QAQ$$

$$⟹A=\lambda P^2+0+0+QAQ$$

$$⟹A=\lambda P^2+QAQ$$

Matrices and Vectors

In the following statements we are dealing with $\{|i⟩\}$ as a orthonormal basis set.

$$I=\mathrm{\Sigma }|i⟩⟨i|$$

$$|\psi ⟩=\mathrm{\Sigma }{\sigma }_i|i⟩,\text{ where }{\sigma }_i=⟨i|\psi ⟩$$

Now, to represent a operator or linear transformation as matrix in orthonormal basis.

$$\begin{gathered} A_{ij}=⟨i|A|j⟩ \\ A={\mathrm{\Sigma }}_{i,j}⟨i|A|j⟩|i⟩⟨j| \\ \text{tr}(A)={\mathrm{\Sigma }}_i⟨i|A|i⟩ \end{gathered}$$

Now diagonalization for any normal matrix.

$$\begin{gathered} M={\mathrm{\Sigma }}_i{\lambda }_i|i⟩⟨i|={\mathrm{\Sigma }}_i{\lambda }_i{P}_i,\text{where}{P}_i^{†}=P_i \\ f(M)={\mathrm{\Sigma }}_{i}f({\lambda }_i)|i⟩⟨i| \end{gathered}$$

where ${\lambda }_i$ are eigenvalues of $M$ under a given orthonormal basis set $\{|i⟩\}$ for vector space $V$, each $|i⟩$ is an eigenvector of $M$ with eigenvalue ${\lambda }_i$.

If $M$ is Hermitian, all eigenvalues $({\lambda }_i\text{ s})$ are real.

Tensor Products

• $z|vw⟩=(z|v⟩)\otimes (|w⟩)=(|v⟩)\otimes (z|w⟩)$
• $(|v_1⟩+|v_2⟩)\otimes |w⟩=|v_{1}w⟩+|v_{2}w⟩$
• $|v⟩\otimes (|w_1⟩+|w_2⟩)=|v{w}_1⟩+|v{w}_2⟩$
• $|\psi {⟩}^{{\otimes }^k}=|\psi ⟩\otimes \dots \otimes |\psi ⟩\text{ k times}$
• $(A\otimes B{)}^{†}=A^{†}\otimes B^{†}$

Linear Product

$A\otimes B$ forms the linear operator that acts on $V\otimes W$ vector space givern that $A$ acts on $V$ and $B$ acts on $W$.

$$(A\otimes B)({\mathrm{\Sigma }}_i{a}_i|v_i⟩\otimes |w_i⟩)={\mathrm{\Sigma }}_i{a}_{i}A|v_i⟩\otimes B|w_i⟩$$

Inner Product

$$({\mathrm{\Sigma }}_i{a}_i|v_i⟩\otimes |w_i⟩,{\mathrm{\Sigma }}_i{a}_j|v_j^{\prime }⟩\otimes |w_j^{\prime }⟩)={\mathrm{\Sigma }}_{ij}{a}_i^{\ast }{b}_j⟨v_i|v_j^{\prime }⟩⟨w_i|w_j^{\prime }⟩$$

Trace

Properties of trace are given below as follows.

• $\text{tr}(A)={\mathrm{\Sigma }}_i{A}_{ii}$
• $\text{tr}(A)={\mathrm{\Sigma }}_i⟨i|A|i⟩$ for orthonormal basis
• $\text{tr}(AB)=\text{tr}(BA)$
• $\text{tr}(zA+B)=z\cdot \text{tr}(A)+\text{tr}(B)$

The above properties yield certain implications as follows.

• $\text{tr}(UA{U}^{†})=tr(A)$
• $\text{tr}(A|\psi ⟩⟨\psi |)={\mathrm{\Sigma }}_i⟨i|A|\psi ⟩⟨\psi |i⟩$
• $\text{tr}(A)=\sum _i{\lambda }_i$, $\text{det}(A)=\prod _i{\lambda }_i$ with algebraic multiplicities

$$||A||=\sqrt{tr(A^{†}A)}$$

Partial Trace

Entanglement excludes the possibility of associating state vectors with individual subsystems. Therefore, we introduce density matrices and the corresponding idea of reduction preformed with partial trace.

$$\text{tr}(A\otimes B)=\text{ tr}(A)\cdot \text{tr}(B)$$

$${\rho }_{AB}:{\mathcal{H}}_{\mathcal{A}}\otimes {\mathcal{H}}_{\mathcal{B}}\stackrel{{\text{ tr}}_B}{\to }{\rho }_A:{\mathcal{H}}_{\mathcal{A}}$$

$${\text{ tr}}_B(AB)=A\text{ tr}(B)$$

Hilbert-Schimdt Inner Product

$L_v$ forms the vector space of operators over the Hilbert space $V$. Then, we can show that $L_v$ is also a Hilbert space with $\text{tr}(A^{†}B)$ as the inner product operator on $L_v\times L_v$.

Also, we have $div(L_v)=dim(V{)}^2$.

Commutator and Anti-commutator

$$\begin{gathered} [A,B]=AB-BA \\ \{A,B\}=AB+BA \end{gathered}$$

Theorem of Simultaneous Diagonalization

Suppose $A$ and $B$ are both Hermitian matrices, then $[A,B]=0$ iff $\mathrm{\exists }$ orthonormal basis such that both $A$ and $B$ are diagonal with respect to that basis.

Polar Value Decomposition

If $A$ is any linear operator and $U$ is a unitary then $J,K$ are positive operators, such that

$$A=UJ=KU,\text{ where }J=\sqrt{A^{†}A}\text{ and }K=\sqrt{A{A}^{†}}$$

Moreover, if $A^{-1}$ exists, then $U$ is unique.

Singular Value Decomposition

SVD in general is given as follows.

$$U\mathrm{\Sigma }{V}^T$$

It generalizes the eigen decomposition of a square normal matrix with an orthonormal eigen basis to any $m\times n$ matrix.

$\mathrm{\Sigma }$ is an $m\times n$ rectangular diagonal matrix with non-negative real numbers on the diagonal (called singular values).

Corollary: If $A$ is a square matrix and $\mathrm{\exists }U,V$ unitaries then $D$ is a diagonal matrix, such that

$$A=UDV$$

where $D$ has non-negative values.

Corollary: If $A$ has non-negative eigenvalues then, $A=U^{†}DU$ is possible where $D$ has non-negative values.

• If $A$ is square both SVD and EVD exist but might not be same.
• If $A$ is a square symmetric matrix both SVD and EVD exist and are equivalent.
• If $A$ is non-square only SVD is possible.

Rank of a matrix

Rank $=$ number of dimensions in column space.

• The row rank is the largest number of rows of $A$ that constitute a linearly independent set.
• The column rank is the largest number of columns of $A$ that constitute a linearly independent set. Moreover, column-rank $=$ row-rank for $A\in {\mathbb{R}}^{m\times n}$.

$$rank(A\in {\mathbb{R}}^{m\times n})\le min(m,n)$$

Matrix is called full rank if equality holds.

• $rank(A^T)=rank(A)$
• $rank(AB)\le min(rank(A),rank(B))$
• $rank(A+B)\le rank(A)+rank(B)$

Projection and Spaces

$$Proj(y;A)=argmi{n}_{v\in R(A)}||v-y|{|}_2=A(A^{T}A{)}^{-1}{A}^{T}y$$

• $\mathcal{N}(A)=\{x\in {\mathbb{R}}^n:Ax=0\}$ denotes all vectors in ${\mathbb{R}}^n$ that land at the origin after transformation. It is also called kernel.
• $\mathcal{R}(A)=\{v\in {\mathbb{R}}^m:v=Ax,x\in {\mathbb{R}}^n\}$ denotes the space spanned by the transformed basis vectors in ${\mathbb{R}}^n$.
• $\mathcal{R}(A^T)$ and $\mathcal{N}(A)$ are orthogonal spaces which together span ${\mathbb{R}}^n$.

Determinant $\ne 0$ implies that the matrix has an inverse.

Quadratic Forms

Reminder: we are in real $\mathbb{R}$ space.

$$x^{T}Ax=(x^{T}Ax{)}^T=x^T(\frac{1}{2}A+\frac{1}{2}{A}^T)x$$

Moore-Penrose Pseudoinverse

$$A^{\text{left inv}}=(A^{†}A{)}^{-1}{A}^{†}$$

$$A^{\text{right inv}}=A^{†}(A{A}^{†}{)}^{-1}$$

This is a pseudo inverse formalism with left and right inverses.

$$\begin{gathered} A^{\text{left inv}}A=I \\ A{A}^{\text{right inv}}=I \end{gathered}$$

Row Echelon Forms

A matrix is in row echelon form if:

• All rows consisting of only zeroes are at the bottom.
• The leading coefficient (also called the pivot) of a nonzero row is always strictly to the right of the leading coefficient of the row above it.

A matrix is in reduced row echelon form if:

• It is in row echelon form.
• The leading entry in each nonzero row is a 1 (called a leading 1).
• Each column containing a leading 1 has zeros in all its other entries.

Spectral Decomposition

Any normal operator $M$ on a vector space $V$ is diagonal with respect to some orthonormal basis for $V$.

Conversely, any diagonalizable operator is normal.

Proof

$$M=(P+Q)M(P+Q)=PMP+QMP+PMQ+QMQ=PMP+QMQ$$

Now we have

$$QM=QM(P+Q)=QMQ\text{ and }Q{M}^{†}=Q{M}^{†}Q$$

$$QMQQ{M}^{†}Q=QM{M}^{†}Q=Q{M}^{†}MQ=Q{M}^{†}QQMQ$$

Thus, if $M$ is normal then $M=PMP+QMQ$ where $PMP=\lambda P^2=\lambda P$ and thus is diagonalizable wrt orthonormal basis for $P$. Similarly, $QMQ$ is also diagonalizable wrt some orthonormal basis for $Q$.

Thus, $M$ is diagonalizable for orthonormal basis of the entire vector space.

$$M=\mathrm{\Sigma }{\lambda }_i|i⟩⟨i|=\mathrm{\Sigma }{\lambda }_i{P}_i$$

Polar Value Decomposition

Let $A$ be a matrix on vector space $V$. Then there exists unitary $U$ and positive operators $J$ and $K$ such that, $A=UJ=KU$ where the unique positive operators shall satisfy the equations $J\equiv \sqrt{A^{†}A}$ and $K\equiv \sqrt{A{A}^{†}}$.

Moreover, if $A$ is invertible then $U$ is unique.

Singular Value Decomposition

Let $A$ be a square matrix. Then there exist unitary matrices $U$ and $V$, and a diagonal matrix $D$ with non-negative entries such that $A=UDV$.

The diagonal elements of $D$ are called the singular values of $A$.

Proof

From polar value decomposition we have, $A=SJ=STD{T}^{†}=(ST)D(T^{†})=UDV$.