Diagonal matrix

In linear algebra, a diagonal matrix is a matrix in which the entries outside the main diagonal are all zero; the term usually refers to square matrices. Elements of the main diagonal can either be zero or nonzero. An example of a 2×2 diagonal matrix is $\left[{\begin{smallmatrix}3&0\\0&2\end{smallmatrix}}\right]$ , while an example of a 3×3 diagonal matrix is $\left[{\begin{smallmatrix}6&0&0\\0&5&0\\0&0&4\end{smallmatrix}}\right]$ . An identity matrix of any size, or any multiple of it is a diagonal matrix called a scalar matrix, for example, $\left[{\begin{smallmatrix}0.5&0\\0&0.5\end{smallmatrix}}\right]$ . In geometry, a diagonal matrix may be used as a scaling matrix, since matrix multiplication with it results in changing scale (size) and possibly also shape; only a scalar matrix results in uniform change in scale.

Definition

As stated above, a diagonal matrix is a matrix in which all off-diagonal entries are zero. That is, the matrix $D = (d i, j)$ with $n$ columns and $n$ rows is diagonal if $\forall i,j\in \{1,2,\ldots ,n\},i\neq j\implies d_{i,j}=0.$

However, the main diagonal entries are unrestricted.

The term diagonal matrix may sometimes refer to a rectangular diagonal matrix, which is an $m$ -by- $n$ matrix with all the entries not of the form $d i, i$ being zero. For example: ${\begin{bmatrix}1&0&0\\0&4&0\\0&0&-3\\0&0&0\\\end{bmatrix}}\quad {\text{or}}\quad {\begin{bmatrix}1&0&0&0&0\\0&4&0&0&0\\0&0&-3&0&0\end{bmatrix}}$

More often, however, diagonal matrix refers to square matrices, which can be specified explicitly as a square diagonal matrix. A square diagonal matrix is a symmetric matrix, so this can also be called a symmetric diagonal matrix.

The following matrix is square diagonal matrix: ${\begin{bmatrix}1&0&0\\0&4&0\\0&0&-2\end{bmatrix}}$

If the entries are real numbers or complex numbers, then it is a normal matrix as well.

In the remainder of this article we will consider only square diagonal matrices, and refer to them simply as "diagonal matrices".

Vector-to-matrix diag operator

A diagonal matrix $\mathbf {D}$ can be constructed from a vector $\mathbf {a} ={\begin{bmatrix}a_{1}&\dotsm &a_{n}\end{bmatrix}}^{\textsf {T}}$ using the $\operatorname {diag}$ operator: $\mathbf {D} =\operatorname {diag} (a_{1},\dots ,a_{n})$

This may be written more compactly as $\mathbf {D} =\operatorname {diag} (\mathbf {a} )$ .

The same operator is also used to represent block diagonal matrices as $\mathbf {A} =\operatorname {diag} (A_{1},\dots ,A_{n})$ where each argument $A_{i}$ is a matrix.

The $\operatorname {diag}$ operator may be written as: $\operatorname {diag} (\mathbf {a} )=\left(\mathbf {a} \mathbf {1} ^{\textsf {T}}\right)\circ \mathbf {I}$ where $\circ$ represents the Hadamard product and $\mathbf {1}$ is a constant vector with elements 1.

Matrix-to-vector diag operator

The inverse matrix-to-vector $\operatorname {diag}$ operator is sometimes denoted by the identically named $\operatorname {diag} (\mathbf {D} )={\begin{bmatrix}a_{1}&\dotsm &a_{n}\end{bmatrix}}^{\textsf {T}}$ where the argument is now a matrix and the result is a vector of its diagonal entries.

The following property holds: $\operatorname {diag} (\mathbf {A} \mathbf {B} )=\sum _{j}\left(\mathbf {A} \circ \mathbf {B} ^{\textsf {T}}\right)_{ij}=\left(\mathbf {A} \circ \mathbf {B} ^{\textsf {T}}\right)\mathbf {1}$

Scalar matrix

A diagonal matrix with equal diagonal entries is a scalar matrix; that is, a scalar multiple λ of the identity matrix $I$ . Its effect on a vector is scalar multiplication by λ. For example, a 3×3 scalar matrix has the form: ${\begin{bmatrix}\lambda &0&0\\0&\lambda &0\\0&0&\lambda \end{bmatrix}}\equiv \lambda {\boldsymbol {I}}_{3}$

The scalar matrices are the center of the algebra of matrices: that is, they are precisely the matrices that commute with all other square matrices of the same size.^[a] By contrast, over a field (like the real numbers), a diagonal matrix with all diagonal elements distinct only commutes with diagonal matrices (its centralizer is the set of diagonal matrices). That is because if a diagonal matrix $\mathbf {D} =\operatorname {diag} (a_{1},\dots ,a_{n})$ has $a_{i}\neq a_{j},$ then given a matrix $\mathbf {M}$ with $m_{ij}\neq 0,$ the $(i,j)$ term of the products are: $(\mathbf {D} \mathbf {M} )_{ij}=a_{i}m_{ij}$ and $(\mathbf {M} \mathbf {D} )_{ij}=m_{ij}a_{j},$ and $a_{j}m_{ij}\neq m_{ij}a_{i}$ (since one can divide by $m_{ij}$ ), so they do not commute unless the off-diagonal terms are zero.^[b] Diagonal matrices where the diagonal entries are not all equal or all distinct have centralizers intermediate between the whole space and only diagonal matrices.^[1]

For an abstract vector space V (rather than the concrete vector space $K^{n}$ ), the analog of scalar matrices are scalar transformations. This is true more generally for a module M over a ring R, with the endomorphism algebra End(M) (algebra of linear operators on M) replacing the algebra of matrices. Formally, scalar multiplication is a linear map, inducing a map $R\to \operatorname {End} (M),$ (from a scalar λ to its corresponding scalar transformation, multiplication by λ) exhibiting End(M) as a R-algebra. For vector spaces, the scalar transforms are exactly the center of the endomorphism algebra, and, similarly, scalar invertible transforms are the center of the general linear group GL(V). The former is more generally true free modules $M\cong R^{n}$ , for which the endomorphism algebra is isomorphic to a matrix algebra.

Vector operations

Multiplying a vector by a diagonal matrix multiplies each of the terms by the corresponding diagonal entry. Given a diagonal matrix $\mathbf {D} =\operatorname {diag} (a_{1},\dots ,a_{n})$ and a vector $\mathbf {v} ={\begin{bmatrix}x_{1}&\dotsm &x_{n}\end{bmatrix}}^{\textsf {T}}$ , the product is: $\mathbf {D} \mathbf {v} =\operatorname {diag} (a_{1},\dots ,a_{n}){\begin{bmatrix}x_{1}\\\vdots \\x_{n}\end{bmatrix}}={\begin{bmatrix}a_{1}\\&\ddots \\&&a_{n}\end{bmatrix}}{\begin{bmatrix}x_{1}\\\vdots \\x_{n}\end{bmatrix}}={\begin{bmatrix}a_{1}x_{1}\\\vdots \\a_{n}x_{n}\end{bmatrix}}.$

This can be expressed more compactly by using a vector instead of a diagonal matrix, $\mathbf {d} ={\begin{bmatrix}a_{1}&\dotsm &a_{n}\end{bmatrix}}^{\textsf {T}}$ , and taking the Hadamard product of the vectors (entrywise product), denoted $\mathbf {d} \circ \mathbf {v}$ :

$\mathbf {D} \mathbf {v} =\mathbf {d} \circ \mathbf {v} ={\begin{bmatrix}a_{1}\\\vdots \\a_{n}\end{bmatrix}}\circ {\begin{bmatrix}x_{1}\\\vdots \\x_{n}\end{bmatrix}}={\begin{bmatrix}a_{1}x_{1}\\\vdots \\a_{n}x_{n}\end{bmatrix}}.$

This is mathematically equivalent, but avoids storing all the zero terms of this sparse matrix. This product is thus used in machine learning, such as computing products of derivatives in backpropagation or multiplying IDF weights in TF-IDF,^[2] since some BLAS frameworks, which multiply matrices efficiently, do not include Hadamard product capability directly.^[3]

Matrix operations

The operations of matrix addition and matrix multiplication are especially simple for diagonal matrices. Write $diag(a 1, ..., a n)$ for a diagonal matrix whose diagonal entries starting in the upper left corner are $a 1, ..., a n$ . Then, for addition, we have

$\operatorname {diag} (a_{1},\,\ldots ,\,a_{n})+\operatorname {diag} (b_{1},\,\ldots ,\,b_{n})=\operatorname {diag} (a_{1}+b_{1},\,\ldots ,\,a_{n}+b_{n})$

and for matrix multiplication,

$\operatorname {diag} (a_{1},\,\ldots ,\,a_{n})\operatorname {diag} (b_{1},\,\ldots ,\,b_{n})=\operatorname {diag} (a_{1}b_{1},\,\ldots ,\,a_{n}b_{n}).$

The diagonal matrix $diag(a 1, ..., a n)$ is invertible if and only if the entries $a 1, ..., a n$ are all nonzero. In this case, we have

$\operatorname {diag} (a_{1},\,\ldots ,\,a_{n})^{-1}=\operatorname {diag} (a_{1}^{-1},\,\ldots ,\,a_{n}^{-1}).$

In particular, the diagonal matrices form a subring of the ring of all $n$ -by- $n$ matrices.

Multiplying an $n$ -by- $n$ matrix $A$ from the left with $diag(a 1, ..., a n)$ amounts to multiplying the $i$ -th row of $A$ by $a i$ for all $i$ ; multiplying the matrix $A$ from the right with $diag(a 1, ..., a n)$ amounts to multiplying the $i$ -th column of $A$ by $a i$ for all $i$ .

Operator matrix in eigenbasis

As explained in determining coefficients of operator matrix, there is a special basis, $e 1, ..., e n$ , for which the matrix $\mathbf {A}$ takes the diagonal form. Hence, in the defining equation ${\textstyle \mathbf {A} \mathbf {e} _{j}=\sum _{i}a_{i,j}\mathbf {e} _{i}}$ , all coefficients $a_{i,j}$ with $i \neq j$ are zero, leaving only one term per sum. The surviving diagonal elements, $a_{i,i}$ , are known as eigenvalues and designated with $\lambda _{i}$ in the equation, which reduces to $\mathbf {A} \mathbf {e} _{i}=\lambda _{i}\mathbf {e} _{i}$ . The resulting equation is known as eigenvalue equation^[4] and used to derive the characteristic polynomial and, further, eigenvalues and eigenvectors.

In other words, the eigenvalues of $diag(λ 1, ..., λ n)$ are $λ 1, ..., λ n$ with associated eigenvectors of $e 1, ..., e n$ .

Properties

The determinant of $diag(a 1, ..., a n)$ is the product $a 1 \dots a n$ .
The adjugate of a diagonal matrix is again diagonal.
Where all matrices are square,
- A matrix is diagonal if and only if it is triangular and normal.
- A matrix is diagonal if and only if it is both upper- and lower-triangular.
- A diagonal matrix is symmetric.
The identity matrix $I n$ and zero matrix are diagonal.
A 1×1 matrix is always diagonal.
The square of a 2×2 matrix with zero trace is always diagonal.

Applications

Diagonal matrices occur in many areas of linear algebra. Because of the simple description of the matrix operation and eigenvalues/eigenvectors given above, it is typically desirable to represent a given matrix or linear map by a diagonal matrix.

In fact, a given $n$ -by- $n$ matrix $A$ is similar to a diagonal matrix (meaning that there is a matrix $X$ such that $X -1 AX$ is diagonal) if and only if it has $n$ linearly independent eigenvectors. Such matrices are said to be diagonalizable.

Over the field of real or complex numbers, more is true. The spectral theorem says that every normal matrix is unitarily similar to a diagonal matrix (if $AA * = A * A$ then there exists a unitary matrix $U$ such that $UAU *$ is diagonal). Furthermore, the singular value decomposition implies that for any matrix $A$ , there exist unitary matrices $U$ and $V$ such that $U * AV$ is diagonal with positive entries.

Operator theory

In operator theory, particularly the study of PDEs, operators are particularly easy to understand and PDEs easy to solve if the operator is diagonal with respect to the basis with which one is working; this corresponds to a separable partial differential equation. Therefore, a key technique to understanding operators is a change of coordinates—in the language of operators, an integral transform—which changes the basis to an eigenbasis of eigenfunctions: which makes the equation separable. An important example of this is the Fourier transform, which diagonalizes constant coefficient differentiation operators (or more generally translation invariant operators), such as the Laplacian operator, say, in the heat equation.

Especially easy are multiplication operators, which are defined as multiplication by (the values of) a fixed function–the values of the function at each point correspond to the diagonal entries of a matrix.

Notes

^ Proof: given the elementary matrix $e_{ij}$ , $Me_{ij}$ is the matrix with only the i-th row of M and $e_{ij}M$ is the square matrix with only the M j-th column, so the non-diagonal entries must be zero, and the ith diagonal entry much equal the jth diagonal entry.
^ Over more general rings, this does not hold, because one cannot always divide.

References

^ "Do Diagonal Matrices Always Commute?". Stack Exchange. March 15, 2016. Retrieved August 4, 2018.
^ Sahami, Mehran (2009-06-15). Text Mining: Classification, Clustering, and Applications. CRC Press. p. 14. ISBN 9781420059458.
^ "Element-wise vector-vector multiplication in BLAS?". stackoverflow.com. 2011-10-01. Retrieved 2020-08-30.
^ Nearing, James (2010). "Chapter 7.9: Eigenvalues and Eigenvectors" (PDF). Mathematical Tools for Physics. ISBN 978-0486482125. Retrieved January 1, 2012.

Sources

Horn, Roger Alan; Johnson, Charles Royal (1985), Matrix Analysis, Cambridge University Press, ISBN 978-0-521-38632-6

[1] Proof: given the elementary matrix $e_{ij}$ , $Me_{ij}$ is the matrix with only the i-th row of M and $e_{ij}M$ is the square matrix with only the M j-th column, so the non-diagonal entries must be zero, and the ith diagonal entry much equal the jth diagonal entry.

[2] Over more general rings, this does not hold, because one cannot always divide.

[3] "Do Diagonal Matrices Always Commute?". Stack Exchange. March 15, 2016. Retrieved August 4, 2018.

[4] Sahami, Mehran (2009-06-15). Text Mining: Classification, Clustering, and Applications. CRC Press. p. 14. ISBN 9781420059458.

[5] "Element-wise vector-vector multiplication in BLAS?". stackoverflow.com. 2011-10-01. Retrieved 2020-08-30.

[6] Nearing, James (2010). "Chapter 7.9: Eigenvalues and Eigenvectors" (PDF). Mathematical Tools for Physics. ISBN 978-0486482125. Retrieved January 1, 2012.

[a]

[b]

[1]

[2]

[3]

[4]

v t e Matrix classes
Explicitly constrained entries	Alternant Anti-diagonal Anti-Hermitian Anti-symmetric Arrowhead Band Bidiagonal Bisymmetric Block-diagonal Block Block tridiagonal Boolean Cauchy Centrosymmetric Conference Complex Hadamard Copositive Diagonally dominant Diagonal Discrete Fourier Transform Elementary Equivalent Frobenius Generalized permutation Hadamard Hankel Hermitian Hessenberg Hollow Integer Logical Matrix unit Metzler Moore Nonnegative Pentadiagonal Permutation Persymmetric Polynomial Quaternionic Signature Skew-Hermitian Skew-symmetric Skyline Sparse Sylvester Symmetric Toeplitz Triangular Tridiagonal Vandermonde Walsh Z
Constant	Exchange Hilbert Identity Lehmer Of ones Pascal Pauli Redheffer Shift Zero
Conditions on eigenvalues or eigenvectors	Companion Convergent Defective Definite Diagonalizable Hurwitz Positive-definite Stieltjes
Satisfying conditions on products or inverses	Congruent Idempotent or Projection Invertible Involutory Nilpotent Normal Orthogonal Unimodular Unipotent Unitary Totally unimodular Weighing
With specific applications	Adjugate Alternating sign Augmented Bézout Carleman Cartan Circulant Cofactor Commutation Confusion Coxeter Distance Duplication and elimination Euclidean distance Fundamental (linear differential equation) Generator Gram Hessian Householder Jacobian Moment Payoff Pick Random Rotation Seifert Shear Similarity Symplectic Totally positive Transformation
Used in statistics	Centering Correlation Covariance Design Doubly stochastic Fisher information Hat Precision Stochastic Transition
Used in graph theory	Adjacency Biadjacency Degree Edmonds Incidence Laplacian Seidel adjacency Tutte
Used in science and engineering	Cabibbo–Kobayashi–Maskawa Density Fundamental (computer vision) Fuzzy associative Gamma Gell-Mann Hamiltonian Irregular Overlap S State transition Substitution Z (chemistry)
Related terms	Jordan normal form Linear independence Matrix exponential Matrix representation of conic sections Perfect matrix Pseudoinverse Row echelon form Wronskian
Mathematics portal List of matrices Category:Matrices