6  Vectors and matrices

Vectors and matrices are among the basic building blocks of quantum chemistry. They give us a convenient way of collecting many numbers into a single mathematical object and, more importantly, of describing transformations between such collections of numbers.

For example, suppose that a molecular orbital is expanded in a basis of \(n\) functions,

\[ \phi = c_1\chi_1+c_2\chi_2+\cdots+c_n\chi_n. \]

Instead of considering the coefficients \(c_1,\ldots,c_n\) separately, we collect them in a vector

\[ \mathbf{c}=\begin{bmatrix}c_1\\c_2\\\vdots\\c_n\end{bmatrix}. \]

A vector is an object whose components describe some quantity with respect to a chosen basis. In quantum chemistry, vectors may contain orbital coefficients, amplitudes, coordinates, concentrations, values of a function on a grid, and much more.

The set of all real vectors with \(n\) components is denoted by \(\mathbb{R}^n\), while \(\mathbb{C}^n\) denotes the corresponding set of vectors with complex components. We sometimes use \(\mathbf{F}\) if the underlying number field is arbitrary.

We can add vectors component by component and multiply them by scalars. For example,

\[ \begin{bmatrix}x_1\\x_2\end{bmatrix} + \begin{bmatrix}y_1\\y_2\end{bmatrix} = \begin{bmatrix}x_1+y_1\\x_2+y_2\end{bmatrix}, \qquad \alpha \begin{bmatrix}x_1\\x_2\end{bmatrix} = \begin{bmatrix}\alpha x_1\\\alpha x_2\end{bmatrix}. \]

These operations obey a collection of familiar rules: vector addition is associative and commutative, there is a zero vector and an additive inverse, and scalar multiplication distributes over addition. A set equipped with operations satisfying these rules is called a vector space. The importance of this abstraction is that the same algebra applies to many different kinds of objects. Ordinary coordinate vectors, functions, polynomials, and quantum-mechanical states can all form vector spaces.

6.1 Matrices as transformations

Once quantities have been represented by vectors, we need a way of describing operations on them. This is one of the main roles of matrices.

An \(m\times n\) matrix is a rectangular array of numbers,

\[ A=\begin{bmatrix} A_{11} & A_{12} & \cdots & A_{1n}\\ A_{21} & A_{22} & \cdots & A_{2n}\\ \vdots & \vdots & \ddots & \vdots\\ A_{m1} & A_{m2} & \cdots & A_{mn} \end{bmatrix}. \]

We write \(A\in\mathbb{R}^{m\times n}\) when the entries are real and \(A\in\mathbb{C}^{m\times n}\) when they may be complex.

A matrix can act on a vector. If \(A\) is an \(m\times n\) matrix and \(\mathbf{x}\) has \(n\) components, then

\[ \mathbf{y}=A\mathbf{x} \]

is a vector with \(m\) components. Its components are

\[ y_i=\sum_{j=1}^n A_{ij}x_j. \]

Notice which indices are summed, which are “free”, i.e., not summed. For example,

\[ \begin{bmatrix}a&b\\c&d\end{bmatrix} \begin{bmatrix}x\\y\end{bmatrix} = \begin{bmatrix}ax+by\\cx+dy\end{bmatrix}. \]

Thus, a matrix takes one vector as input and produces another vector as output. This point of view is fundamental: matrices represent linear transformations.

If one matrix transforms \(\mathbf{x}\) into \(\mathbf{y}\) and another transforms \(\mathbf{y}\) into \(\mathbf{z}\),

\[ \mathbf{y}=B\mathbf{x}, \qquad \mathbf{z}=A\mathbf{y}, \]

then the combined transformation is

\[ \mathbf{z}=A(B\mathbf{x})=(AB)\mathbf{x}. \]

This explains why matrix multiplication is defined as it is. For \(A\in\mathbb{F}^{m\times n}\) and \(B\in\mathbb{F}^{n\times p}\), the product \(AB\in\mathbb{F}^{m\times p}\) has entries

\[ (AB)_{ij}=\sum_{k=1}^n A_{ik}B_{kj}. \]

Matrix multiplication therefore describes the composition of linear transformations. The order matters: \(AB\) means that \(B\) acts first and \(A\) acts second. Consequently, matrix multiplication is generally not commutative,

\[ AB\neq BA. \]

This is an important difference between matrices and ordinary numbers.

6.2 Transpose

So far, we have written vectors as columns. Sometimes, however, we need a vector to appear as a row. The operation that exchanges rows and columns is called the transpose.

For an \(m\times n\) matrix \(A\), its transpose \(A^T\) is the \(n\times m\) matrix defined by

\[ (A^T)_{ij}=A_{ji}. \]

For example,

\[ A=\begin{bmatrix}1&2&3\\4&5&6\end{bmatrix} \qquad\Longrightarrow\qquad A^T=\begin{bmatrix}1&4\\2&5\\3&6\end{bmatrix}. \]

In particular, the transpose turns a column vector into a row vector,

\[ \mathbf{x}= \begin{bmatrix}x_1\\x_2\\x_3\end{bmatrix} \qquad\Longrightarrow\qquad \mathbf{x}^T= \begin{bmatrix}x_1&x_2&x_3\end{bmatrix}. \]

Why is this useful? One reason is that it allows us to write the ordinary scalar product of two real vectors using matrix multiplication:

\[ \mathbf{x}^T\mathbf{y} = \begin{bmatrix}x_1&\cdots&x_n\end{bmatrix} \begin{bmatrix}y_1\\\vdots\\y_n\end{bmatrix} = \sum_{i=1}^n x_i y_i. \]

The transpose therefore occurs naturally whenever rows and columns need to be interchanged. An important identity is

\[ (AB)^T=B^TA^T. \]

Notice that the order is reversed.

6.3 Complex vectors and the Hermitian transpose

Quantum mechanics requires complex numbers, so we must extend these ideas from real vectors to complex vectors. Here an important subtlety appears.

For real vectors, the scalar product is \(\mathbf{x}^T\mathbf{y}\). If we simply used the same expression for a complex vector \(\mathbf{z}\), we would obtain

\[ \mathbf{z}^T\mathbf{z}=\sum_i z_i^2, \]

which need not even be real. This cannot serve as a sensible squared length.

Instead, the complex scalar product uses the complex conjugate of the first vector:

\[ \sum_i z_i^*z_i=\sum_i |z_i|^2. \]

This quantity is always real and non-negative. We therefore need an operation that both transposes a matrix and complex-conjugates its entries. This is the Hermitian transpose, also known as the conjugate transpose or the Hermitian adjoint.

The Hermitian transpose of \(A\) is denoted by \(A^H\), or oftentimes \(A^\dagger\), and defined by

\[ (A^H)_{ij}=A_{ji}^*. \]

Equivalently,

\[ A^H=(A^*)^T. \]

For example,

\[ A=\begin{bmatrix}1&i\\2-i&3\end{bmatrix} \qquad\Longrightarrow\qquad A^H=\begin{bmatrix}1&2+i\\-i&3\end{bmatrix}. \]

The scalar product of two complex vectors can now be written compactly as

\[ \mathbf{x}^H\mathbf{y}=\sum_i x_i^*y_i, \]

and in particular,

\[ \|\mathbf{x}\|^2 = \mathbf{x}^H\mathbf{x}=\sum_i |x_i|^2\geq 0. \]

For a real matrix, complex conjugation has no effect, and therefore

\[ A^H=A^T \quad \text{(for a real matrix)}. \]

Just as for the ordinary transpose, taking the Hermitian transpose reverses the order of a product:

\[ (AB)^H=B^H A^H. \]

The Hermitian transpose is especially important in quantum mechanics. A matrix satisfying

\[ A^H=A \]

is called Hermitian. Hermitian matrices have real eigenvalues and represent observables in the usual matrix formulation of quantum mechanics.

A matrix satisfying

\[ U^H U=UU^H=I \]

is called unitary. Unitary matrices preserve scalar products and lengths. They therefore describe, among other things, changes between orthonormal bases and quantum-mechanical time evolution.

These ideas - vectors, matrices, matrix products, transposes, and Hermitian transpose - provide the algebraic language needed for much of the linear algebra used throughout quantum chemistry.

Exercises: Basic vector and matrix calculations

Exercise 6.1 (Basic vector operations) Consider the vectors

\[ \mathbf{x}=(1,2,-1),\qquad \mathbf{y}=(3,-1,2),\qquad \mathbf{z}=(0,2,1) \]

in the vector space \(\mathbb{R}^3\). (We here write the vectors as an ordered tuple instead of a column.)

  1. Compute \(\mathbf{x}+\mathbf{y}\).

  2. Compute \(\mathbf{x}-\mathbf{y}\).

  3. Compute \(2\mathbf{x}\) and \(-3\mathbf{y}\).

  4. Compute the linear combination \(2\mathbf{x}+3\mathbf{y}\).

  5. Compute \(\mathbf{x}+2\mathbf{y}-\mathbf{z}\).

  6. Find numbers \(\alpha\) and \(\beta\) such that \(\alpha \mathbf{x}+\beta \mathbf{y}=(7,0,3)\).

  7. Verify by direct calculation that \(2(\mathbf{x}+\mathbf{y})=2\mathbf{x}+2\mathbf{y}\). Which vector-space axiom does this illustrate?

  8. Is \((1,2)\) an element of this vector space? Is \((0,0,0)\) an element? Explain briefly.

Exercise 6.2 (Basic matrix-vector products) Consider the matrices and vectors

\[ A=\begin{pmatrix}2&-1\\3&4\end{pmatrix},\qquad B=\begin{pmatrix}1&0&2\\-1&3&1\end{pmatrix},\qquad \mathbf{x}=\begin{pmatrix}3\\2\end{pmatrix},\qquad \mathbf{y}=\begin{pmatrix}1\\-2\\3\end{pmatrix}. \]

  1. Compute \(A\mathbf{x}\).

  2. Compute \(B\mathbf{y}\).

  3. Compute \(A(2\mathbf{x})\) and \(2(A\mathbf{x})\). Verify that they are equal.

  4. Can the products \(A\mathbf{y}\) and \(B\mathbf{x}\) be formed? Explain your answer in terms of the dimensions of the matrices and vectors.

Exercise 6.3 (Matrix-matrix products) Consider the matrices

\[ A=\begin{pmatrix}1&2\\3&-1\end{pmatrix},\qquad B=\begin{pmatrix}2&0\\-1&4\end{pmatrix},\qquad C=\begin{pmatrix}1&2&0\\-1&3&2\end{pmatrix}. \]

  1. Compute \(AB\).

  2. Compute \(BA\). Is \(AB=BA\)?

  3. Compute \(AC\). What are the dimensions of the resulting matrix?

  4. Can the product \(CA\) be formed? Can the product \(BC\) be formed? Explain your answers using the dimensions of the matrices.

Exercise 6.4 (Linear independence and basis) We work in the space \(V = \mathbb{R}^2\), plane vectors. We define two vectors, \[ \mathbf{b}_1 = \begin{bmatrix} 1 \\ 2 \end{bmatrix}, \quad \mathbf{b}_2 = \begin{bmatrix} -1 \\ 1 \end{bmatrix}. \]

  1. Show by elementary means that the two vectors are linearly independent, that is: If \(x_1 \mathbf{b}_1 + x_2 \mathbf{b}_2 = 0\), then \(x_1=x_2=0\) must hold.
  2. The vectors form a basis since they are linearly independent. Explain in your own words what a basis is.
  3. Make a 2D drawing or plot of them as arrows from the origin.

Exercise 6.5 (Gaussian elimination and vector decomposition) Let

\[ \mathbf{v} = \begin{bmatrix} 2\\2 \end{bmatrix} \]

Using Gaussian elimination, compute the coefficients \(x_1\) and \(x_2\) such that

\[ \mathbf{v} = x_1 \mathbf{b}_1 + x_2 \mathbf{b}_2. \]

Illustrate this decomposition in the graph from the previous exercise, using the parallelogram rule: The sum of two vectors is obtained by placing the origin of one at the end of the second, or vice versa.

Exercise 6.6 (Office coordinates) A person walks from home. First, they walk to the subway station. This is 1 km north and 500 m east as the crow flies. The subway takes the person to the city centre, which 3 km east and 2 km south of the subway station, as the crow flies. The person then walks 200 m south and 400 m west to their office.

What are the coordinates of the office, relative to the person’s home?

What is the distance to home, as the crow flies?

Exercise 6.7 (Boat sailing on the ocean) A boat is sailing on the ocean, at a constant speed 20 km/h relative to the water surface, in the northeast direction. However, the water surface is moving too, at a constant velocity. After two hours, the boat is located 20 km north and 10 km east of the starting point. What is the speed and direction of ocean drift?

Exercise 6.8 (An exercise by Robert Beezer) A three-digit number has two properties. The tens-digit and the ones-digit add up to 5. If the number is written with the digits in the reverse order, and then subtracted from the original number, the result is 792. Use a system of equations to find all of the three-digit numbers with these properties.

Exercises: More vectors and matrices

Exercise 6.9 (Compute the action of a matrix on vectors) Let \(A\) be the \(n\times n\) matrix defined by \(A_{i,i+1} = i^2\) for \(1 \leq i < n\), and zero otherwise. Compute the action of \(A\) and \(A^T\) on the standard basis for \(\mathbb{R}^n\). Compute the action of \(A\) and \(A^T\) on an arbitrary vector \(\mathbf{x}\in \mathbb{R}^n\).

Exercise 6.10 (Matrix-matrix products, rows and columns) A useful way to understand matrix-matrix multiplication is to think of a matrix as a collection of vectors. This gives two complementary interpretations of the product \(AB\):

  • \(A\) acts separately on each column of \(B\).
  • \(B\) acts separately on each row of \(A\), when the rows are regarded as row vectors.

These viewpoints are useful both for understanding matrix multiplication and for computations. For example, if the columns of \(B\) represent several vectors, then \(AB\) applies the linear transformation \(A\) to all of these vectors at once.

  1. Consider \[ A=\begin{bmatrix}1&2\\-1&3\end{bmatrix}, \qquad B=\begin{bmatrix}1&0&2\\2&-1&1\end{bmatrix}. \] Write \(B\) as a collection of its three column vectors, \[ B=[\mathbf b_1,\mathbf b_2,\mathbf b_3]. \] Write down \(\mathbf b_1\), \(\mathbf b_2\), and \(\mathbf b_3\) explicitly.

  2. Compute \(A\mathbf b_1\), \(A\mathbf b_2\), and \(A\mathbf b_3\) separately. Put the three resulting vectors next to each other as the columns of a new matrix: \[ [A\mathbf b_1,A\mathbf b_2,A\mathbf b_3]. \]

  3. Compute \(AB\) directly using ordinary matrix multiplication. Compare your answer with part b.

  4. Now consider the general case. Let \[ A\in M(n,m,\mathbb F), \qquad B\in M(m,o,\mathbb F), \] and write the columns of \(B\) as \[ B=[\mathbf b_1,\mathbf b_2,\ldots,\mathbf b_o]. \] Explain why \[ AB=[A\mathbf b_1,A\mathbf b_2,\ldots,A\mathbf b_o]. \] In other words, explain why the \(j\)th column of \(AB\) is \(A\mathbf b_j\).

  5. There is a similar interpretation in terms of rows. Write \(A\) using its row vectors, \[ A= \begin{bmatrix} \mathbf a_1^T\\ \mathbf a_2^T\\ \vdots\\ \mathbf a_n^T \end{bmatrix}. \] Show that \[ AB= \begin{bmatrix} \mathbf a_1^T B\\ \mathbf a_2^T B\\ \vdots\\ \mathbf a_n^T B \end{bmatrix}. \] Thus, the \(i\)th row of \(AB\) is the row vector \(\mathbf a_i^T B\).

  6. Suppose the columns of \(B\) are several vectors that we want to transform using the same linear transformation \(A\). Explain why the identity from part d means that we can collect all the vectors into one matrix \(B\) and perform a single matrix-matrix multiplication \(AB\).

Exercise 6.11 Show that each column of \(C = AB\) is a linear combination of the columns of \(A\) with coefficients determined by \(B\), \[ \mathbf{c}_i = \sum_j \mathbf{a}_j B_{ji}. \] Show a similar result for the rows of \(C\).