MATMUL, DOT_PRODUCT and TRANSPOSE
Three intrinsics do the arithmetic of linear algebra: matmul multiplies matrices, dot_product is the inner product of two vectors, and transpose swaps the two dimensions of a matrix. They are standard, they check the shapes they can see, and they are the right way to write a product that does not need a tuned library. This lesson runs all three and then writes the same product with two loops to show what the intrinsic replaces.
A matrix times a vector
matmul(a, x) with a rank-two a and a rank-one x is the product that takes one number per row of a: the dot product of each row with x. The result is a rank-one array whose length is the number of rows.
The file matmul.f90 builds a two by three matrix with reshape, which fills it column by column:
program matmul_demo
implicit none
real :: a(2, 3), b(3, 2), x(3)
real :: y(2), c(2, 2), at(3, 2)
a = reshape([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], [2, 3])
b = reshape([1.0, 0.0, 0.0, 1.0, 1.0, 1.0], [3, 2])
x = [1.0, 2.0, 3.0]
y = matmul(a, x)
c = matmul(a, b)
at = transpose(a)
print *, "a :", a
print *, "x :", x
print *, "y :", y
print *, "c :", c
print *, "at :", at
print *, "dot :", dot_product(x, x)
end program matmul_demo
gfortran -Wall -Wextra -std=f2018 matmul.f90 -o matmul
./matmul
a : 1.00000000 2.00000000 3.00000000 4.00000000 5.00000000 6.00000000
x : 1.00000000 2.00000000 3.00000000
y : 22.0000000 28.0000000
c : 1.00000000 2.00000000 9.00000000 12.0000000
at : 1.00000000 3.00000000 5.00000000 2.00000000 4.00000000 6.00000000
dot : 14.0000000
Read the first line as a matrix: the print shows the six values in memory order, so the first column is 1 and 2, the second is 3 and 4, the third is 5 and 6. The product with x is 1*1 + 3*2 + 5*3 = 22 for the first row and 2 + 8 + 18 = 28 for the second, which is what the output holds.
A matrix times a matrix
Two rank-two arguments multiply as matrices: the columns of the first must match the rows of the second, and the result has the rows of the first and the columns of the second. Here a is two by three and b is three by two, so c is two by two, printed in memory order as c(1,1) c(2,1) c(1,2) c(2,2).
The numbers agree with the definition: c(1,2) = 1*1 + 3*1 + 5*1 = 9 and c(2,2) = 2 + 4 + 6 = 12, so the second column of c is 9 and 12. A vector argument on either side is promoted: matmul(x, a) for a rank-one x is the product of a row vector with a, and the result has the length of the columns.
TRANSPOSE and DOT_PRODUCT
transpose(a) is the same values with the two indices swapped, so the three by two result of the last line is a read the other way round: 1 3 5 is its first row and 2 4 6 its second. The intrinsic copies the data, which is the cost to remember for a large matrix in an inner loop.
dot_product(x, x) is the sum of the products of two vectors of the same length, the length squared here: 1 + 4 + 9 = 14. It is the same value as sum(x * x) and the same as matmul of a vector with itself in the rank-one form; the three spellings are the same arithmetic, and the intrinsic is the one a reader recognises.
The same product by hand
The intrinsic is a statement of the mathematics, not a special case of the language: two loops compute it. The program below fills a three by four matrix with a formula, multiplies it by a vector with two loops, and compares the result with matmul.
program hand_demo
implicit none
integer, parameter :: nr = 3, nc = 4
real :: a(nr, nc), x(nc), y(nr), ref(nr)
integer :: i, j
do j = 1, nc
do i = 1, nr
a(i, j) = real(i + 10 * j)
end do
end do
x = [(real(j), j = 1, nc)]
y = 0.0
do i = 1, nr
do j = 1, nc
y(i) = y(i) + a(i, j) * x(j)
end do
end do
ref = matmul(a, x)
print *, "y :", y
print *, "ref :", ref
print *, "err :", maxval(abs(y - ref))
end program hand_demo
gfortran -Wall -Wextra -std=f2018 hand.f90 -o hand
./hand
y : 310.000000 320.000000 330.000000
ref : 310.000000 320.000000 330.000000
err : 0.00000000
The rows differ by ten because the formula adds ten per column, and the products differ by ten for the same reason. The error is exactly zero here because every number is small and exactly representable; comparing two reals from a longer computation is a subject of its own, and the rule for it is a tolerance rather than an equality.
A shape the compiler refuses
The compiler knows the shapes of arrays with explicit bounds, so a product that cannot be formed is caught before the program runs:
program badmul
implicit none
real :: a(2, 3), b(2, 2)
a = 1.0
b = 2.0
print *, matmul(a, b)
end program badmul
gfortran -Wall -Wextra -std=f2018 badmul.f90 -o badmul
badmul.f90:7:18:
7 | print *, matmul(a, b)
| 1
Error: Different shape on dimension 2 for argument 'matrix_a' and dimension 1 for argument 'matrix_b' at (1) for intrinsic matmul
An allocatable or assumed-shape matrix carries its shape at run time only, and the same mismatch becomes a runtime error or a wrong answer depending on the build. The check inside the intrinsic is the reason to prefer the intrinsic over two loops: the loops would read the wrong element and say nothing.
Exercises
- Print matmul(x, a) for the same a and x. What is its length, and why?
- Multiply c by a from the first program. Which shape may the result have?
- Change the formula of hand.f90 to real(i * j) and run it again. Is the error still zero?
- Replace dot_product(x, x) with sum(x * x). Is the result identical for every array?
Questions
A new set of questions in a new order every time. Choose one answer for each, then check.
-
What does
matmul(a, x)give for a rank-two a and a rank-one x?Answer
A vector with one entry per row of a
Each entry is the dot product of a row with x.
-
What shapes may be multiplied as
matmul(a, b)?Answer
An m by n matrix and an n by p matrix
The columns of the first must match the rows of the second.
-
What does
transpose(a)do?Answer
Swaps the two indices, giving a copy with the shape reversed
The element at (i, j) moves to (j, i).
-
What does this program print?
program p implicit none real :: a(2, 2), x(2) a = reshape([1.0, 2.0, 3.0, 4.0], [2, 2]) x = [1.0, 1.0] print *, matmul(a, x) end program pAnswer
4.00000000 6.00000000
The rows are 1 3 and 2 4, so the products are 4 and 6.
-
What is
dot_product(x, x)for x = [1, 2, 3]?Answer
14.0
The squares are 1, 4 and 9, and their sum is 14.
-
What does the compiler report for
matmul(a, b)with a 2 by 3 and b 2 by 2?Answer
Error: Different shape on dimension 2 for argument matrix_a and dimension 1 for argument matrix_b
The inner dimensions do not agree, and both are known.
-
What is the shape of
c = matmul(a, b)for a 2 by 3 and b 3 by 2?Answer
2 by 2
The result takes the rows of the first and the columns of the second.
-
Which is the cost of
transpose(a)?Answer
It copies the elements into a new array
The intrinsic returns a new array with swapped indices.
-
What is printed by
print *, afor an array filled with values 1 to 6 by RESHAPE to a 2 by 3?Answer
The values in memory order: 1 2 3 4 5 6
A print writes the elements in memory order, column by column.
-
Which expression is the same arithmetic as
dot_product(x, y)?Answer
sum(x * y)
The inner product is the sum of the element products.
The programs on this page were built with GNU Fortran (Alpine 15.2.0) 15.2.0 and run on Alpine Linux 3.24.3 (x86_64) in a Docker container.
Article author: Arthur Isaev