AN INFORMAL APPROACH TO FORMAL INNER PRODUCTS

Richard Hammack

The fact that linear algebra is a distillation of our perception of physical space gives teachers and students of the subject a vast source of heuristic arguments. Almost every concept or result in finite–dimensional linear algebra can be illustrated or explained with a well–chosen geometric figure or schematic diagram.

The formal definition of an inner product seems to be an exception. Students who appreciate geometric reasoning are often mystified when, usually in their second linear algebra course, they first encounter this definition. They agree that an inner product gives a space a sense of length and orthogonality, but they are perplexed by the fact that its formal properties seem to have no geometric rationale. “Why,” they ask, “is the definition phrased the way it is?”

In teaching a second–semester course recently, I devised an answer to that question. My strategy is the reverse of the standard one. I begin with intuitive notions about length and orthogonality, and derive the formal inner product from them, rather than the other way around. This article is an outline of my approach.

When introducing the formal definition of an inner product, I first remind the students of the standard inner product on ℝ^{n}. They have been exposed to it in their previous courses, and have an understanding of how it gives ℝ^{n} metric properties. Then I give the formal definition of an inner product, pointing out that the standard inner product on ℝ^{n} satisfies this definition.

An inner product on a vector space V over a field 𝔽, where 𝔽 is either the real numbers ℝ or the complex numbers ℂ, is a function ⟨ , ⟩: V × V → 𝔽 with the following properties. If u, v, w ∈ V, and c ∈ 𝔽, then

1. ⟨u + v, w⟩ = ⟨u, w⟩ + ⟨v, w⟩

2. ⟨cv, w⟩ = c⟨v, w⟩

3. ⟨w, w⟩ > 0 if w ≠ 0

4. ⟨v, w⟩ = \overline{⟨w, v⟩}. Further, from properties 1, 2, and 4, it follows that

5. ⟨u, v + w⟩ = ⟨u, v⟩ + ⟨u, w⟩

6. ⟨v, cw⟩ = \overline{c}⟨v, w⟩. The bar represents complex conjugation, which is superfluous when 𝔽 = ℝ [1].

To explain the reasons behind properties 1–6, I tell my students to first consider the case of real vector spaces, that is to assume V is a finite dimensional vector space over 𝔽 = ℝ. (We momentarily postpone the complex case 𝔽 = ℂ.) I ask them to imagine that V is magically endowed with length and orthogonality. I then explain that we are going to make three intuitively reasonable assumptions about the behavior of this length and orthogonality, and that these assumptions will lead naturally to a function ⟨ , ⟩: V × V → ℝ satisfying properties 1–6.

The first assumption involves length. Intuitively, the length of a vector w in V should be a nonnegative real number which we will denote as ∥w∥. If w ≠ 0, we view the span of w as a copy of the real line in V. Multiplying w by a number c ∈ ℝ scales w along this line, and we expect that the length of w gets scaled by a factor of |c|. This means ∥cw∥ = |c| ∥w∥. This is our first assumption.

\underline{Assumption A}: There is a length function ∥ ∥: V → ℝ satisfying ∥w∥ > 0 whenever w ≠ 0, and ∥cw∥ = |c| ∥w∥ for all w ∈ V and c ∈ 𝔽.

Next, consider orthogonality. Our experience with physical space suggests that two vectors v and w can be orthogonal, and we express this relationship as v ⊥ w. Intuition demands that a nonzero vector w is not orthogonal to itself, nor to any of its nonzero scalar multiples. Moreover, the set span(w)^{⊥} of all vectors orthogonal to every element of span(w) is a subspace of V having dimension dim(V) − 1. Therefore, V = span(w) ⊕ span(w)^{⊥}, and the following assumption is reasonable.

\underline{Assumption B}: There is a symmetric relation ⊥ on V with the property that for any vector w ∈ V, the set span(w)^{⊥} = {v ∈ V |v ⊥ cw for all c ∈ 𝔽} is a subspace, and V = span(w) ⊕ span(w)^{⊥}.

A triangle in V can be thought of as two independent vectors v and w (regarded as “sides”), plus the additional side v − w. A triangle is right if two of its sides are orthogonal. Our final assumption is that similarity of right triangles is a meaningful concept.

\underline{Assumption C}: If two right triangles share a common (nonright) angle, then they are similar in the usual Euclidean sense. That is, ratios of corresponding sides are equal.

Suppose now that these assumptions hold for our finite-dimensional space V over ℝ. If w ∈ V is a nonzero vector, then Assumption B guarantees the existence of a projection operator E_{w}: V → V whose range is the one-dimensional space span(w), and whose kernel is the hyperplane span(w)^{⊥}. Notice that for any v ∈ V, the vector E_{w}(v) is a scalar multiple of w. This observation leads to the definition of a function ⟨ , ⟩: V × V → 𝔽. We will subsequently verify that this function satisfies Properties 1–6, so we call it the inner product on V.

\underline{Definition 1}: Given Assumptions A, B, and C, the inner product on V is the function ⟨ , ⟩: V ×V → 𝔽 defined as follows. If w is zero, then ⟨v, w⟩ = 0. Otherwise,

E_{w}(v) = ⟨v, w⟩\frac{w}{∥w∥^{2}} .

Here is the intuitive picture. The vector E_{w}(v) is in the one-dimensional space span(w) = span(w/∥w∥^{2}). The inner product ⟨v, w⟩ is the real number that w/∥w∥^{2} must be multiplied by to equal E_{w}(v). This is illustrated in Figure 1.

Figure 1.

The number ⟨v, w⟩ gives information about the spatial relationship between v and w. Notice that ⟨v, w⟩ = 0 if and only if v ∈ ker(E_{w}), meaning v ∈ span(w)^{⊥}, so v is orthogonal to w. Also, since w = E_{w}(w) = ⟨w, w⟩w/∥w∥^{2}, we have ⟨w, w⟩ = ∥w∥^{2}.

Now we verify that ⟨ , ⟩ satisfies properties 1–4. Properties 1 and 2 are just rephrasings of the fact that the projection E_{w} is linear. By Definition 1, properties 1 and 2 are obviously valid when w = 0, so let us assume w ≠ 0. To justify property 1, just notice that since E_{w}(u + v) = E_{w}(u) + E_{w}(v), Definition 1 gives

⟨u + v, w⟩\frac{w}{∥w∥^{2}} = ⟨u, w⟩\frac{w}{∥w∥^{2}} + ⟨v, w⟩\frac{w}{∥w∥^{2}} ,

so ⟨u + v, w⟩ = ⟨u, w⟩ + ⟨v, w⟩. Likewise, since E_{w}(cv) = cE_{w}(v), we have

⟨cv, w⟩\frac{w}{∥w∥^{2}} = c⟨u, w⟩\frac{w}{∥w∥^{2}} ,

so ⟨cv, w⟩ = c⟨v, w⟩, which is property 2.

Property 3 states the intuitively obvious fact that the projection of a nonzero vector to its own span produces a nonzero vector. If w is nonzero, then

w = E_{w}(w) = ⟨w, w⟩\frac{w}{∥w∥^{2}} ,

which means ⟨w, w⟩ = ∥w∥^{2}, so ⟨w, w⟩ > 0 by Assumption A.

We will next prove Property 4, which, in our case 𝔽 = ℝ, states that ⟨v, w⟩ = ⟨w, v⟩. This property is certainly true if v ⊥ w, because both ⟨v, w⟩ and ⟨w, v⟩ are zero. Thus, assume that v and w are not orthogonal, that is neither E_{v}(w) nor E_{w}(v) is zero. There are two possible configurations that can happen here, illustrated in Figures 2A and 2B. In Figure 2A, E_{w}(v) and E_{v}(w) are both positive scalar multiples of w and v, respectively, and in Figure 2B they are both negative scalar multiples. Thus, ⟨v, w⟩ and ⟨w, v⟩ are either both positive or both negative. In either case, Assumption C gives

\frac{∥E_{w}(v)∥}{∥v∥} = \frac{∥E_{v}(w)∥}{∥w∥},

and using Definition 1 and Assumption 1, this becomes

\frac{|⟨v, w⟩|}{∥w∥ ∥v∥} = \frac{|⟨w, v⟩|}{∥v∥ ∥w∥}.

From this it follows that ⟨v, w⟩ = ⟨w, v⟩, which is property 4 in the case 𝔽 = ℝ. (It should be mentioned that Assumption C suggests right triangles conform to our usual Euclidean expectations. For instance, in Figure 2A, E_{v}(w) is not a negative scalar multiple of v. Assumption C can be made more mathematically precise, but its informal wording is better suited to classroom exposition.)

Figure 2A. Figure 2B.

We have now justified properties 1–4 for real vector spaces (that is for 𝔽 = ℝ), and properties 5 and 6 follow immediately from these.

The remainder of this article concerns the complex case 𝔽 = ℂ. It turns out that in this case Assumptions A, B, and C are still reasonable, but we need to be thoughtful about their interpretation. If w is a nonzero vector, then we view span(w) as a copy of the complex plane V. Multiplication of w by a scalar c = re^{iθ} ∈ ℂ should be a rotation of w by theta radians in this plane, followed by a scaling by the factor r = |c| (see Figure 3). Thus, as in the case 𝔽 = ℝ, we expect that ∥cw∥ = |c| ∥w∥, which is the content of Assumption A.

Figure 3.

Likewise, Assumption B still makes sense as long as we agree that a nonzero vector should never be orthogonal to any nonzero multiple of itself. This leads to what is, at first glance, a somewhat less intuitive phenomenon: Assumption B implies that w and iw are not othogonal even though they may “look” orthogonal in span(w). This is actually reasonable. Intuition suggests that a nonzero scalar multiple of a vector w should not be orthogonal to w. Thus, in asserting that Assumption B holds for complex vector spaces, we are abstracting to ℂ^{n} this “obvious” fact about ℝ^{n}.

Again, we define the inner product ⟨ , ⟩ by Definition 1. The only difference is that now ⟨v, w⟩ can be a complex number. Given v, w ∈ V, E_{w}(v) is in the one-dimensional space span(w) = span(w/∥w∥^{2}). The inner product ⟨v, w⟩ is the scalar (possibly a complex number) that w/∥w∥^{2} must be multiplied by to equal E_{w}(v). This is illustrated in Figure 4. (Compare Figures 1 and 4.)

Figure 4.

Properties 1, 2, and 3 follow exactly as in the real case, but it takes a little more work to justify property 4. To do this, we first derive property 6.

Property 6 is clearly true if either w or c is zero. Suppose then that neither c nor w is zero. Notice that the projections E_{w} and E_{cw} are equal, for they are both projections onto the space span(w) = span(cw), and both have null space span(w)^{⊥} = span(cw)^{⊥}. Thus, E_{w}(v) = E_{cw}(v), which by Definition 1 means

\begin{aligned}⟨v, w⟩\frac{w}{∥w∥^{2}} & = ⟨v, cw⟩\frac{cw}{∥cw∥^{2}} = ⟨v, cw⟩\frac{cw}{|c|^{2}∥w∥^{2}} \\ & = ⟨v, cw⟩\frac{cw}{c\overline{c}∥w∥^{2}} = \frac{⟨v, cw⟩}{\overline{c}} \frac{w}{∥w∥^{2}} \end{aligned} .

(Assumption B was used in the second equality.) Looking at the first and last terms in this expression, we see ⟨v, cw⟩ = \overline{c}⟨v, w⟩, which is property 6.

Finally, we turn to property 4. First, we are going to show first that property 4 is true when ⟨v, w⟩ is real, in which case we want to show ⟨v, w⟩ = ⟨w, v⟩. Certainly this is true when v ⊥ w, for then ⟨v, w⟩ = 0 = ⟨w, v⟩. So suppose v and w are not orthogonal, that is neither ⟨v, w⟩ nor ⟨w, v⟩ is zero. Now, since ⟨v, w⟩ is a nonzero real number, we have a situation that is illustrated in Figure 2, and as in the case 𝔽 = ℝ, this gives ⟨v, w⟩ = ⟨w, v⟩.

In general, ⟨v, w⟩ will be a nonreal number, and we want to verify ⟨v, w⟩ = \overline{⟨w, v⟩}. Since

⟨\overline{⟨w, v⟩}w, v⟩ = \overline{⟨w, v⟩}⟨w, v⟩ ∈ ℝ,

what was said in the previous paragraph implies

⟨\overline{⟨w, v⟩}w, v⟩ = ⟨v, \overline{⟨w, v⟩}w⟩.

Combining this observation with property 6 gives

\overline{⟨w, v⟩}⟨w, v⟩ = ⟨\overline{⟨w, v}⟩w, v⟩ = ⟨v, \overline{⟨w, v}⟩w⟩ = ⟨w, v⟩⟨v, w⟩.

Comparing the first and last expressions of this equality, we get the desired result ⟨v, w⟩ = \overline{⟨w, v⟩}.

We have now confirmed that Assumptions A, B, and C lead to a function ⟨ , ⟩ satisfying properties 1–6. On the other hand, given a function ⟨ , ⟩ satisfying properties 1–6, and defining ∥v∥ = \sqrt{⟨v, v⟩}, and v ⊥ w if ⟨v, w⟩ = 0, one easily verifies Assumptions A, B and C. Consequently, these assumptions are equivalent to the existence of an inner product ⟨ , ⟩.

I believe this geometric presentation of inner products gives students a firmer conceptual ground on which to build than does the standard axiomatic presentation. I can introduce inner products from this perspective in a 50-minute class, and still have some time left over to talk about some related topics, such as the Cauchy-Schwartz inequality. I have had success with this method, and hope that other instructors find it useful.

Mathematics Subject Classification (2000): 15-01

\underline{Reference}

1. K. Hoffman and R. Kunze, Linear Algebra, second edition, Prentice Hall, 1971.

Richard Hammack Department of Mathematics Randolph–Macon College P. O. Box 5005 Ashland, Virginia 23005 email: rhammack.rmc.edu