Home > Informal Classroom Notes > Understanding (real- and complex-valued) Inner Products

Understanding (real- and complex-valued) Inner Products

This short note addresses two issues.

  • How can we intuitively understand a complex-valued inner product?
  • If an inner product structure is given to a vector space, how can we understand the resulting geometry?

Since inner products are associated with angles, and since we can understand angles, there is temptation to interpret inner products in terms of angles. I advocate against this being the primary means of interpreting inner products.

An inner product \langle \cdot, \cdot \rangle induces the norm \| x \| = \sqrt{\langle x,x \rangle}. Importantly, the inner product can be recovered from this norm by the polarisation identity. Therefore, understanding the geometry of an inner product is the same as understanding the geometry of the norm, and for the latter, it often suffices to consider what the unit ball looks like. For me, the norm is the primary structure giving the space its geometry.

What then is the purpose of the inner product? Not all norms have the same properties. Under some norms, projection onto a closed subspace may not be unique, for example. When interested in shortest-norm optimisation problems, the most desirable situation to be in is for the square of the norm to be quadratic, since then differentiating it produces a linear equation. In infinite dimensions, what does it mean for the square of a norm to be quadratic?

The presence of an inner product structure means the square of the norm is quadratic. Furthermore, the inner product “decomposes” the norm in a way that gives direct access to the derivative of the norm squared.

The remaining issue is how to understand complex-valued inner products. Given the above, the natural starting place is to consider endowing a complex vector space with a norm. Keeping the axioms of a real-valued normed vector space seems sensible; it implies that scaling a vector by e^{\jmath \theta} does not change its norm (because \| e^{\jmath \theta} x \| = | x |\,\| x \| = \| x \|).

Then one asks what it means for the square of a norm to be quadratic. From the real-valued case, one guesses that one wants to be able to represent the square of the norm as a bilinear form: \| x \|^2 = \langle x, x \rangle, where \langle \cdot, \cdot \rangle is linear in each of its arguments. Following the letter of the law, this would mean \| \alpha x \|^2 = \langle \alpha x, \alpha x \rangle = \alpha^2 \langle x,x \rangle = \alpha^2 \|x\|^2. In the complex case though, \alpha^2 need not equal |\alpha|^2. This explains why one tweaks the definition and instead considers sesquilinear forms which are linear in one argument and conjugate linear in the other: \langle \alpha x, x \rangle = \alpha \langle x, x \rangle = \langle x, \bar\alpha x \rangle. Indeed, one then correctly has that \|\alpha x \|^2 = \langle \alpha x, \alpha x \rangle = \alpha \bar\alpha \langle x , x \rangle = |\alpha|^2 \|x\|^2. With this tweak, one can verify that the complex-valued case works the same way as the real-valued case.

By treating the norm as the primary structure, one does not have to worry about giving an intuitive meaning to the inner product of two vectors not being a purely real-valued number; the inner product is there merely to expose the square of the norm as being quadratic. A complex-valued inner product is recoverable from its norm and hence no geometric information is lost. (Of course, orthogonality remains an important concept.) If one really wanted, one could play around with examples in \mathbb{C}^2 to get a better feel for what it means for \langle x , y \rangle = \jmath, for example, however, unless one encounters a particular problem encountering this level of detail, thinking in terms of norms is cleaner and more efficient. (If \langle x, y \rangle = r e^{\jmath\theta} then \langle x, e^{\jmath \theta} y \rangle = r, so that by “rotating” a complex-valued vector in the two-dimensional real-valued vector space that it spans, one can always return to thinking about real-valued inner products.)

  1. July 12, 2014 at 9:48 pm

    Respected Sir,
    Thanks a lot for this. It is really refreshing to see a Math related article on this blog after a long time. I enjoyed the earlier ones on Differential Geometry very much. Is it possible to save these informal notes as pdf files for my own reading pleasure???
    I agree that the norm induces a much general structure. The only normed spaces which allow for the parallelogram law, however, are those in which the norms have been induced by inner products, right??. How would this be interpreted in the context of your statements? Is it the length of the vector that lends a space “geometry” or the angles between pairs of vectors?? I believe in the case of manifolds, Riemannian manifolds work with inner products while Finsler manifolds work with more general norms. So how would one analyse the geometries?
    Wouldn’t it be that the angles are the tools which allow for a richer geometry rather than just the lengths??
    I hope my questions make sense,

    • July 13, 2014 at 12:11 am

      Thanks for your interest. Pressing the “print” button, then “save to pdf”, might work. Correct, a norm comes from an inner product if and only if it satisfies the parallelogram law. While many bounds in practice come from the triangle inequality, the parallelogram law gives an additional bound: if the norms of x and y are fixed, then \|x+y\| and \|x-y\| must change in opposite ways, in that as one gets bigger, the other must get smaller. This can be visualised by squashing a parallelogram. As one diagonal gets smaller, the other gets bigger. It is not wrong to state all the geometry comes from the norm. If you arrange all the points to be the correct distance apart, then they automatically form the correct angles between themselves (because the norm uniquely determines the inner product). Certainly, some norms lead to a “nicer” or at least a better understood geometry, and if you like, some norms (those satisfying the parallelogram law) allow for an angle to be specified. At the end of the day though, how you choose to view it is, of course, personal preference. While everything I said still applies to each individual tangent space of a manifold, understanding the geometry of manifolds themselves is considerably harder. It is true that Riemannian manifolds have a richer geometry than Finsler manifolds, but that does not contradict what I wrote because I restricted my comments to vector spaces. The linear structure of a vector space greatly simplifies matters.

  2. July 13, 2014 at 2:47 am

    Thank you.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: