Linear Algebra · Undergraduate

Inner Product Spaces: Length and Angle for Functions

Quick answer

An inner product generalizes the dot product: it is symmetric, linear in each slot, and positive for every nonzero vector. On functions over [−1, 1], ⟨f, g⟩ = ∫ f(x)g(x) dx is an inner product, so functions get lengths, angles and orthogonality. Then projection works as in ℝⁿ. Gram–Schmidt turns 1, x, x² into orthogonal polynomials, and projecting a function onto a subspace gives its best approximation: the line closest to eˣ on [−1, 1] is about 1.175 + 1.104x.

What you'll learn

  • Check the properties of an inner product
  • Compute norms, angles and orthogonality for functions
  • Apply Gram–Schmidt to polynomials
  • Find a best approximation by orthogonal projection

The dot product, generalized

Length, angle and projection in Rn\mathbb{R}^n all came from the dot product. An inner product on a vector space is any rule ⟨u,v⟩\langle u, v\rangle with the dot product’s key properties:

  1. ⟨u,v⟩=⟨v,u⟩\langle u, v\rangle = \langle v, u\rangle;
  2. ⟨au+bw,v⟩=a⟨u,v⟩+b⟨w,v⟩\langle au + bw, v\rangle = a\langle u, v\rangle + b\langle w, v\rangle;
  3. ⟨v,v⟩>0\langle v, v\rangle > 0 whenever v≠0v \ne 0.

For continuous functions on [−1,1][-1, 1], the rule

⟨f,g⟩=∫−11f(x) g(x) dx\langle f, g\rangle = \int_{-1}^{1} f(x)\,g(x)\,dx

qualifies. Every definition from Rn\mathbb{R}^n carries over: ∥f∥=⟨f,f⟩\|f\| = \sqrt{\langle f, f\rangle}, ff and gg are orthogonal when ⟨f,g⟩=0\langle f, g\rangle = 0, and the projection of ff onto uu is ⟨f,u⟩⟨u,u⟩u\tfrac{\langle f, u\rangle}{\langle u, u\rangle}u.

Why projection gives the best approximation

The proofs in Rn\mathbb{R}^n used only the three properties, never the coordinates. So in any inner product space, the projection f^\hat{f} of ff onto a subspace WW leaves an error f−f^f - \hat{f} orthogonal to WW. For any other gg in WW, the Pythagorean theorem gives

∥f−g∥2=∥f−f^∥2+∥f^−g∥2≥∥f−f^∥2\|f - g\|^2 = \|f - \hat{f}\|^2 + \|\hat{f} - g\|^2 \ge \|f - \hat{f}\|^2

The projection is the closest point of WW to ff, now measured by the integral of the squared error. This is least squares with infinitely many data points.

The line closest to eˣ on [−1, 1] The curve y = eˣ from −1 to 1, with two lines. The dashed tangent line 1 + x matches eˣ at x = 0 but drifts away at the ends. The best approximation 1.175 + 1.104x crosses the curve twice and stays close across the whole interval. −11123xy
  • y = eˣ
  • tangent line 1 + x
  • best line 1.175 + 1.104x
The line closest to eˣ on [−1, 1]

Worked examples

Common mistakes

Practice problems

  1. On [−1,1][-1, 1], find ⟨x,x3⟩\langle x, x^3\rangle.

    Answer

    25\tfrac{2}{5}

    Full solution

    ∫−11x4 dx=25\int_{-1}^1 x^4\,dx = \tfrac{2}{5}.

  2. On [−1,1][-1, 1], find the multiple of xx closest to x3x^3.

    Answer

    35x\tfrac{3}{5}x

    Full solution

    ⟨x3,x⟩⟨x,x⟩=2/52/3\tfrac{\langle x^3, x\rangle}{\langle x, x\rangle} = \tfrac{2/5}{2/3}.

  3. On [−1,1][-1, 1], find the best constant approximation to x2x^2.

    Answer

    13\tfrac{1}{3}

    Full solution

    ⟨x2,1⟩⟨1,1⟩=2/32\tfrac{\langle x^2, 1\rangle}{\langle 1, 1\rangle} = \tfrac{2/3}{2}, the average value of x2x^2.

  4. Are 11 and xx orthogonal on [0,1][0, 1]?

    Answer

    No

    Full solution

    ∫01x dx=12≠0\int_0^1 x\,dx = \tfrac{1}{2} \ne 0.

  5. Show that ⟨u,v⟩=2u1v1+3u2v2\langle u, v\rangle = 2u_1v_1 + 3u_2v_2 is an inner product on R2\mathbb{R}^2, and find ∥(1,1)∥\|(1, 1)\|.

    Answer

    ∥(1,1)∥=5\|(1, 1)\| = \sqrt{5}

    Full solution

    It is symmetric and linear, and ⟨v,v⟩=2v12+3v22>0\langle v, v\rangle = 2v_1^2 + 3v_2^2 > 0 for v≠0v \ne 0. For (1,1)(1, 1), 2+3=52 + 3 = 5.

  6. Is ⟨u,v⟩=u1v1−u2v2\langle u, v\rangle = u_1v_1 - u_2v_2 an inner product on R2\mathbb{R}^2?

    Answer

    No

    Full solution

    ⟨(0,1),(0,1)⟩=−1<0\langle (0, 1), (0, 1)\rangle = -1 < 0, which breaks positivity.

  7. Check that x2−13x^2 - \tfrac{1}{3} is orthogonal to 11 on [−1,1][-1, 1].

    Answer

    ∫−11(x2−13)dx=23−23=0\int_{-1}^1 \left(x^2 - \tfrac{1}{3}\right)dx = \tfrac{2}{3} - \tfrac{2}{3} = 0

    Full solution

    The integral of x2x^2 is 23\tfrac{2}{3} and that of 13\tfrac{1}{3} is 23\tfrac{2}{3}.

  8. A student finds the best quadratic approximation to exe^x on [−1,1][-1, 1] by adding the projections onto 11, xx and x2x^2 separately. Why is that wrong?

    Answer

    The coefficient formula needs an orthogonal basis, and 11 and x2x^2 are not orthogonal.

    Full solution

    ⟨1,x2⟩=23≠0\langle 1, x^2\rangle = \tfrac{2}{3} \ne 0, so the separate projections overlap. Use 1,x,x2−131, x, x^2 - \tfrac{1}{3} instead.

Frequently asked questions

What is an inner product?

A rule ⟨u, v⟩ giving a number for each pair of vectors, which is symmetric, linear in each vector, and has ⟨v, v⟩ > 0 for every nonzero v.

What is the standard inner product for functions?

⟨f, g⟩ = ∫ₐᵇ f(x)g(x) dx over an interval. The length of f is ‖f‖ = √⟨f, f⟩.

When are two functions orthogonal?

When ⟨f, g⟩ = 0. On [−1, 1], 1 and x are orthogonal, and so is any even function with any odd function.

How do you find a best approximation?

Project f onto the subspace using an orthogonal basis: the coefficient of each basis vector uᵢ is ⟨f, uᵢ⟩/⟨uᵢ, uᵢ⟩. The result minimizes ‖f − g‖ over the subspace.

What are the Legendre polynomials?

The orthogonal polynomials from Gram–Schmidt applied to 1, x, x², … on [−1, 1]. Up to scaling they begin 1, x, x² − 1/3.

What to learn next