Least Square Approximations for Mathematicians

A friend of mine, a math PhD, was seeing a lady who was working on her BSc. in CS. She was in a “numerical methods” class, and she needed to understand least squares. Naturally, she turned to my friend and asked to be tutored. Now, there was no reason my friend would ever need to know least squares, and indeed he did not. He asked me what least squares was. My explanation started with “Let’s take the Hilbert space of…”. I assumed he’d work out the equations, and explain those to her. He took an, ahem, “alternate” route of starting by explaining Hilbert spaces. That — that worked out less well than could be expected.

In the interest of completeness, here is the explanation. Let {x_1,..,x_n} be numbers, and let H:{x_i}->R be the vector space of real-valued functions. There is a natural inner product structure on H: <f,g>=\sum f(x_i)g(x_i). Let id be the function x_i->x_i, and 1 be the function x_i->1. Let H_0 be the space spanned by id and 1. Let L:H->H_0 be the orthogonal projection. L(f) is therefore an affine function, and it is the “least squares approximation” of f.

Exercise: find a formula for the projection in terms of f(x_i).


One Response to Least Square Approximations for Mathematicians

  1. drklaymen says:

    Wow, that’s an old story!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: