A friend of mine, a math PhD, was seeing a lady who was working on her BSc. in CS. She was in a “numerical methods” class, and she needed to understand least squares. Naturally, she turned to my friend and asked to be tutored. Now, there was no reason my friend would ever need to know least squares, and indeed he did not. He asked me what least squares was. My explanation started with “Let’s take the Hilbert space of…”. I assumed he’d work out the equations, and explain those to her. He took an, ahem, “alternate” route of starting by explaining Hilbert spaces. That — that worked out less well than could be expected.
In the interest of completeness, here is the explanation. Let {x_1,..,x_n} be numbers, and let H:{x_i}->R be the vector space of real-valued functions. There is a natural inner product structure on H: <f,g>=\sum f(x_i)g(x_i). Let id be the function x_i->x_i, and 1 be the function x_i->1. Let H_0 be the space spanned by id and 1. Let L:H->H_0 be the orthogonal projection. L(f) is therefore an affine function, and it is the “least squares approximation” of f.
Exercise: find a formula for the projection in terms of f(x_i).
October 20, 2009 at 10:24 pm |
Wow, that’s an old story!