Normal Equations and Projection Matrices

Solution Preview

1. Consider the following subspaces of R^4
V=span{v1,v2,v3}, W=span{w1,w2,w3}
where v1=(1,2,1,-2)^T w1=(1,1,1,1)^T
v2=(2,3,1,0)^T w2=(1,0,1,-1)^T
v3=(1,2,2,-3)^T w3=(1,3,0,-4)^T

a)Find two systems of homogeneous linear equations whose solution spaces are V and W, respectively.
b)Find a basis for V∩W by joining the systems obtained in (a) in one system and solving it. Explain why the solution of the latter is exactly V∩W.
c) Find dim V and dim W and calculate the dimension of V+W using the dimension formula

2. Solving the normal equation, find the least squares solution x0 of the system of linear equations

x+y+2z=2
x+2y+3z=-3
x+2y+z=4
x+y+4z=1

should use Maple or Matlab for calculations

3. Let W=span {u,v,w} where
u=(1,1,1,1)^T
v=(1,1,1,0)^T
w=(0,1,1,1)^T and b=(2,1,0,1)^T

a) Find the projection w=projw(b) of b onto W
b) Find the projection matrix P onto W and check that w=Pb
c) Find w'W such that b=w+w'

4. Prove that Null (A^TA)=Null(A)

Solutions:

1. Given:
(1)

a) Any vector which belongs to V can be expressed as linear combination:

(2)

By solving the algebraic system in ,  and  enclosing the first 3 equations, we will get the values of these unknowns as function of x, y and z. Replacing in the last equation of (2), a linear relation in x, y, z and t will be found; this is the homogeneous equation we are looking for.
(3)
The system is consistent and it has the unique solution:
...