Learntofish's Blog

A blog about math, physics and computer science

Understanding the Fourier Transform intuitively

Posted by Ed on August 28, 2009

In the following I will explain how the Fourier transform can be understood intuitively. We will start with basis vectors in \mathbb{R}^3 and make the transition to basis functions.

The Fourier Transform
For a function f(x) the Fourier transform is given by f(x)=\frac{1}{\sqrt{2\pi}} \int_{-\infty}^{+\infty} c(k) e^{ikx} dk with c(k) = \frac{1}{\sqrt{2 \pi}} \int_{-\infty}^{+\infty} f(t) e^{-ikt} dt (see here and here). In order to understand this formula we consider vectors in \mathbb{R}^3:

Every vector can be represented as a linear combination of the basis vectors (1,0,0), (0,1,0) and (0,0,1). Take the vector v=(9,8,4). This vector can be written as a linear combination of the basis vectors u_1 = (1,0,0), u_2=(0,1,0), u_3=(0,0,1) in the following way:
(9,8,4) = 9 \cdot (1,0,0) + 8 \cdot (0,1,0) + 4 \cdot (0,0,1)
(9,8,4) = 9 \cdot u_1 + 8 \cdot u_2 + 4 \cdot u_3
Let’s call the numbers in front of the basis vectors coefficients:
(9,8,4) = c_1 \cdot u_1 + c_2 \cdot u_2 + c_3 \cdot u_3
where c_1=9, c_2=8 and c_3=4 are the coefficients.

What if we choose different basis vectors, for example:
b_1=(3,0,0), b_2=(0,2,0), b_3=(0,0,2)

Then our vector v=(9,8,4) can be written as:
(9,8,4)=3 \cdot b_1 + 4 \cdot b_2 + 2 \cdot b_3
Thus, for our new basis vectors b_1, b_2 and b_3 the coefficients are 3, 4 and 2.

In general, you can write a vector v as a linear combination of
basis vectors b_k , where in front of the basis vectors you have the coefficients c_k:
v = \sum_{k=1}^{3} c_k b_k

In our last example we had b_1=(3,0,0), b_2=(0,2,0), b_3=(0,0,2) together with the coefficients c_1=3, c_2=4, c_3=2. Just check the formula v = \sum_{k=1}^{3} c_k b_k by plugging in the above values:
v = c_1 \cdot b_1 + c_2 \cdot b_2 + c_3 \cdot b_3 =3 \cdot (3,0,0) + 4 \cdot (0,2,0) + 2 \cdot (0,0,2) = (9,8,4)

———————–

So, every vector can be represented by some basis vectors b_k as v = \sum_{k=1}^{3} c_k b_k.
More general, if we have a vector with n entries instead of 3 we write v = \sum_{k=1}^{n} c_k b_k = c_1 b_1 + c_2 b_2 + ... + c_n b_n.
Now, let’s make a step from the discrete to the continuous case.
Say we have a function f(x) and we also want to ask whether it’s possible to represent f(x) as a linear combination of “basis vectors”.

Question: Is it possible to write f(x) = \sum_{k=1}^n c_k b_k
Let us be more specific and ask: Can we write f(x) as a sum
of b_k = e^{ikx}? So, the new basis vectors are b_k=e^{ikx}.

The question then becomes:

Question: Is it possible to write f(x) = \sum_{k=1}^n c_k e^{ikx}
Indeed, it is possible with a correction for the values of k.
Instead of going from k=1 to n we use infinitely many basis vectors and write
k=- \infty to k=+ \infty.
Corrected version:
f(x) = \sum_{k=-\infty}^{+\infty} c_k e^{ikx}

To finally get to the Fourier integral we replace the sum by an integral:
f(x) = \frac{1}{\sqrt{2 \pi}} \int_{-\infty}^{+\infty} c(k) e^{ikx} dk
(The factor \frac{1}{\sqrt{2 \pi}} is inserted for normalization issues)

One question remains though: How do the coefficients c(k) look like?
We can calculate c(k) by c(k) = \frac{1}{\sqrt{2 \pi}} \int_{-\infty}^{+\infty} f(t) e^{-ikt} dt

One can prove that c(k) has this form by plugging c(k) into the formula of f(x) and evaluating the double integral (see here).

 

Some remarks
1) Summary
In summary, consider the Fourier integral f(x) = \frac{1}{\sqrt{2 \pi}} \int_{-\infty}^{+\infty} c(k) e^{ikx} dk as a linear combination of basis vectors e^{ikx} with the coefficients c(k).

2) Functions as basis vectors
You might ask “Why can I consider the functions e^{ikx} as basis vectors?”
The functions e^{ikx} fulfill some properties similar to those of
basis vectors u_1=(1,0,0), u_2=(0,1,0), u_3=(0,0,1) in \mathbb{R}^3.
u_1, u_2 and u_3 are orthonormal to each other:
(i) orthogonal:
\langle u_1 , u_2 \rangle = 0
\langle u_1 , u_3 \rangle = 0
\langle u_2 , u_3 \rangle = 0
with \langle a, b \rangle being the inner product of two vectors a and b.
(ii) normal:
\langle u_1 , u_1 \rangle = 1
\langle u_2 , u_2 \rangle = 1
\langle u_3 , u_3 \rangle = 1
In short: \langle u_k, u_l \rangle = \delta_{kl} where \delta_{kl} is the Kronecker delta.

For functions f(x) and g(x) we can define an inner product (L2 inner product) as:
\langle f(x), g(x) \rangle = \int_{-\infty}^{+\infty} \overline{f(x)} g(x) dx.
One can show that for two basis vectors b_{k}= e^{ikx} / \sqrt{2 \pi} and b_{k'}= e^{ik'x} / \sqrt{2 \pi} the inner product is:
\langle b_k, b_{k'}\rangle= \frac{1}{2 \pi}\langle e^{ikx},e^{ik'x} \rangle = \frac{1}{2 \pi} \int_{-\infty}^{+\infty} \overline{e^{ikx}} e^{ik'x} dx = \frac{1}{2 \pi} \int_{-\infty}^{+\infty} e^{ix(k'-k)} dx = \delta(k'-k)
(see here). This looks like the Kronecker delta from above.
In particular b_k and b_{k'} are orthogonal to each other since \langle b_k, b_{k'}\rangle = \delta(k'-k) = 0 for k \neq k'.
The difference to the Kronecker delta is that the vectors b_{k}= e^{ikx} / \sqrt{2 \pi} can not be normalized to 1 since \langle b_k, b_{k}\rangle = \delta(k-k) = \infty.

3) Interpreting c(k) as projection
In \mathbb{R}^3 we represented the vector v=(9,8,4) as a linear combination of the basis vectors u_1=(1,0,0), u_2=(0,1,0), u_3=(0,0,1):
v=\sum_{k=1}^{3} c_k u_k = c_1 u_1 + c_2 u_2 + c_3 u_3 = 9 \cdot u_1 + 8 \cdot u_2 + 4 \cdot u_3
You can immediately “see” that the coefficients are c_1=9, c_2=8, c_3=4. But you can also get the coefficients by projecting v onto the basis vectors (Note that we have to project on normalized basis vectors):

projection

And as you may know, the projections can be calculated with the dot product:
c_1 = \langle u_1, v \rangle = 9
c_2 = \langle u_2, v \rangle = 8
c_3 = \langle u_3, v \rangle = 4
So, we can write v= \sum_{k=1}^{3} c_k u_k = \sum_{k=1}^{3} \langle u_k, v \rangle u_k

Now, for a function f(x) the Fourier transform is given by f(x) = \frac{1}{\sqrt{2 \pi}} \int_{-\infty}^{+\infty} c(k) e^{ikx} dk
Similar to above we will interpret the coefficients c(k) as the projection of f(x) onto the basis vectors b_k = e^{ikx} / \sqrt{ 2 \pi }:

projection2

But what is the result of the inner product c(k) = \langle b_k, f(x) \rangle = \langle e^{ikx} / \sqrt{ 2 \pi }, f(x) \rangle?
Recall from above that for two functions f(x) and g(x) we defined the inner product as L2 inner product \langle f(x), g(x) \rangle = \int_{-\infty}^{+\infty} \overline{f(x)} g(x) dx.
Let’s use this to calculate c(k):
c(k) = \langle b_k, f(x) \rangle = \langle e^{ikx} / \sqrt{ 2 \pi }, f(x) \rangle = \frac{1}{\sqrt{ 2 \pi }} \langle e^{ikx} , f(x) \rangle
= \frac{1}{\sqrt{ 2 \pi }} \int_{-\infty}^{+\infty} \overline{e^{ikx}} f(x) dx = \frac{1}{\sqrt{ 2 \pi }} \int_{-\infty}^{+\infty} e^{-ikx} f(x) dx = \frac{1}{\sqrt{ 2 \pi }} \int_{-\infty}^{+\infty} f(t) e^{-ikt} dt
(In the last step I just changed the integration variable from x to t)

 

4) Some examples of Fourier transform and coefficients
On this website the Fourier transform is defined slightly different (compare the factor in front of the integral sign):
f(x)=\frac{1}{2\pi} \int_{-\infty}^{+\infty} \hat{f}(k) e^{ikx} dk
with \hat{f}(k) being the coefficient that can be calculated by
\hat{f}(k) = \int_{-\infty}^{+\infty} f(x) e^{-ikx} dx
Examples for this Fourier transform are given here.

5) Applications for the Fourier transform
Have a look at What is a Fourier Transform and what is it used for? and Applications of Fourier Transform in Communications, Astronomy, Geology and Optics.

6) Java Applets
Approximation of a function by a Fourier transform
The applet (click on Start Function FFT) shows you how you can approximate a function by a Fourier transform (though they talk about the related Fast Fourier Transform). The more coefficients you use the better the approximation becomes.

Applet: Rectangular pulse approximation by Fourier Transform
This applet shows how to approximate a rectangular shaped pulse. If you don’t use enough basis vectors (bandwidth is limited) then the pulse will not look rectangular anymore. This is important in electronics where you want to transmit a signal but the bandwidth is limited.

 

References
[1] U. H. Gerlach, Linear mathematics in infinite dimensions Signals, Boundary Value Problems and Special Functions, Febuary 2007, Beta Edition, http://www.math.ohio-state.edu/~gerlach/math/BVtypset/BVtypset.html, “The Fourier Integral Theorem”, http://www.math.ohio-state.edu/~gerlach/math/BVtypset/node31.html

[2] Weisstein, Eric W. Fourier Transform, August 2009. From MathWorld–A Wolfram Web Resource. http://mathworld.wolfram.com/FourierTransform.html

[3] Jochen Rau, Elementary Mathematical Methods for Physics, Lectures at Dresden University of Technology, winter 1996/97, http://www.mpipks-dresden.mpg.de/~jochen/methoden/intro.html, “Definition” and “Examples” in Lecture 8, http://www.mpipks-dresden.mpg.de/~jochen/methoden/topics/ft_def.html, http://www.mpipks-dresden.mpg.de/~jochen/methoden/topics/ft_ex.html

Advertisements

2 Responses to “Understanding the Fourier Transform intuitively”

  1. Olumide said

    Thanks.

    If it is true that the Fourier transform

    f(\omega) = \int_{-\infty}^{+\infty} f(x) exp{-j 2 \pi \omega x } dx

    computes the coordinates of a function f(x) in an infinite dimensional space spanned by the orthogonal basis vectors exp{-j 2 \pi \omega x }. Why is it that f(x) is represented as a combination of the basis vectors exp{j 2 \pi \omega x } i.e.

    f(x) = \int_{-\infty}^{+\infty} f(w) exp{j 2 \pi \omega x } dx

    instead of the original set of basis vectors exp{-j 2 \pi \omega x } ?

  2. Ed said

    Thank you for your comment. I suppose you mean f(x) and \hat{f}(\omega). They are different functions, so don’t forget the little hat over \hat{f}(\omega).

    The minus sign comes from the definition of the L2 inner product: In my post above I tried to derive the coefficients c(k) or \hat{f}(k). I did this by considering a projection, and for projections you use the L2 inner product or scalar product. The L2 inner product involves complex conjugation of e^{ikx} which is e^{-ikx}.

    I hope this helps.

    Ed

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: