# Learntofish's Blog

## Understanding the Fourier Transform intuitively

Posted by Ed on August 28, 2009

In the following I will explain how the Fourier transform can be understood intuitively. We will start with basis vectors in $\mathbb{R}^3$ and make the transition to basis functions.

The Fourier Transform
For a function $f(x)$ the Fourier transform is given by $f(x)=\frac{1}{\sqrt{2\pi}} \int_{-\infty}^{+\infty} c(k) e^{ikx} dk$ with $c(k) = \frac{1}{\sqrt{2 \pi}} \int_{-\infty}^{+\infty} f(t) e^{-ikt} dt$ (see here and here). In order to understand this formula we consider vectors in $\mathbb{R}^3$:

Every vector can be represented as a linear combination of the basis vectors (1,0,0), (0,1,0) and (0,0,1). Take the vector v=(9,8,4). This vector can be written as a linear combination of the basis vectors $u_1 = (1,0,0), u_2=(0,1,0), u_3=(0,0,1)$ in the following way:
$(9,8,4) = 9 \cdot (1,0,0) + 8 \cdot (0,1,0) + 4 \cdot (0,0,1)$
$(9,8,4) = 9 \cdot u_1 + 8 \cdot u_2 + 4 \cdot u_3$
Let’s call the numbers in front of the basis vectors coefficients:
$(9,8,4) = c_1 \cdot u_1 + c_2 \cdot u_2 + c_3 \cdot u_3$
where $c_1=9$, $c_2=8$ and $c_3=4$ are the coefficients.

What if we choose different basis vectors, for example:
$b_1=(3,0,0), b_2=(0,2,0), b_3=(0,0,2)$

Then our vector v=(9,8,4) can be written as:
$(9,8,4)=3 \cdot b_1 + 4 \cdot b_2 + 2 \cdot b_3$
Thus, for our new basis vectors $b_1, b_2$ and $b_3$ the coefficients are 3, 4 and 2.

In general, you can write a vector v as a linear combination of
basis vectors $b_k$ , where in front of the basis vectors you have the coefficients $c_k$:
$v = \sum_{k=1}^{3} c_k b_k$

In our last example we had $b_1=(3,0,0), b_2=(0,2,0), b_3=(0,0,2)$ together with the coefficients $c_1=3, c_2=4, c_3=2$. Just check the formula $v = \sum_{k=1}^{3} c_k b_k$ by plugging in the above values:
$v = c_1 \cdot b_1 + c_2 \cdot b_2 + c_3 \cdot b_3 =3 \cdot (3,0,0) + 4 \cdot (0,2,0) + 2 \cdot (0,0,2) = (9,8,4)$

———————–

So, every vector can be represented by some basis vectors $b_k$ as $v = \sum_{k=1}^{3} c_k b_k$.
More general, if we have a vector with n entries instead of 3 we write $v = \sum_{k=1}^{n} c_k b_k = c_1 b_1 + c_2 b_2 + ... + c_n b_n$.
Now, let’s make a step from the discrete to the continuous case.
Say we have a function f(x) and we also want to ask whether it’s possible to represent f(x) as a linear combination of “basis vectors”.

Question: Is it possible to write $f(x) = \sum_{k=1}^n c_k b_k$
Let us be more specific and ask: Can we write f(x) as a sum
of $b_k = e^{ikx}$? So, the new basis vectors are $b_k=e^{ikx}$.

The question then becomes:

Question: Is it possible to write $f(x) = \sum_{k=1}^n c_k e^{ikx}$
Indeed, it is possible with a correction for the values of k.
Instead of going from k=1 to n we use infinitely many basis vectors and write
$k=- \infty$ to $k=+ \infty$.
Corrected version:
$f(x) = \sum_{k=-\infty}^{+\infty} c_k e^{ikx}$

To finally get to the Fourier integral we replace the sum by an integral:
$f(x) = \frac{1}{\sqrt{2 \pi}} \int_{-\infty}^{+\infty} c(k) e^{ikx} dk$
(The factor $\frac{1}{\sqrt{2 \pi}}$ is inserted for normalization issues)

One question remains though: How do the coefficients c(k) look like?
We can calculate c(k) by $c(k) = \frac{1}{\sqrt{2 \pi}} \int_{-\infty}^{+\infty} f(t) e^{-ikt} dt$

One can prove that c(k) has this form by plugging c(k) into the formula of f(x) and evaluating the double integral (see here).

Some remarks
1) Summary
In summary, consider the Fourier integral $f(x) = \frac{1}{\sqrt{2 \pi}} \int_{-\infty}^{+\infty} c(k) e^{ikx} dk$ as a linear combination of basis vectors $e^{ikx}$ with the coefficients $c(k)$.

2) Functions as basis vectors
You might ask “Why can I consider the functions $e^{ikx}$ as basis vectors?”
The functions $e^{ikx}$ fulfill some properties similar to those of
basis vectors $u_1=(1,0,0), u_2=(0,1,0), u_3=(0,0,1)$ in $\mathbb{R}^3$.
$u_1, u_2$ and $u_3$ are orthonormal to each other:
(i) orthogonal:
$\langle u_1 , u_2 \rangle = 0$
$\langle u_1 , u_3 \rangle = 0$
$\langle u_2 , u_3 \rangle = 0$
with $\langle a, b \rangle$ being the inner product of two vectors a and b.
(ii) normal:
$\langle u_1 , u_1 \rangle = 1$
$\langle u_2 , u_2 \rangle = 1$
$\langle u_3 , u_3 \rangle = 1$
In short: $\langle u_k, u_l \rangle = \delta_{kl}$ where $\delta_{kl}$ is the Kronecker delta.

For functions f(x) and g(x) we can define an inner product (L2 inner product) as:
$\langle f(x), g(x) \rangle = \int_{-\infty}^{+\infty} \overline{f(x)} g(x) dx$.
One can show that for two basis vectors $b_{k}= e^{ikx} / \sqrt{2 \pi}$ and $b_{k'}= e^{ik'x} / \sqrt{2 \pi}$ the inner product is:
$\langle b_k, b_{k'}\rangle= \frac{1}{2 \pi}\langle e^{ikx},e^{ik'x} \rangle = \frac{1}{2 \pi} \int_{-\infty}^{+\infty} \overline{e^{ikx}} e^{ik'x} dx = \frac{1}{2 \pi} \int_{-\infty}^{+\infty} e^{ix(k'-k)} dx = \delta(k'-k)$
(see here). This looks like the Kronecker delta from above.
In particular $b_k$ and $b_{k'}$ are orthogonal to each other since $\langle b_k, b_{k'}\rangle = \delta(k'-k) = 0$ for $k \neq k'$.
The difference to the Kronecker delta is that the vectors $b_{k}= e^{ikx} / \sqrt{2 \pi}$ can not be normalized to 1 since $\langle b_k, b_{k}\rangle = \delta(k-k) = \infty$.

3) Interpreting c(k) as projection
In $\mathbb{R}^3$ we represented the vector v=(9,8,4) as a linear combination of the basis vectors $u_1=(1,0,0), u_2=(0,1,0), u_3=(0,0,1)$:
$v=\sum_{k=1}^{3} c_k u_k = c_1 u_1 + c_2 u_2 + c_3 u_3 = 9 \cdot u_1 + 8 \cdot u_2 + 4 \cdot u_3$
You can immediately “see” that the coefficients are $c_1=9, c_2=8, c_3=4$. But you can also get the coefficients by projecting v onto the basis vectors (Note that we have to project on normalized basis vectors):

And as you may know, the projections can be calculated with the dot product:
$c_1 = \langle u_1, v \rangle = 9$
$c_2 = \langle u_2, v \rangle = 8$
$c_3 = \langle u_3, v \rangle = 4$
So, we can write $v= \sum_{k=1}^{3} c_k u_k = \sum_{k=1}^{3} \langle u_k, v \rangle u_k$

Now, for a function f(x) the Fourier transform is given by $f(x) = \frac{1}{\sqrt{2 \pi}} \int_{-\infty}^{+\infty} c(k) e^{ikx} dk$
Similar to above we will interpret the coefficients c(k) as the projection of f(x) onto the basis vectors $b_k = e^{ikx} / \sqrt{ 2 \pi }$:

But what is the result of the inner product $c(k) = \langle b_k, f(x) \rangle = \langle e^{ikx} / \sqrt{ 2 \pi }, f(x) \rangle$?
Recall from above that for two functions f(x) and g(x) we defined the inner product as L2 inner product $\langle f(x), g(x) \rangle = \int_{-\infty}^{+\infty} \overline{f(x)} g(x) dx$.
Let’s use this to calculate c(k):
$c(k) = \langle b_k, f(x) \rangle = \langle e^{ikx} / \sqrt{ 2 \pi }, f(x) \rangle = \frac{1}{\sqrt{ 2 \pi }} \langle e^{ikx} , f(x) \rangle$
$= \frac{1}{\sqrt{ 2 \pi }} \int_{-\infty}^{+\infty} \overline{e^{ikx}} f(x) dx = \frac{1}{\sqrt{ 2 \pi }} \int_{-\infty}^{+\infty} e^{-ikx} f(x) dx = \frac{1}{\sqrt{ 2 \pi }} \int_{-\infty}^{+\infty} f(t) e^{-ikt} dt$
(In the last step I just changed the integration variable from x to t)

4) Some examples of Fourier transform and coefficients
On this website the Fourier transform is defined slightly different (compare the factor in front of the integral sign):
$f(x)=\frac{1}{2\pi} \int_{-\infty}^{+\infty} \hat{f}(k) e^{ikx} dk$
with $\hat{f}(k)$ being the coefficient that can be calculated by
$\hat{f}(k) = \int_{-\infty}^{+\infty} f(x) e^{-ikx} dx$
Examples for this Fourier transform are given here.

5) Applications for the Fourier transform
Have a look at What is a Fourier Transform and what is it used for? and Applications of Fourier Transform in Communications, Astronomy, Geology and Optics.

6) Java Applets
Approximation of a function by a Fourier transform
The applet (click on Start Function FFT) shows you how you can approximate a function by a Fourier transform (though they talk about the related Fast Fourier Transform). The more coefficients you use the better the approximation becomes.

Applet: Rectangular pulse approximation by Fourier Transform
This applet shows how to approximate a rectangular shaped pulse. If you don’t use enough basis vectors (bandwidth is limited) then the pulse will not look rectangular anymore. This is important in electronics where you want to transmit a signal but the bandwidth is limited.

References
[1] U. H. Gerlach, Linear mathematics in infinite dimensions Signals, Boundary Value Problems and Special Functions, Febuary 2007, Beta Edition, http://www.math.ohio-state.edu/~gerlach/math/BVtypset/BVtypset.html, “The Fourier Integral Theorem”, http://www.math.ohio-state.edu/~gerlach/math/BVtypset/node31.html

[2] Weisstein, Eric W. Fourier Transform, August 2009. From MathWorld–A Wolfram Web Resource. http://mathworld.wolfram.com/FourierTransform.html

[3] Jochen Rau, Elementary Mathematical Methods for Physics, Lectures at Dresden University of Technology, winter 1996/97, http://www.mpipks-dresden.mpg.de/~jochen/methoden/intro.html, “Definition” and “Examples” in Lecture 8, http://www.mpipks-dresden.mpg.de/~jochen/methoden/topics/ft_def.html, http://www.mpipks-dresden.mpg.de/~jochen/methoden/topics/ft_ex.html

1. ### Olumidesaid

Thanks.

If it is true that the Fourier transform

$f(\omega) = \int_{-\infty}^{+\infty} f(x) exp{-j 2 \pi \omega x } dx$

computes the coordinates of a function $f(x)$ in an infinite dimensional space spanned by the orthogonal basis vectors $exp{-j 2 \pi \omega x }$. Why is it that $f(x)$ is represented as a combination of the basis vectors $exp{j 2 \pi \omega x }$ i.e.

$f(x) = \int_{-\infty}^{+\infty} f(w) exp{j 2 \pi \omega x } dx$

instead of the original set of basis vectors $exp{-j 2 \pi \omega x }$ ?

2. ### Edsaid

Thank you for your comment. I suppose you mean $f(x)$ and $\hat{f}(\omega)$. They are different functions, so don’t forget the little hat over $\hat{f}(\omega)$.

The minus sign comes from the definition of the L2 inner product: In my post above I tried to derive the coefficients $c(k)$ or $\hat{f}(k)$. I did this by considering a projection, and for projections you use the L2 inner product or scalar product. The L2 inner product involves complex conjugation of $e^{ikx}$ which is $e^{-ikx}$.

I hope this helps.

Ed