After this post, I decided to organize my understanding of convolution in this series.
Probability Theory
We shall define some concepts and theorems:
A function is said to be a measurable map from the measure space
to
if
for all . If
with
, then
is called a random vector. If
, then
is called a random variable. Here we shall believe that properties of integration on measurable function from the measure theory are fully justified, and we define the following:
If is a random variable on
, then we define its expected value to be
It is not hard to show that if is measurable, then
is also a random variable. From this point, we need to know how to compute the expected value with change of variables.
Theorem 1:
Let be a random vector of
with distribution
, i.e.,
. If
is a measurable function from
to
with
or
, then
This thoerem can be proved by considering the integral in four stages.1Indicator functions → Simple functions → Nonnegative functions → Integrable functions Using the similar technique, we can get the following corollary.
Corollary 1:
Suppose that the probability measure has
for all . For any
with
or
, we have
We will use this corollary later. For now, since we will be working with two or more random variables, one version of the well-known Fubini’s theorem will be stated here.
Fubini’s Theorem:
Suppose that is the product of two measure spaces
and
. If
or
, then
we consider the theorem 1 together with the Fubini’s Theorem and state the following theorem.
Theorem 2:
Suppose and
are independent and have distributions
and
. If
is a measurable function with
or
, then
Now we can investigate the concept of convolution.
Theorem 3:
If and
are independent,
and
, then
Remark that the notation is exactly
where
is the probability measure with distribution function
. That is, it means “integrate with respect to the measure
with distribution function
. Here, the integral
is called the convolution of and
, denoted by
. To make it more applicable, the following corollary will be introduced.
Corollary 2:
Suppose that with density
and
with distribution function
are independent. Then
has density
When has density
, the last formulat can be written as
The first part of this corollary can be proved by Theorem 3 and Fubini’s theorem, and the second part can be proved with a help of corollary 2.
At this point, the last equation looks much more familiar. Yes, without the context, it is exactly the convolution in Fourier Analysis. To see how this concept can be approached from different aspects, check out the other two volumes of this series (Still working).
Most of the content above is based on Probability: Theory and Examples by Rick Durrett, which is a comprehensive textbook for graduate probability theory. I also learned a lot from the class MAT235A taught by Professor Janko Gravner. Thanks for his kindly teaching!
Leave a Reply