Of all the random choices you made in your entire life, none of them can prevent you from meeting this article.

Anonymous

Note: The goal of this article is to bring mathematics to the general public. For math students and researchers, the book An Introduction To Stochastic Differential Equations by Lawrence C.Evans is a decent start, given that you have some understanding of probability theory from a measure theory standpoint. Math popularization is a thing that I always wanted to do but failed for many excuses. Thanks for SoME1 to give me a reason to start and hope you enjoy this article.

Motivation: What Does It Mean to Be Stochastic?

Life is full of randomness, and no one knows what will happen in the next second. Human literature has always been imagining the ability to predict the future, as it doesn’t seem to be plausible in real life. Therefore, many characters and figures were made, and stories were told. But in mathematics, what does it mean to be random? As a subject of describing our world, how to measure such randomness?

One can argue that the definition of randomness is ambiguous. If you flip a coin, one may say that no one knows it is a head or a tail. In mathematics, this phenomenon can be described as probabilities with certain formulation. However, life is not that easy. For instance, what about a stock market?

The word “stochastic” is simply a fancier way of saying randomness. Mathematicians sometimes do this, well, like using the word “orthogonal” rather than the word “perpendicular”. This article aims to give a brief introduction to Stochastic Differential Equations without using too much mathematical language. I assumed that the reader has some basic knowledge of probability theory (up to probability distributions) and differential equations. If not, the previous hyperlinks to 3Blue1Brown’s Youtube videos should be sufficient. If you don’t even want to spend time watching other things, it is okay because you can still understand 95% of this article.

From Random Variables to Stochastic Processes

Let’s say that you are drunk after a party, and you are trying to find a way home. To make our illustration simpler, let’s assume that you are drunk heavily, and every move you make is random. That is, for every direction in one move, there is a probability value assigned to one direction that you may move forward, and the entire process consists of infinitely many moves. This is one kind of stochastic process, and more specifically, it is called a Random Walk. As a famous saying goes,

A drunk man will find his way home, but a drunk bird may get lost forever.

Shizuo Kakutani

It means that a two-dimensional random walker, as we assume the drunk man won’t fall off a cliff and won’t climb up to a tree, will get back to the original starting point of the walk almost surely; but the probability of a three-dimensional random walker get back to the original starting point decreases to 34%, as a drunk bird is flying on a three-dimensional space.

5 Two-Dimensional Random Walks Generated By Matlab (1 unit per step for 400 steps)

In mathematics, the action of assigning probability values to different directions is formulated as a function. The input of this function is the result of the action, like moving in which direction, and the output of the function is an assigned probability. We have a new name for these functions, called Random Variables. A Stochastic Process is then defined as a sequence of these kinds of functions or random variables. Another example will be tossing a fair coin multiple times. At every time step, a random variable assigns both heads and tails with probabilities of 1/2. The sequence of multiple tosses, or the sequence of these multiple identical random variables, consists of a stochastic process. Just for reference, this stochastic process is called a Bernoulli Process.

It is easy to see that the above examples are discrete. In other words, they can be decomposed into finite steps, and one can only consider a certain value. For other examples, like time, is not discrete but continuous. For the random walks on-grid above, if the walkers wait for a random time between steps, it is now a Continuous-Time Stochastic Process.

Adding Randomness to Functions

Imagine that we have a function like the following:

A Smooth Function

This function is called the Sampling Function and is frequently used in signal processing. We can see that it is smooth and clean, but the functions, in reality, is not always this smooth and clean. Instead, the orange line below is more like this function in real life.

A Smooth Function With Its Oscillating Brother

If we define the original function, the blue line, as f(t) varying in time, we can define the oscillating function to be f(t)+\zeta(t). The additional component \zeta(t) represents the ups and downs of the original function. To visualize this concept, we named it “Noise“. In the example above, we define the noise function to follow a specific distribution, and in this case, this distribution is the Gaussian Distribution.

From Differential Equations to Stochastic Differential Equations

Roughly speaking, the idea of Stochastic Differential Equations is the combination of random processes, noise, and differential equations. Let’s begin to consider the simple population growth model

    \[\frac{dN}{dt}=a(t)N(t)\]

where N(t) is the size of the population at time t, and a(t) is the relative rate
of growth at time t. This is a differential equation, and it is easy to find a solution to this differential equation as long as we know the value of a(t). But the randomness strikes again, what if a(t) is not completely known? We may model a(t) by adding some other random values, namely, the noise.

    \[a(t)=r(t)+\zeta(t)\]

In this case, the solution of the differential equation also inherits the randomness behavior. In general, these kinds of differential equations are called stochastic differential equations. More formally, A Stochastic Differential Equation describes a system that is originally described by a differential equation, but disturbed by random noise. A stochastic differential equation involves random processes, and its resulting solution is also a random process.

Of course, mathematics is not that simple. For the sake of rigor and generality, language needs to be more concise and abstract, and many questions need to be clarified. For instance, how to define the “noise”? What does it mean for a stochastic process to solve a stochastic differential equation? Do the stochastic differential equations have a solution? Many mathematicians have paved the way for us, and thankfully we don’t have to worry about them.

Reference

Leave a Reply