Lesson_10 – Statistics Course

Homework_14RA

SDE and methods for numerical solution.

A solution to an SDE is itself a stochastic function, which means that its value X(t) at any given time t is a random variable.

The Euler-Maruyama (EM) and Milstein methods

These methods are based on the truncated Ito-Taylor expansion.
Unfortunately, in many cases analytic solutions are not available for these equations, so we are required to use numerical methods to approximate the solution.

If we add a random element or elements to the deterministic differential equation, we have transition from an ordinary differential equation to SDE.

Some analytical and numerical solutions considered numerical approximations of random periodic solutions for SDEs. On the other hand, constructed a Milstein scheme by adding an error correction term for solving stiff SDEs.

Remember SDE form:
dX(t,w) = μ(t,X(t,w))dt + σ(t,X(t,w))dW(t,w) and X(t₀,w) = X₀
where μ is the drift coefficient
where σ is the the diffusion coefficient

The Monte Carlo simulation:

Monte Carlo methods are numerical methods, where random numbers are used to conduct a computational experiment. Numerical solution of stochastic differential equations can be viewed as a type of Monte Carlo calculation.

In Monte Carlo simulation, the entire system is simulated a large number of times.
So, a set of suitable sample paths is produced on [t₀,T].
Each simulation is equally likely, referred to as a realization of the system. For each realization, all of the uncertain parameters are sampled.

Stochastic Taylor series expansion for produce a sample path solution to the SDE on [t₀,T].

The Taylor formula plays a very significant role in numerical analysis. We can obtain the approximation of a sufficiently smooth function in a neighborhood of a given point to any desired order of accuracy with the Taylor formula.

Ito-Taylor expansion obtained via Ito’s formula

First we can obtain an Ito-Taylor expansion for the stochastic case.

dX(t) = μ(X(t))dt + σ(X(t))dW(t)
where μ and σ satisfy a linear growth bound and are sufficiently smooth.

Once we have the Ito-Taylor expansion, we can construct numerical integration schemes for the proces.

Ito-Taylor expansion was based upon the use of multiple stochastic integrals. Itô–Taylor expansions are characterized by the choice of the multiple integrals which appear in them.

Euler-Maruyama method

The Milstein method is another technique for the approximate numerical solution of a stochastic differential equation.

If we truncate Ito’s formula of the stochastic Taylor series after the first order terms, we obtain the Euler method or Euler-Maruyama method.

Milstein method

If we truncate the stochastic Taylor series after second order terms, we obtain the Milstein method.

There are many other approximation schemes for SDEs, ie: Runge-Kutta method(which achieves the same convergence properties as the Milstein method, but without the need to compute derivatives of the σ(⋅) function.

What has been observed
(https://www.researchgate.net/publication/313191865_On_the_convergence_of_the_Euler-Maruyama_method_and_Milstein_scheme_for_the_solution_of_
stochastic_differential_equations)

Milstein method is more accurate than its counterpart, Euler Maruyama method. For Milstein method, the errors decrease as N increases for all N values.
Overall, Milstein method performed consistently better than Euler Maruyama method with respect to accuracy.
Euler Maruyama method is the simplest numerical method for solving stochastic differential equation but has slow convergence.

https://www.sciencedirect.com/science/article/pii/S0377042703004643
https://en.wikipedia.org/wiki/Euler%E2%80%93Maruyama_method
https://advancesindifferenceequations.springeropen.com/articles/10.1186/s13662-018-1466-5

Recommended
https://hautahi.com/sde_simulation

Homework_24R

Stochastic differential equations (SDE). What are the differences respect to the ordinary differential equations (ODE). Try to understand and explain in your own words why the Itô calculus has been introduced and what is the main intuition behind the Itô integral.

SDE vs ODE

An equation containing the derivatives of one ormore dependent variables, with respect to one or more independent variables, is said to be a differential equation.
Solve a differential equation means to find the function g(x) (called solution or integral) that makes the expression identically satisfied.

A ordinary differential equation (ODE) is a mathematical expression within which we find an unknown function y(x) and its first derivative y'(x).
A stochastic differential equation (SDE) is a differential equation in which one or more of the terms is a stochastic process, resulting in a solution which is also a stochastic process.

So:

A ordinary differential equation: dx(t)/dt = a(t)x(t) and x(0) = x₀
When we take the ODE and assume that a(t) is not a deterministic parameter but rather a stochastic parameter, we get a stochastic differential equation.

Itô calculus and Itô integral

Itô calculus extends the methods of calculus to stochastic processes such as Brownian motion. It has important applications in mathematical finance and stochastic differential equations.
The central concept is the Itô stochastic integral, a stochastic generalization of the Riemann–Stieltjes integral in analysis.

The Itô stochastic integral amounts to an integral with respect to a function which is not differentiable at any point and has infinite variation over every time interval.

https://people.unica.it/claudioconversano/files/2015/10/Dispense01_EN.pdf
https://ethz.ch/content/dam/ethz/special-interest/mavt/dynamic-systems-n-control/idsc-dam/Lectures/Stochastic-Systems/SDE.pdf
https://en.wikipedia.org/wiki/It%C3%B4_calculus#Differentiation_in_It%C3%B4_calculus

Homework_23R

The Geometric Brownian motion and its importance for applications.
The Ornstein-Uhlenbeck / Vasicek models and the concept of mean reversion.

The Geometric Brownian motion

A geometric Brownian motion (GBM) (also known as exponential Brownian motion) is a continuous-time stochastic process in which the logarithm of the randomly varying quantity follows a Brownian motion(the random motion of particles suspended in a medium) with drift (the change of the average value of a stochastic (random) process).

A stochastic process S_t is said to follow a GBM if it satisfies the following stochastic differential equation (SDE):

dS_t = μS_tdt + σS_t dW_t

Brownian motion is often used to explain the movement of time series variables, and in corporate finance the movement of asset prices.
A common assumption for stock markets is that theyfollow Brownian motion, where asset prices are constantly changing often by random amounts

Ornstein-Uhlenbeck or Vasicek process

The Ornstein-Uhlenbeck or Vasicek process is a stochastic process which is stationary, Gaussian, and Markovian.
Over time, the process tends to drift towards its long-term mean: such a process is called mean-reverting.
The Vasicek process is the unique solution to the following stochastic differential equation:

dX_t = k(p – X_t)dt + σdW_t

When p = 0 we talk of Ornstein model.
K is the “speed of reversion”: characterizes the velocity at which such trajectories will regroup around p in time

The Ornstein Uhlenbeck process is often used to model interest rates because of its mean reverting property.
Mean reversion is the process that describes that when the short-rate X_t is high, it will tend to be pulled back towards the long-term average level; when the rate is low, it will have an upward drift towards the average level. In Vasicek’s model the short-rate is pulled to a mean level p at a rate of k. The mean reversion is governed by the stochastic term σdW_t which is normally distributed.

https://en.wikipedia.org/wiki/Stochastic_drift
https://en.wikipedia.org/wiki/Geometric_Brownian_motion
https://towardsdatascience.com/geometric-brownian-motion-559e25382a55
https://ro.uow.edu.au/cgi/viewcontent.cgi?article=1705&context=aabfj
https://en.wikipedia.org/wiki/Vasicek_model
https://rstudio-pubs-static.s3.amazonaws.com/19584_ce31e798cffb430982fd2f8979b1a87f.html
https://www.sciencedirect.com/topics/economics-econometrics-and-finance/mean-reversion

Homework_24A

Refine your statistical application and your simulation station in the following way. Do a complete test, both “smart monkey” and “dumb monkey”, ( https://en.wikipedia.org/wiki/Monkey_testing ), fixing all issues and making all the desired final refinements to your 2 main applications developed during the course.

Code VB.NET

Simulation Application

https://drive.google.com/file/d/1OXINcmOoxhJx-ptDG324MYrnrg_xfqZx/view?usp=sharing

What I fixed:
Glivenko-Cantelli process
Vasicek process
Form interface and Button enable/disable
Calculation of histograms
Input control

Statistical application

https://drive.google.com/file/d/1QOazNsVqjGlmlNBqgmUSb53zzyTSTnJt/view?usp=sharing

What I fixed:
Form interface and button control
Input control
Drawing of histograms

Unit test:

https://docs.microsoft.com/it-it/visualstudio/test/walkthrough-creating-and-running-unit-tests-for-managed-code?view=vs-2019

https://docs.microsoft.com/it-it/visualstudio/test/getting-started-with-unit-testing?view=vs-2019

we can manually test programs, by entering random values or build programs that test the application, as described above.

Homework_23A

Add to your simulation program, the simulation of the SDE for Geometric Brownian motion and Vasicek process.

#Code VB.Net

https://drive.google.com/file/d/1H0-ZDydHAcNYE99PgGGagWG48yS0qGPj/view?usp=sharing

The code contains the stochastic processes of the applications:

13_A
- where represent the sum of i Bernoulli random variables p^x(1-p)^(1-x) observed at the times j=1, …, i.
- Represent with histogram the distribution of f(i) at different time. (f(i) is the sum of i Bernoulli random variables)
- compute the absolute and relative frequency of the f(i)’s contained in the interval (p-ε, p+ε)

'generate for m paths n points
Public Function Generate_path() As Path
 ...
  for i = 0 to n
      ...
      p = 0.5 ' or a value between 0,1
      Dim v As Double = R.NextDouble
           If v <= p Then
                path.sequence.Add(0)
           else
                path.sequence.Add(1)
      ...
   next
   ...
End Function

15_A
- where illustrate the Glivenko-Cantelli theorem
- and create both the histogram and the empirical CDF of the sample mean

Private Sub Timer1_Tick(sender As Object, e As EventArgs) Handles Timer1.Tick   
   'Increase m and n for  closer to theor function

    For i = prec_m To m - 1
        listOfPath.Add(Generate_glCantPath(i))
    Next
   ...
    n += 1
    prec_m = m
    m += n
End Sub

' Uniform convergence of empirical CDF to the theoretical convergence
'1/n*(sum of indicator function)
Public Function Generate_glCantPath(m_path As Integer) As Path
...
 For i = 0 To n
        Dim ran As Double = R.NextDouble() * 100
        If ran < v Then
            path.sequence.Add(1)
            path.gl_canMean += 1
        Else
            path.sequence.Add(0)
            path.gl_canMean += 0
        End If


    Next

    path.gl_canMean /= n
...
End function

18_A
- where represent Poisson Distribution
- P(i) = P(i-1) + Random step(i), for i = 1, …, n, where Random step(i) is now a Bernoulli random variable with success probability equal to λ * (1/n)
- represent with histogram the distribution of P(i).
- Represent also the distributions of the following quantities (and any other quantity that you think of interest):
  - Distance (time elapsed) of individual jumps from the origin
  - Distance (time elapsed) between consecutive jumps

'generate for m paths n points
Public Function Generate_path() As Path
 ...
  for i = 1 to n +1
      ...
      Dim prob As Double = path.probability(i - 1)
      p = lambda/n' or a value between 0,1
      Dim v As Double = R.NextDouble
           If v <= p Then
                 path.probability.Add(prob + v)
           else
                 path.probability.Add(prob)
      ...
   next
   ...
End Function

20_A
- where represent Brownian Motion
- P(t) = P(t-1) + Random step(t), for t = 1, …, n, where Random step(t) is now: σ * sqrt(1/n) * Z(t), where Z(t) is a N(0,1) random variable
- represent with histogram the distribution of P(t).

'Generate N(0,1)
Public Function Generate_z()
        Dim u1 As Double
        Dim u2 As Double
        Dim s As Double = 1.0

        While s >= 1.0
            u1 = 2.0 * R.NextDouble - 1
            u2 = 2.0 * R.NextDouble - 1
            s = u1 * u1 + u2 * u2
        End While

        Return u1 * Math.Sqrt((-2 * Math.Log(s)) / s)
    End Function

Public Function Generate_pathNormal()
    ...
      For i = 1 To n
            Dim z As Double = Generate_z()
            'Generate Rademacher random variable 
            Dim v As Double = R.NextDouble * 2.0 - 1
            Dim rand As Double = (sigma / Math.Sqrt(n) * z)
            If v <= rand Then
               ...
            End if
   ...
End Function

23_A
- Geometric Brownian motion
- Vasicek process