Math 865 - Stochastic Processes I


Meeting MWF, 1-1:50PM, Snow 152

Office hours: Drop by, or email me for an appointment.




1.) Probability and Random Processes

2.) One Thousand Exercises in Probability

Both by Grimmett and Stirzaker


We will aim to cover the following chapters from the text, and also cover some other topics in more detail.


Quick review of Chapter 1+2+(Little bit of 7) to get used to the book's notation and style.

Chapter 4.12.  Stein's method for Poisson approximation.   See also the survey by Nathan Ross and these nice lecture notes by Partha Dey.

Selected topics from Chapters 5--13, depending on our interests.


Possible topics:


There are many topics to choose from the textbook.  We are open to suggestions. 


Stein's method for Normal approximation (not in the textbook, we will follow Nathan Ross' survey)

Random walk (Chapters 3.9, 3.10, and also the book by Durrett Chapter 4.)

Branching Processes (Chapter 5.4)

Large Deviations (Chapter 5.11)

Markov Chains (Chapter 6) See also this textbook by Levin, Peres, and Wilmer.

Martingales (Chapter 7,12) 

Random Processes and the Ergodic theorem (Chapter 8,9) See also these lecture notes by Sarig.

Brownian Motion (Chapter 13)  See also the book by Morters and Peres.



Your grade will be based on a combination of homework, presentations, and tests.   Students will have chances to present their own research, relevant topics of interest, and interesting exercises in class.


Rough grading schemes: 


Presentations:  Email me a few days in advance to let me know what you want to do, and how much time you expect it to take.


Homework: Do some of the exercises assigned in class and in the lecture notes.  

I'll try to type out all the exercise assigned in class, and I might also come up with a few more

Hand them in when you are ready, but please don't wait till the end of the semester!

Feel free to discuss the exercises with your classmates or ask me for help.


Test 1: Feb 27.   test1.pdf

Test 2: April 27  test2.pdf

Homework deadlines:  April 15 for exercises that have appeared before Spring break and May 6 for exercises that have appeared after the break.

Presentations:  Please come and see me before the Easter holiday.


Scheme 1


Presentations 40%

Homework 40%

Test 1 10%

Test 2 10%


Scheme 2


Presentations 20%

Homework 60%

Test 1 10%

Test 2 10%


Scheme 3


Presentations 25%

Homework 25%

Test 1 25%

Test 2 25%


(Your mark will the maximum of the marks of each scheme.)




Some lecture notes and exercises.


Measure theory lite


Review of Binomials and Poisson random variables


Intro to coupling


Intro to TV-distance


Intro to Stein-Chen


Intro to Stein's method


Intro to Markov chains


Classical Coupling of Markov Chains


Stationary distributions and return times


Reversible Markov chains


A little more measure theory


Abstract expectation


Abstract conditional expectation


Conditional expectation in Ell 2.


Introduction to martingales


Optional stopping


Martingale convergence theorem


Some recommended exercises from the text for self-study and review.  You may have already seen similar questions in previous courses. The solutions appear in One Thousand Exercises in Probability.


1.22, 1.24, 1.35, 1.5.1, 1.8.14, 1.8.16, 1.8.17, 1.8.18




3.4.7, 3.52, 3.11.13, 3.11.18, 3.1135, 3.11.40


4.12.1, 4.12.6.


5.6.1, 5.6.2, 5.6.3, 5.6.4, 5.6.5


6.1.2, 6.1.7, 6.1.8, 6.3.9,  6.5.6,  6.14.4, 6.15.43, 6.15.44,


7.11.16, 7.11.17, 7.11.21



Other textbooks:


Finite Markov chains and algorithmic applications, Haggstrom

Lectures on the coupling method, Lindvall

Markov Chains, Norris

Probability theory and example, Durrett

Probability with martingales, Williams

Ergodic theory, Petersen




Some more presentation topics.  We are open to suggestions from the textbook or other papers/sources.  You may need to be at school for these links to work:



Density and Uniqueness in percolation


Iterating Von Neumann's Procedure for Extracting Random Bits


Avoidance Coupling


Poisson Approximation and the Chen-Stein Method


Greedy clearing of persistent Poissonian dust


Shuffling Cards and Stopping Times


Upcrossing inequalities for stationary sequences and applications


How to Gamble If You're In a Hurry


Exact sampling with coupled Markov chains and applications to statistical mechanics


Generating random spanning trees more quickly than the cover time


Stationary random graphs on Z with prescribed iid degrees and Finite mean connections


Poisson thickening


Bernoulli Schemes of the Same Entropy are Finitarily Isomorphic


Improved bounds for the symmetric rendezvous value on the line


On choosing and bounding probability metrics


Closed form summation for classical distributions: variations on a theme of de Moivre


Poisson approximation for dependent trails


A Probabilistic Proof of the Lindeberg-Feller Central Limit Theorem