Let (\Omega,{\cal A},\Pr) be a probability space, let S be a countable nonempty set, and let T=\mathbb R_{\ge0} (T for "time"). Equip S with the
discrete metric, so that we can make sense of
right continuity of functions \mathbb R_{\ge0}\to S. A continuous-time Markov chain is defined by: • A
probability vector \lambda on S (which below we will interpret as the
initial distribution of the Markov chain), and • A
rate matrix Q on S, that is, a function Q:S^2\to\mathbb R such that • for all distinct i,j\in S, 0\le q_{i,j}, • for all i\in S, \sum_{j\in S:j\ne i}q_{i,j}=-q_{i,i}. (Even if S is infinite, this sum is
a priori well defined (possibly equalling +\infty) because each term appearing in the sum is nonnegative.
A posteriori, we know the sum must also be finite (not equal to +\infty), since we're assuming it's equal to -q_{i,i} and we've assumed Q is real valued. Some authors instead use a definition that's word-for-word the same except for a modified stipulation Q:S^2\to\mathbb R\cup\{-\infty\}, and say Q is
stable or
totally stable to mean \operatorname{range}Q\subseteq\mathbb R, i.e., every entry is real valued.) Note that the row sums of Q are 0: \forall i\in S,~\sum_{j\in S}q_{i,j}=0, or more succinctly, Q\cdot1=0. This situation contrasts with the situation for
discrete-time Markov chains, where all row sums of the transition matrix equal unity. Now, let X:T\to S^\Omega such that \forall t\in T~X(t) is ({\cal A},{\cal P}(S))-measurable. There are three equivalent ways to define X being
Markov with initial distribution \lambda and rate matrix Q: via transition probabilities or via the jump chain and holding times. As a prelude to a transition-probability definition, we first motivate the definition of a
regular rate matrix. We will use the transition-rate matrix Q to specify the dynamics of the Markov chain by means of generating a collection of
transition matrices P(t) on S (t\in\mathbb R_{\ge0}), via the following theorem. {{math theorem {{NumBlk|:|P(0)=([i=j])_{i,j\in S},~\forall t\in T~\forall i,j\in S~~(P(t)_{i,j})'=\sum_{k\in S}q_{i,k}P(t)_{k,j}.|}} }} We say Q is
regular to mean that we do have uniqueness for the above system, i.e., that there exists exactly one solution. We say Q is
irregular to mean Q is not regular. If S is finite, then there is exactly one solution, namely P=(e^{tQ})_{t\in T}, and hence Q is regular. Otherwise, S is infinite, and there exist irregular transition-rate matrices on S.{{efn|For instance, consider the example S=\mathbb Z_{\ge0} and Q being the (unique) transition-rate matrix on S such that \forall i\in\mathbb Z_{\ge0}~~Q_{i,i+1}=i^2,~Q_{i,i}=-i^2. (Then the remaining entries of Q will all be zero. Cf.
birth process.) Then Q is irregular. Then, for general infinite S, indexing S by the nonnegative integers \mathbb Z_{\ge0} yields that a suitably modified version of the above matrix Q will be irregular.}} If Q is regular, then for the unique solution P, for each t\in T, P(t) will be a
stochastic matrix. We will assume Q is regular from the beginning of the following subsection up through the end of this section, even though it is conventional to not include this assumption. (Note for the expert: thus we are not defining continuous-time Markov chains in general but only
non-explosive continuous-time Markov chains.)
Transition-probability definition Let P be the (unique) solution of the system (). (Uniqueness guaranteed by our assumption that Q is regular.) We say X is
Markov with initial distribution \lambda and rate matrix Q to mean: for any nonnegative integer n\ge0, for all t_0,\dots,t_{n+1}\in T such that t_0 for all i_0,\dots,i_{n+1}\in I, {{NumBlk|:|\Pr(X_0=i_0,\dots,X_{t_{n+1}}=i_{n+1})=\lambda_{i_0}\prod_{k\in\mathbb Z:0\le k\le n}P(t_{k+1}-t_k)_{i_k,i_{k+1}}.|}} Using induction and the fact that \forall A,B\in{\cal A}~~\Pr(B)\ne0\rightarrow\Pr(A\cap B)=\Pr(A\mid B)\Pr(B), we can show the equivalence of the above statement containing () and the following statement: for all i\in I,~\Pr(X_0=i)=\lambda_i and for any nonnegative integer n\ge0, for all t_0,\dots,t_{n+1}\in T such that t_0 for all i_0,\dots,i_{n+1}\in I such that 0 (it follows that 0), {{NumBlk|:|\Pr(X_{t_{n+1}}=i_{n+1}\mid X_{t_n}=i_n,\dots,X_{t_0}=i_0)=P(t_{n+1}-t_n)_{i_n,i_{n+1}}.|}} It follows from continuity of the functions (P(t)_{i,j})_{t\in T} (i,j\in S) that the trajectory (X_t(\omega))_{t\in T} is almost surely
right continuous (with respect to the
discrete metric on S): there exists a \Pr-null set N such that \{\omega\in\Omega:(X_t(\omega))_{t\in T}\text{ is right continuous}\}\subseteq N.
Jump-chain/holding-time definition Sequences associated to a right-continuous function Let f:T\to S be right continuous (when we equip S with the
discrete metric). Define :h=h(f)=(\inf\{u\in(0,+\infty):f(t+u)\ne f(t)\})_{t\in T})\cup\{+\infty,0\}, let :H=H(f)=(h^{\circ n}0)_{n\in\mathbb Z_{\ge0}} be the
holding-time sequence associated to f, choose s\in S, and let :y=y(f)=\left(\begin{cases}f(\sum_{k\in n}H_k)&\text{ if }\sum_{k\in n}H_k be "the
state sequence" associated to f.
Definition of the jump matrix Π The
jump matrix \Pi, alternatively written \Pi(Q) if we wish to emphasize the dependence on Q, is the matrix \Pi=([i=j])_{i\in Z,j\in S}\cup\bigcup_{i\in S\setminus Z}(\{((i,j), -q_{i,j}/ q_{i, i}):j\in S\setminus\{i\}\}\cup\{((i,i),0)\}), where Z=Z(Q)=\{k\in S:q_{k,k}=0\} is the
zero set of the function (q_{k,k})_{k\in S}.
Jump-chain/holding-time property We say X is
Markov with initial distribution \lambda and rate matrix Q to mean: the trajectories of X are almost surely right continuous, let f be a modification of X to have (everywhere) right-continuous trajectories, \sum_{n\in\mathbb Z_{\ge0}}H(f(\omega))_n=+\infty almost surely (note to experts: this condition says X is non-explosive), the state sequence y(f(\omega)) is a discrete-time Markov chain with initial distribution \lambda (jump-chain property) and transition matrix \Pi(Q), and \forall n\in\mathbb Z_{\ge0}~\forall B\in{\cal B}(\mathbb R_{\ge0})~\Pr(H_n(f)\in B)=\operatorname{Exp}(-q_{Y_n,Y_n})(B) (holding-time property).
Infinitesimal definition We say X is
Markov with initial distribution \lambda and rate matrix Q to mean: for all i\in S, \Pr(X(0)=i)=\lambda_i and for all i,j, for all t and for small strictly positive values of h, the following holds for all t\in T such that 0: :\Pr(X(t+h) = j \mid X(t) = i) = [i=j] + q_{i,j}h + o(h), where the term [i=j] is 1 if i=j and otherwise 0, and the
little-o term o(h) depends in a certain way on i,j,h. The above equation shows that q_{i,j} can be seen as measuring how quickly the transition from i to j happens for i\neq j, and how quickly the transition away from i happens for i= j. ==Properties==