• Let X_n and T_n be as defined in the previous statement. Defining a new stochastic process Y_t:=X_n for t \in [T_n,T_{n+1}), then the process Y_t is called a
semi-Markov process as it happens in a
continuous-time Markov chain. The process is Markovian only at the specified jump instants, justifying the name
semi-Markov. (See also:
hidden semi-Markov model.) • A semi-Markov process (defined in the above bullet point) in which all the holding times are
exponentially distributed is called a
continuous-time Markov chain. In other words, if the inter-arrival times are exponentially distributed and if the waiting time in a state and the next state reached are independent, we have a continuous-time Markov chain. • : \begin{align} & \Pr(\tau_{n+1}\le t, X_{n+1}=j\mid(X_0, T_0), (X_1, T_1),\ldots, (X_n=i, T_n)) \\[3pt] = {} & \Pr(\tau_{n+1}\le t, X_{n+1}=j\mid X_n=i) \\[3pt] = {} & \Pr(X_{n+1}=j\mid X_n=i)(1-e^{-\lambda_i t}), \text{ for all } n \ge1,t\ge0, i,j \in \mathrm{S}, i \ne j \end{align} • The sequence X_n in the Markov renewal process is a
discrete-time Markov chain. In other words, if the time variables are ignored in the Markov renewal process equation, we end up with a
discrete-time Markov chain. • :\Pr(X_{n+1}=j\mid X_0, X_1, \ldots, X_n=i)=\Pr(X_{n+1}=j\mid X_n=i)\, \forall n \ge1, i,j \in \mathrm{S} • If the sequence of \taus is independent and identically distributed, and if their distribution does not depend on the state X_n, then the process is a
renewal. So, if the states are ignored and we have a chain of iid times, then we have a renewal process. • :\Pr(\tau_{n+1}\le t\mid T_0, T_1, \ldots, T_n)=\Pr(\tau_{n+1}\le t)\, \forall n \ge1, \forall t\ge0 ==See also==