Two Properties of Markov Models 

The general equations of an nstate Markov model can be expressed in terms of matrices as MP = dP/dt where P is the column vector P = [P_{1} P_{2} ... P_{n}]^{T} and M is the n´n transition matrix with the components 

_{} 

The symbol l_{i,j} signifies the exponential transition rate from state i to state j. This includes both failure rates and repair rates, and it allows for transitions in both directions between every pair of states. 

For each j, let C_{j} denote the jth column vector of M, excluding the diagonal term M_{j,j}. Hence C_{j} has dimension n1. This vector represents the transition rates from state j to each of the other n1 states. We also define (dP_{j}/dt)^{+} as the rate of probability flow into state j. 

Proposition: At steadystate conditions the quantity (dP_{j}/dt)^{+}/(1P_{j}) for any state j depends on the direction of C_{j} but is independent of the magnitude. 

Proof: Let m denote the (n1)´(n1) matrix formed by deleting the jth row and column from the full transition matrix M, and let p denote the column vector formed by deleting the jth element from full state vector P. Then the equations of state give 

_{} 

Setting dp/dt = 0 for the steadystate condition and solving this equation for p gives 

_{} 

Now, letting I denote the identity row vector I = [1 1 ... 1] of dimension n1, the conservation equation can be written as 1  P_{j} = Ip. Hence, multiplying both sides of the above equation by I and substituting from the conservation equation, we have 

_{} 

We also note that for the steadystate condition (dP_{j}/dt)^{+} is equal to the rate of probability flowing out of state j, which is simply P_{j} I C_{j}. Consequently we have 

_{} 

The column vector C_{j} can be written as the product of a scalar f and a unit vector U_{j} pointing in the direction of C_{j}. Making this substitution, the scalar f cancels out, and we are left with

_{} 

Hence the left hand quantity depends on the direction of C_{j}, but not on the magnitude, which was to be shown. 

Discussion: This proposition tells us that the steadystate "hazard rate" (the left hand quantity in the preceding equation) for any state j is independent of the overall rate of outflow from that state, but it does depend on how that outflow is distributed to the other states of the model. 

Proposition: If the only nonzero component of C_{j} is l_{j,k}, then for steadystate conditions the quantity (dP_{j}/dt)^{+}/(1P_{j}) is equal to the reciprocal of the mean time of transition from state k to state j. 

Proof: The reciprocal of the hazard rate is 

_{} 

which is a scalar equal to the sum of the numbers in the kth column of m^{1}. We wish to show that this equals the mean time of transition from state k to state j. To find an explicit expression for this mean time, it is most convenient to suppress the "return path" by setting l_{j,k} = 0, and then begin with the initial condition P_{k}(0) = 1 and integrate the time from t = 0 to infinity, weighted according to dP_{j}(t)/dt, which is the probability density function for entry into state j. Hence the mean time is given by 

_{} 

Notice that setting l_{j,k} equal to zero prior to solving for P_{j}(t) does not affect the time required for an object to transition from state k to state j, because this transition does not involve the return path from j back to k. Now, since we have suppressed the return path, the system equations are simply dp/dt = mp , which has the explicit dynamic solution 

_{} 

Notice that, since our initial condition is P_{k}(0) = 1, and state k is the exclusive return state from state j, the vector p(0) equals the vector U_{j} defined previously. We also have the conservation equation

_{} 

Substituting into the mean time integral gives 

_{} 

which was to be shown. 

Discussion: In retrospect the proof of this proposition is slightly superfluous, because it's essentially an immediate corollary of the previous proposition. Notice that the hazard rate of any state j is independent of the magnitude of the outflow rate from that state (by the previous proposition), so we can consider the limit as l_{j,k} approaches infinity. In this limit the value of P_{j} approaches zero, so the hazard rate goes to (dP_{j}/dt)^{+}, which is just the flow rate into state j (which also equals the flow rate out of state j). Also, it is stipulated that all the flow from state j goes to state k, so each element of the steadystate flow proceeds from state k to state j (by some route) and then immediately jumps back to state k to begin again. The steadystate flow rate (dP_{j}/dt)^{+} in the limit as l_{j,k} approaches infinity is therefore the reciprocal of the mean time of transition from state k to state j. It follows from the previous proposition that, for any value of l_{j,k}, the hazard rate (dP_{j}/dt)^{+}/(1Pj) equals the reciprocal of the mean time to transition from state k to state j, which was to be shown. 
