Question 17 supplemental.  What is the Markov chain transition probability model?

Answer

To calculate the optimal policy, it is necessary to describe the behavior of the covariates. Makis and Jardine[1] have combined the Weibull PHM with a non-homogeneous discrete Markov process to predict the future development of covariates and failure times. Every covariate is approximated by a discrete variable with a finite number of values (states) and a joint transition probability matrix is calculated. Obviously, this method reduces the information obtained from the data, but usually does not affect the final result significantly (i.e. the decision policy). This is because the covariate measurements (at least in oil and vibration analysis) show a large amount of random variation, and the optimal policy is calculated from the average values of costs and replacement times (see eq. (6)). Also, this method follows engineering practice where often the covariate values, such as ppm (parts per million) of metal particles are classified as in “normal”, “warning” and “alarm” state, or vibration as “very smooth”, “smooth, “rough”, “very rough”. An alternative method that does not use discrete approximation would be to use a state space model for prediction of covariates, as it is done for furnace erosion prediction[2].

 

Every covariate state represents covariate values between two limits, including possibly  for the first and last limit. Usually 4 or 5 covariate states are defined according to the distribution of covariate values and practical experience. The Markov process is assumed non-homogeneous to represent different behavior of covariates in time. The non-homogeneity can be included either by some time dependent functional model for the transition matrix, or by dividing the time scale into intervals in which the transition probabilities are homogeneous. The later method is simpler and close to practical experience. For example, for most machines, working age is usually divided into “run-in” period, “normal” period, and “wear-out” period. This method is applied in the paper.

 

If the measurements (inspections) are performed at regular time points   , D being a fixed inspection interval, the m-dimensional Markov covariate process is defined as , . The transition probabilities  can be estimated from the data. If the inspection interval D is not constant, then the transition rates , , can be estimated. The transition probabilities can be then calculated from transition rates. For more details, see[3]. The transition probabilities should be estimated separately for each selected time interval.