Purchase Solution

Markov Chain and Steady State Probabilities

Not what you're looking for?

Ask Custom Question

1. A production process contains a machine that deteriorates rapidly in both quantity and output under heavy usage, so it is inspected at the end of each day. Immediately after inspection the condition of the machine is noted and classified into one of four possible states:

State
0: good as new
1: Operable - minimum deterioration
2: Operable - major deterioration
3: Inoperable and replaced by a good new machine

The process can be modeled as a Markov Chain with the following transition matrix:

State...............0................1................2...............3
0.....................0................7/8............1/16..........1/16
1.....................0................3/4.............1/8...........1/8
2.....................0.................0...............1/2............1/2
3.....................1.................0................0...............0

a) find the steady state probabilities of being in each state
b) if the daily costs of being in states 0, 1, 2, 3 are $0, $1000, $3000, $6000 respectively, what is the expected long run daily cost?
c) reclassify state 3 as an absorbing state and calculate the expected number of days for a 'good as new' machine to become inoperable.

Purchase this Solution
Solution Summary

The steady state probability is calculated from the transition matrix.

Purchase this Solution
Free BrainMass Quizzes
Exponential Expressions

In this quiz, you will have a chance to practice basic terminology of exponential expressions and how to evaluate them.

Probability Quiz

Some questions on probability

Geometry - Real Life Application Problems

Understanding of how geometry applies to in real-world contexts

Know Your Linear Equations

Each question is a choice-summary multiple choice question that will present you with a linear equation and then make 4 statements about that equation. You must determine which of the 4 statements are true (if any) in regards to the equation.

Solving quadratic inequalities

This quiz test you on how well you are familiar with solving quadratic inequalities.

View More Free Quizzes
Related BrainMass Solutions

  • Markov Processes

    It also answers a numerical on Markov process that asks for the construction of the matrix of transition probabilities, and for providing a system of equations for calculating the steady state probabilities.

  • PERT/CPM - Markov Process.

    Solution contains answers of multiple choice questions , the matrix of transition probabilities and Provide a system of equations for calculating the steady state probabilities.

  • Markov Chain: Drawing the State Diagram of the Chain

    392323 Markov Chain: Drawing the State Diagram of the Chain See attached file for better format.

  • Markov Chain Problem with Four Parts

    The solution gives detailed steps on completing a markov chain problem with four parts using hospital example.

  • Problems: Probability, Markov Chain, Baye's Theorem

    Find a steady state distribution vector for the Markov chain with transition [0.25 0.75] P= [0.2 0.8] 4. A couple has 8 foster children, including 3 girls and 5 boys. Two-thirds of the girls have brown eyes.

  • Some questions about a Markov chain.

    17951 Some questions about a Markov chain. Category: Statistics > Simple Regression Subject: birth-death process Details: Let a Markov chain have transition probabilities Pi,i+1 = 1/(i+1) and Pi,0 = i/(i+1) for i=0,1,2, . . . .

  • Markov Processes

    Let be the long run or steady state market shares AS we know in steady state the difference in market shares/transition probabilities between two periods in the long run are negligible or zero.

  • eigenvalues and eigenvectors of matrix

    Finish the last row to make the matrix A a Markov matrix and find the steady state vector. 4. Compute (A^H)A and A(A^H) for A = [j 1 j; 1 j j].

  • Stochastic Processes : Poisson Process and Markov Chains

    Then {X0,X1,X2,.. .} is a Markov chain. (a) Find the transition probabilities of the Markov chain. (b) Find the stationary distribution of the Markov chain.

View More Related Solutions