Please see the attached file for the fully formatted problems.
At the beginning of each year, my car in good, fair, or broken-down condition. A good car will good at the beginning of the next year with the probability 0.85; fair with the probability 0.10; or broken-down quality with a probability of 0.05. A fair car will be fair of the next year with the probability 0.70 or broken-down quality with a probability of 0.30. It costs $6,000 to purchase a good car; a fair car can be traded in for $2,000; a broken-down car has no trade-in value and must immediately be replaced by a good car. It costs $1,000 per year to operate a good car and $1,500 to operate a fair car. Should I replace my car as soon as it becomes fair, or should I drive my car until it breaks-down? Assume that the cost of operating a car during a year depends of the type of car on hand at the beginning of the year (after a new car, if any arrives).
A Markov chain problem is investgated. The solution will point you in the right direction, but does not give a final solution.