# how to calculate optimal policy for a MDP with deterministic transition

 0 Suppose for a MDP all the transitions are deterministic. Then can there by any other easy (other than value iteration/policy iteration) algorithm to calculate optimal policy ? asked 20 Jun '14, 11:35 sosha 100●1●3●7 accept rate: 0%

 0 The only advantage of deterministic transitions is that for finite horizon problems you can either run the value iteration problem forward in time or backward in time. Bellman's book on dynamic programming has several examples of Markov decision processes with deterministic transition functions. answered 24 Feb '16, 00:49 adityam 11●1 accept rate: 0%
 0 Nothing different. Deterministic case is a special case of stochastic case. Usually, we don't call a deterministic problem MDP even though it is not wrong to do so technically. We call it just dynamic programming. (DP includes MDP) For finite time horizon problem, by backward induction For infinite time horizon problem, by value/policy iteration. When transition is deterministic, calculation is much easier, since you don't need to calculate Expectation of next period value functions. You just add the next period value function, instead. answered 26 Feb '16, 23:25 ksphil 667●1●8 accept rate: 14%
 toggle preview community wiki

By Email:

Markdown Basics

• *italic* or _italic_
• **bold** or __bold__
• image?![alt text](/path/img.jpg "Title")
• numbered list: 1. Foo 2. Bar
• to add a line break simply add two spaces to where you would like the new line to be.
• basic HTML tags are also supported

Tags: