The wikipedia page for POMDP http://en.wikipedia.org/wiki/Partially_observable_Markov_decision_process says the components needed to specify a POMDP. Later it talks about belief vector. What is this for? Do, we need it to specify POMDP ? Also, I don't understand the line "An agent needs to update its belief upon taking the action \(a\) and observing \(o\). Why do we need observation? With the current belief vector and transition probabilities, we can compute the next belief.

asked 10 Jun '14, 14:32

sosha's gravatar image

sosha
100137
accept rate: 0%

edited 10 Jun '14, 14:37

fbahr's gravatar image

fbahr ♦
4.6k716


POMDP(Partially Observable Markov Decision Process) is an MDP(Markov Decision Process) where we cannot observe which state we are, currently. But, there is some hints(observation) available so that we can guess our current state. But, the hint itself can be probabilistic, one hint doesn't tell us where we are(state).

Still it is MDP so we have to make a decision on each epoch(period) and we may move to other state and get some hint(partially observable) on where we are. Belief is the guess on the our current state.

So, the effort to trying to guess the current state(where we are) is the belief update. Each period, we will get new observation(hint) so we can improve our belief (belief update).

If we don't know the relationship (at least stochastic) between the observation and state, we are clueless.

You can get the above from the website I suggested. Even though the website doesn't have formula, it will give you basic ideas.

www.pomdp.org

link

answered 10 Jun '14, 19:53

ksphil's gravatar image

ksphil
66717
accept rate: 14%

edited 10 Jun '14, 20:06

Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "Title")
  • image?![alt text](/path/img.jpg "Title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Tags:

×58

Asked: 10 Jun '14, 14:32

Seen: 1,000 times

Last updated: 10 Jun '14, 21:54

OR-Exchange! Your site for questions, answers, and announcements about operations research.