The wikipedia page for POMDP says the components needed to specify a POMDP. Later it talks about belief vector. What is this for? Do, we need it to specify POMDP ? Also, I don't understand the line "An agent needs to update its belief upon taking the action \(a\) and observing \(o\). Why do we need observation? With the current belief vector and transition probabilities, we can compute the next belief.

asked 10 Jun '14, 14:32

sosha's gravatar image

accept rate: 0%

edited 10 Jun '14, 14:37

fbahr's gravatar image

fbahr ♦

POMDP(Partially Observable Markov Decision Process) is an MDP(Markov Decision Process) where we cannot observe which state we are, currently. But, there is some hints(observation) available so that we can guess our current state. But, the hint itself can be probabilistic, one hint doesn't tell us where we are(state).

Still it is MDP so we have to make a decision on each epoch(period) and we may move to other state and get some hint(partially observable) on where we are. Belief is the guess on the our current state.

So, the effort to trying to guess the current state(where we are) is the belief update. Each period, we will get new observation(hint) so we can improve our belief (belief update).

If we don't know the relationship (at least stochastic) between the observation and state, we are clueless.

You can get the above from the website I suggested. Even though the website doesn't have formula, it will give you basic ideas.


answered 10 Jun '14, 19:53

ksphil's gravatar image

accept rate: 14%

edited 10 Jun '14, 20:06

Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here



Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text]( "Title")
  • image?![alt text](/path/img.jpg "Title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported



Asked: 10 Jun '14, 14:32

Seen: 1,098 times

Last updated: 10 Jun '14, 21:54

OR-Exchange! Your site for questions, answers, and announcements about operations research.