I want a good book/reference to learn POMDP properly. Also, I want to know recent research on it. Any suggestion will be helpful. Thanks in advance.

asked 10 Jun '14, 11:53

sosha's gravatar image

accept rate: 0%

you would like to learn POMDP properly; I would be happy already if I knew what POMDP stands for...

(10 Jun '14, 12:36) Marco Luebbecke ♦

@Marco: Partially Observable Markov Decision Process. I thought this is a well known notation. BTW, is this the reason for downvoting ?

(10 Jun '14, 12:53) sosha

yes :-) I'd appreciate a little context, your background, etc. Not that I could recommend anything here, but I did not know about this standard abbrev. Lots of different people here.

(10 Jun '14, 13:11) Marco Luebbecke ♦

@Marco: I am a 1st yr PhD student working in the area of Stochastic Approximation, Reinforcement Learning, Dynamic Programming. I am aware of measure-theoretic probability theory.

(10 Jun '14, 13:26) sosha

Unfortunately, we don't have many active members in Stochastic optimization (excluding stochastic programming) here in OR-exchange, compared to members in Deterministic optimization or Mathematical programming (including stochastic programming).

And, the POMDP is mostly utilized from CS or EE, (example, robot mouse finding the exit in maze)

Even though POMDP is a general version of MDP, the application areas and the solution methodologies are quite different from MDP.

Probably, you have to learn MDP first if you didn't yet. and then visit the POMDP website.


I think it is the best place you can start.

I don't know any book for POMDP.


answered 10 Jun '14, 13:31

ksphil's gravatar image

accept rate: 14%

@ksphil: Actually, that website does not have any formulas, everything is writtwn in words. BTW, do you know any good discussion forum for stochastic optimization.

(10 Jun '14, 13:34) sosha

@sosha Even though stochastic part is small here compared to deterministic part, OR-exchange is the best place regrading to Operations Research as far as I know.

I have a hypothesis that stochastic people tend to be more shy than deterministic or the size of population are significantly smaller.

Let me know if you find the better place.

For POMDP, CS or EE related forum could be better place.

(10 Jun '14, 13:57) ksphil

Here are some slides on MDPs and POMDPs that I heard Mykel Kochenderfer (Stanford aero/astro prof) present recently. They provide a nice high-level overview.

He recommends Markov Decision Processes in Artificial Intelligence (edited by Sigaud and Buffet) as a good reference.


answered 10 Jun '14, 15:58

mbloem's gravatar image

accept rate: 50%

@mbloem: Thanks for the info.

(11 Jun '14, 06:18) sosha

Here is a nice reference for an application of POMDP in breast cancer screening. The paper is part of the first author's dissertation at the University of Wisconsin, and it won a number of awards. I tried finding a copy of his dissertation online with no luck; it may not be publicly available yet.

My suggestion for finding recent research on this topic would be to search top journals as well as a thesis/dissertation database. Your academic institution may have such a database available through the library system - that is how I access mine. I just searched on a ProQuest database using the search term "POMDP" and found 700+ results. Looking at dissertations can serve several purposes including that they often represent cutting-edge research, and they can provide very nice literature reviews that those of us who are not familiar with the POMDP literature could not provide.


answered 12 Jun '14, 15:24

Andreas's gravatar image

accept rate: 12%

edited 12 Jun '14, 15:24

Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here



Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "Title")
  • image?![alt text](/path/img.jpg "Title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported



Asked: 10 Jun '14, 11:53

Seen: 6,640 times

Last updated: 12 Jun '14, 15:24

OR-Exchange! Your site for questions, answers, and announcements about operations research.