I am a graduate student at Stanford, and I was wondering how much knowledge of Stochastic Modeling and Stochastic Control would help me as I get into data sciences as a career. Have to decide on my coursework for the quarter. Would appreciate if somebody could shed some light on how widely used these are.


asked 04 Apr '14, 20:02

shadowblade360's gravatar image

accept rate: 0%

I can only speak from my own limited vantage point.... (my opinions are necessarily limited by my own experience and observations -- folks whose experiences are broader than mine, please feel free to contradict me). Where I'm coming from: I studied optimal control in school and am now in a "data sciency" sort of role.

If I had to choose between the two courses, I'd choose stochastic modeling. But rather than focus on mastering specific algorithms, I would try to learn how to identify pitfalls and limitations in the modeling of stochastic systems. In particular, I would want to focus on mastering the basic concepts of statistics, such as independence and heteroskedasticity. Independence is a very common assumption in statistical modeling but doesn't always hold in real life, which leads to serious errors. A very simple example is this: let's say you have a model Y = f(X1, X2), and X1 and X2 have particular pdfs. You run a Monte Carlo, drawing from the pdfs of X1 and X2 independently and then calculating Y. Problem is, you forgot to check if X1 and X2 were actually independent in the first place! If it turns out they're not, and your answer might several orders of magnitude off than if were to draw from a joint distribution of X1 and X2. This actually happened in a published paper and the conclusions arrived at turned out to quite wrong.

In my opinion, stochastic control has limited practical application in industry apart from simple traditional algorithms like LQG and Kalman filters. Real-life control systems typically usually deal with stochasticity through simple feedback and/or robust tunings, rather than through sophisticated stochastic control techniques.

Some people have tried to apply stochastic control theory to finance, economics and other fourth quadrant (to borrow N.N. Taleb's term) type systems, where behavior is nonlinear and unpredictable (cannot be modeled with a reasonable degree of accuracy due to fat tails). I am personally suspicious of the applicability of said theory in such domains.

In short: I think that sophisticated stochastic control techniques could conceivably work for physical systems (that have well-defined behaviors, known variability, etc.), but there are simpler ways to perform control on these systems. On the other hand, sophisticated stochastic control techniques don't actually work for 4th-quadrant systems but the mathematics is so elegant that it fools us into thinking it does, and we can end up being dangerously wrong. So unless you wanted to do research on stochastic control, or have an academic interest in the subject, I would skip the course.

(That said, you might want to read up about Kalman filters in your spare time. They're useful to know about, but you don't need a whole course on them.)


answered 06 Apr '14, 22:46

Gilead's gravatar image

Gilead ♦
accept rate: 15%

edited 06 Apr '14, 23:25

Thanks a ton for the advice! I appreciate that you took the time out to help.

(06 Apr '14, 23:00) shadowblade360

Hi Gilead,

I was wondering if it is a good idea not to go for either. I have other courses too that I can choose from, and even though they aren't data sciency at all, I would still gain from them. I was wondering if stochastic modeling is worth the opportunity cost of missing out on courses that don't contribute to my career plans ie data science. The other option of course is to stick to stochastic modeling that it seems would give me a moderate level of help in my career.

I am spending a lot of tuition. Would appreciate if you could help understand if stochastic modeling is useful at all

(07 Apr '14, 00:35) shadowblade360

I'm not really qualified to give you any specific guidance because like I said, I studied optimal control in grad school and stumbled into this trade by accident. Most of my co-workers followed similar paths. The only common denominator is that we all have backgrounds in heavily quantitative disciplines. I don't know what advice to offer except to say that if I were in your shoes, I'd pick courses that I'm most interested in. If there are more interesting courses than these, go fo r those. A lot of data-sciency stuff can be picked up fairly easily on the job.

(07 Apr '14, 18:01) Gilead ♦

Sounds good! Thanks for sharing your views! This helps.

(08 Apr '14, 01:41) shadowblade360

I would say this is a wide open question. The only right answer would be 'It depends'.

I assume that the courses will teach you...

  • Stochastic modeling - Basic probability theory, Markov chain, Queueing theory, (Maybe Markov decision process and/or Brownian motion)

  • Stochastic Control - MDP, Bellman function in terms of PDE, HJB equation...

I would take Stochastic Modeling, even though it will not be directly related to data analysis techniques. (there could be some overlaps because statistics and probability are related a lot)

Most of case, when you get the data, the data would be from some systems and a lot of cases from stochastic systems. So, if you don't have any knowledge on Stochastic system, data analysis itself can go wrong or the result can be interpreted in wrong way.

I would NOT take Stochastic Control unless interested in this topic. It is continuous version of Markov decision process. It requires knowledge of stochastic modeling as a prerequisite. Due to the continuous state variables, you need to use PDE (partial differential equation). This would be one of most attractive areas in stochastic optimization.

But, again it doesn't directly help to learn data analysis techniques, And it is not required to understand the system in most cases.

However, as I mentioned, it all depends. You don't know what kind of data you will handle and the data is from what kind of systems.


answered 06 Apr '14, 16:13

ksphil's gravatar image

accept rate: 14%

Thanks for the advice! This really helps. I was also wondering if it is a good idea to go for either of the courses at all. If stochastic modeling isnt too useful for a career in data science, I might as well not take it up. Do you think there is a solid benefit to be gained here (subjective obviously)?

(07 Apr '14, 00:37) shadowblade360

Again it depends. If you find other courses interesting or related to data analysis, you don't want to scarifies the opportunity to take those courses in order to take 'stochastic model'.

(07 Apr '14, 10:09) ksphil

Thanks for the advice! This helps a ton! :)

(08 Apr '14, 02:09) shadowblade360
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here



Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "Title")
  • image?![alt text](/path/img.jpg "Title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported



Asked: 04 Apr '14, 20:02

Seen: 9,412 times

Last updated: 08 Apr '14, 02:09

OR-Exchange! Your site for questions, answers, and announcements about operations research.