Questions Tagged With stochastic-optimizationhttp://www.or-exchange.com/tags/stochastic-optimization/?type=rssquestions tagged <span class="tag">stochastic-optimization</span>enTue, 10 Apr 2018 06:28:50 -0400Stochastic optimisation for recruitment and training estimation - where to start?http://www.or-exchange.com/questions/15539/stochastic-optimisation-for-recruitment-and-training-estimation-where-to-start<p>I'm looking for advice on where to get started in terms of literature, techniques, keywords and standard formulations etc.</p>
<p>An organisation has a complex training program (represented as a digraph) where recruits of various types pass through training courses to reach placements within various offices/departments. Passing through all courses to reach a placement may take several years, and 5-10 courses. Courses have sessions at set times, maximum capacities, and random failure rates. Departments have random attrition each year.</p>
<p>The problem is how to maintain the required placements within the various offices/departments with some specified probability over some horizon. The decision variables are how many to recruit initially, and how many to advance along the various paths through the training continuum. (Recruits are not necessarily pre-committed to ultimate destinations.)</p>
<p>I see stochastic optimisation techniques often defined in terms of maximising expectations but this problem is more - find minimal recruitment such that the desired risk level across the placements is satisfied during the horizon.</p>
<p>Where to get started with this problem? Is there literature for this type of problem and if so what are the relevant keywords? What are the natural techniques to try first?</p>mathstudentTue, 10 Apr 2018 06:28:50 -0400http://www.or-exchange.com/questions/15539/stochastic-optimisation-for-recruitment-and-training-estimation-where-to-startstochastic-optimizationliterature-researchProving unequality between two-stage LP and its deterministic problemshttp://www.or-exchange.com/questions/15476/proving-unequality-between-two-stage-lp-and-its-deterministic-problems<p>Given the following <strong>deterministic equivalent</strong> definition of <strong>two-stage</strong> stochastic linear problems:</p>
<p>\[K = \min_{x} \left\{ c^T{x} + \sum_{k=1}^{M} p_kq_k^Ty_k ~|~ Ax = b; T_kx + W_ky_k = h_k; y_k \geq 0, x \geq 0 \,\,\forall ~k=1,2, \ldots M \right\} \]</p>
<p>where \((T_k, W_k, h_k, y_k)\) occurs with probability \(p_k > 0 ~\forall ~k = 1,2, \ldots, M\)</p>
<p>Now, I want to prove the following claim:</p>
<p>For \(M \geq 2\),</p>
<p>\[K \neq p_1 \min_{x} \left\{c^{T}x + q_1^{T} y_1 ~|~ Ax = b; T_1 x + W_1 y_1 = h_1; y_1 \geq 0, x \geq 0 \right\} + \\ \ldots + p_M min_{x} \left \{c^{T}x + q_M^{T} y_M ~|~ Ax = b; T_M x + W_M y_M = h_M; y_M \geq 0, x \geq 0 \right\}\]</p>
<p><strong>1st approach:</strong> </p>
<p>First, since \(\sum_{k=1}^{M}p_k = 1\), I planned to establish</p>
<p>\[min_{x} \left\{c^Tx + \sum_{k=1}^{M} p_kq_k^Ty_k ~|~ Ax = b; T_kx + W_ky_k = h_k; y_k \geq 0, x \geq 0 ~\forall ~k=1,2,\ldots M \right\} \\ > \sum_{k=1}^{M} min_{x} \left\{p_kc^Tx + p_kq_k^Ty_k ~|~ Ax = b; T_kx + W_ky_k = h_k; y_k \geq 0, x \geq 0 \right\}\]</p>
<p>However, such inequality seems not to always hold, as we know nothing about the signs of the coefficients of the objective function, besides the fact that the constraints form a polyhedron as a feasible region. The reverse sign is also not correct for <strong>every</strong> cases, I think. So I abandon this approach.</p>
<p><strong>2nd approach:</strong></p>
<p>Define optimal solutions \(x^{*}\) for K, and \(x_{1}^{*},x_{2}^{*}, \ldots, x_{M}^{*}\) for</p>
<p>\(\min_{x} \left\{p_kc^Tx + p_kq_k^Ty_k ~|~ Ax = b; T_kx + W_ky_k = h_k; y_k \geq 0, x \geq 0 \right\} ~\forall ~k=1,2,\ldots, K\)</p>
<p>By the sake of contradiction, assume</p>
<p>\[K = \sum_{k=1}^{M} \min_{x} \left\{p_kc^{T}x + p_kq_k^{T} y_k ~|~ Ax = b; T_{k}x + W_{k} y_k = h_k; y_k \geq 0, x \geq 0 \right\}\]</p>
<p>This is equivalent to: \(c^Tx^{*} = \sum_{k=1}^{M} p_{k} c^{T} x_k^{*}\).</p>
<p>However, note that \(Ax^{*} = Ax_{1}^{*} = Ax_{2}^{*} = \ldots = Ax_{M}^{*}\), and \(T_kx^{*} = T_kx_{1}^{*} = \ldots = T_kx_{M}^{*}\). So, does the first set of equality imply \(x^{*} = x_{1}^{*} = x_{2}^{*} = \ldots = x_{M}^{*}\), if \(A\) is an invertible matrix? But if that's the case, shouldn't this means \(\sum_{k=1}^{M} p_kc^Tx_k^{*} = c^Tx^{*} \sum_{k=1}^{M} p_k = c^Tx^{*}\) (since \(\sum_{k=1}^{M}p_k = 1\))? But this implies the claim above is wrong...</p>
<p><strong>My question:</strong> Could someone please help review my attempt above, and let me know how you could arrive at the desired contradiction, or if the claim is actually false? Any other suggestions for other approaches would be really appreciated.</p>ghjkWed, 28 Feb 2018 03:19:42 -0500http://www.or-exchange.com/questions/15476/proving-unequality-between-two-stage-lp-and-its-deterministic-problemsstochastic-optimizationtwo-stageindividual vs joint constraints in stochastic programminghttp://www.or-exchange.com/questions/15210/individual-vs-joint-constraints-in-stochastic-programming<p>In stochastic programming (SP), a common constraint is of the form
$$ \mathbb{P}(f(x,\xi) \leq 0) \geq \alpha. $$
If \(f\) is affine in \(x\) and \(\xi\), then we have the chance constraint of the form
$$ \mathbb{P}(A(\xi)x + b(\xi)+ c \leq 0) \geq \alpha $$
for \(A\in \mathbb{R}^{m \times n} \).
In this case, we can represent the joint constraint with \(m\) individual constraints of the form
$$ \mathbb{P}(a_i^T(\xi)x + b_i(\xi) + c \leq 0) \geq \alpha. $$
To confirm, we can do this because each dimension is independent? How can I better see this?</p>
<p>Also, if \(f\) were not linear in \(x\) and \(\xi\), then we are unable to separate the joint constraint into constraint components, right? A simple, introductory reference on this idea would be much appreciated!</p>jjjjjjTue, 19 Dec 2017 15:51:01 -0500http://www.or-exchange.com/questions/15210/individual-vs-joint-constraints-in-stochastic-programmingstochastic-optimizationStochastic programming, stochastic optimization, and robust programminghttp://www.or-exchange.com/questions/15208/stochastic-programming-stochastic-optimization-and-robust-programming<p>How should I think about the differences between stochastic optimization <a href="https://en.wikipedia.org/wiki/Stochastic_optimization">(SO)</a> and stochastic programming <a href="https://en.wikipedia.org/wiki/Stochastic_programming">(SP)</a>? From Wikipedia, it seems that SO is a framework that uses randomness to solve a <em>pre-existing</em> optimization problem whereas SP uses randomness to <em>formulate</em> an optimization problem. </p>
<p>If this is appropriate, then how does robust optimization <a href="https://en.wikipedia.org/wiki/Robust_optimization">(RO)</a>, which I might call <em>robust programming</em> in light of Wikipedia's SO/SP pages, fit in? It seems that SP makes use of probabilistic tools to work with explicit (distributional form) representations of uncertainty whereas RP assumes makes no explicit use of probabilistic tools outside of assuming known support for an uncertainty set. Is this the primary distinction? Is there a way to view RP as a subclass of SP problems? </p>jjjjjjMon, 18 Dec 2017 17:05:59 -0500http://www.or-exchange.com/questions/15208/stochastic-programming-stochastic-optimization-and-robust-programmingstochastic-optimizationrobust-optimizationSolving a two-stage stochastic integer program with "general integer first- and second-stage variables"http://www.or-exchange.com/questions/14867/solving-a-two-stage-stochastic-integer-program-with-general-integer-first-and-second-stage-variables<p>Hi everybody. I'm really glad that OR-X is back again.</p>
<p>I'm working on a <strong>stochastic</strong> supply chain procurement problem with various purchasing and transportation decisions. While the problem is completely linear, unfortunately, <strong>both first- and second-stage variables are general integer variables</strong> (their values can go up as far as thousands). Hence, I'm trying to solve a two-stage stochastic integer program with "integer first-stage" and "integer recourse". Finally, the uncertainty is handled via scenarios. Right now, I'm looking for possible solution methods that could handle such problem. So far, I've found some references such as <a href="https://www.math.hu-berlin.de/~stefan/RoemVigPSH.pdf">here</a> and <a href="https://link.springer.com/article/10.1007/s10107-016-1006-6">here</a>, that implement B&B tree search with different strategies to generate cuts.</p>
<p>I was wondering if anyone has prior experience with this type of problem and some hints and points to the relevant literature. In particular, I'm interested in something relatively easy that could be implemented in GAMS (using its BCH facility in case of needing to do a B&B search) as the rest of project is being handled via GAMS. However, this is not a deal-breaker.</p>
<p>Thanks in advance for your time and attention.</p>
<p>This question is also cross-posted <a href="https://www.researchgate.net/post/Solving_a_two-stage_stochastic_integer_program_with_general_integer_first-and_second-stage_variables">here</a>.</p>EhsanThu, 20 Jul 2017 13:28:47 -0400http://www.or-exchange.com/questions/14867/solving-a-two-stage-stochastic-integer-program-with-general-integer-first-and-second-stage-variablesinteger-programmingstochastic-optimizationtwo-stageFuzzy mathematical programming vs stochastic programminghttp://www.or-exchange.com/questions/14232/fuzzy-mathematical-programming-vs-stochastic-programming<p>Dear all</p>
<p>As we know, there are some approaches to tackle uncertainty in mathematical programming amongst with stochastic programming, robust optimization, and fuzzy mathematical programming. To the best of my knowledge, SP is used when there are enough data to estimate the probability distribution function, RO is used when we just know the interval that data varies (continuous form), and FMP is used when data are vague, ambiguous, or when data are not sufficient to estimate PDF of data.</p>
<p>There are a lot of problem in operations management in which there is no historical data, such supply chain network design, location of warehouses and so on. In this situation, it seems more appropriate to use FMP while in most prestigious journals such as Management science, Operations Research, etc, SP and RO are more acceptable while these approaches increase the complexity of model and in some cases make them nonlinear.</p>
<p>Can anyone tell me why SP and RO are more desirable to model uncertainty in these journals and North of America?</p>
<p>Thanks</p>Amin-ShSat, 01 Oct 2016 16:06:35 -0400http://www.or-exchange.com/questions/14232/fuzzy-mathematical-programming-vs-stochastic-programmingrobust-optimizationstochastic-optimizationfuzzy-mathematical-programmingFailed to optimize LPhttp://www.or-exchange.com/questions/13969/failed-to-optimize-lp<p>I wrote a monte carlo simulation and I used sample average approximation (SAA) method to optimize my model. when i generate L random samples from (10,....100) it shows me Failed to optimize LP. but for L=5 it gives me optimal and feasible result.
what is the source of the error it is logic?</p>NGoldMon, 27 Jun 2016 16:09:34 -0400http://www.or-exchange.com/questions/13969/failed-to-optimize-lpstochastic-optimizationcplexsimulationWorked example of 2-stage stochastic programming with constraintshttp://www.or-exchange.com/questions/13793/worked-example-of-2-stage-stochastic-programming-with-constraints<p>I am looking for any worked examples on multi-stage stochastic programming with recourse but failed to find any.
People most of the times assume a distribution is known (which in practice is not and a model of the evolution needs to be determined) and then describe all the rest. I know this is problem-specific but I would like to see examples of this from scratch, e.g. we have this raw historical data, we obtain this distribution, formulate the problem and solve it.</p>
<p>I might probably be asking for a lot but any resources would help. </p>lstavrWed, 25 May 2016 18:05:35 -0400http://www.or-exchange.com/questions/13793/worked-example-of-2-stage-stochastic-programming-with-constraintsprogrammingstochastic-processesstochastic-optimizationLagrangian formulation for nonlinear stochastic programminghttp://www.or-exchange.com/questions/13603/lagrangian-formulation-for-nonlinear-stochastic-programming<p>What are the conditions for formulating a Lagrangian, if the system's state is stochastic? </p>
<p>Some notation: \( x \in X \subset \mathbb{R}^d\) is the control variable, \(u \in \mathbb{R}^n\) is the state vector, \( ( \Omega, S, P ) \) with \( \omega \in \Omega , 0 \leq P(A \in S) \leq 1\) is a probability space. We know that the state u obeys an equilibrium equation, \( G(u(x, \omega)) = 0\), and we wish to optimize a differentiable function of the state, \( F(u(x, \omega))\). Given any realization \( \omega \), \( x \mapsto u(x, \cdot)\) is differentiable, whereas \(F(u)\) is a simple function e.g. \( \|u\|^2 \). If the equilibrium equation is linear, we can write it as \(G(x,\omega) u = b\), where \(b\) is a known forcing vector.</p>
<p>In general, if we formulate a Lagrangian functional \(L = F(x(u, \omega)) + \langle \lambda, G(u(x, \omega)) \rangle\), the multiplier vector \(\lambda\) will itself be a random variable (since the state function \(x \mapsto G(x, \omega)\) is random, there is a distinct multiplier for each realization, i.e. scenario, let's say we have \(N\) of them), and consequently, we will need to solve \(N\) adjoint problems, in addition to \(N\) "forward" solves, for a sample-average estimate of the subgradient \(\partial_x L\). </p>
<p>I wonder if this reasoning is fruitful and whether it ties in with the more established linear stochastic programming theory.</p>
<p>Some doubts: in this case, what is the definition of the inner product \( \langle \cdot, \cdot \rangle\)? How is the notion of adjoint operator related to that of non-anticipating constraints? </p>
<p>Also, I would be happy if anyone could point me to related references. Thank you in advance</p>ocramzTue, 19 Apr 2016 17:48:45 -0400http://www.or-exchange.com/questions/13603/lagrangian-formulation-for-nonlinear-stochastic-programmingstochastic-optimizationconvex-optimizationBook sequence from linear programming to stochastic programminghttp://www.or-exchange.com/questions/13335/book-sequence-from-linear-programming-to-stochastic-programming<p>Could someone please give me a list (in ascending order of difficulty) of books that can get me from a novice (basic mathematical programming) to an experienced level?</p>
<p>I am more interested in the issues of modelling and solving stochastic programming (stochastic control) problems, specifically multi-stage SP ones, however I realized that I need to start from the very basics. </p>
<p>My main difficulties until now are a proper explanation of how to model uncertainty, how mathematical problems can be expressed in a DP form and how one generates the dynamics of the problem (mainly distributions, I guess this is called system identification, correct me if I am wrong).</p>
<p>Also any recommendation on examples of problems and solutions using some open-source solver would be highly appreciated.</p>
<p>Thank you a lot in advance.</p>lstavrTue, 09 Feb 2016 10:20:19 -0500http://www.or-exchange.com/questions/13335/book-sequence-from-linear-programming-to-stochastic-programmingbooksstochastic-optimizationmathematical-modelingSuggestions for a Practice Problem in a Stochastic Programming Course Targeted towards Financial Engineering Studentshttp://www.or-exchange.com/questions/13310/suggestions-for-a-practice-problem-in-a-stochastic-programming-course-targeted-towards-financial-engineering-students<p>This semester, I'm teaching a graduate course on stochastic programming targeted towards financial engineering students. The syllabus is fairly standard (I'm using the book of Birge and Louveaux, <em>Introduction to Stochastic Programming</em>). As computational aspects are very important, I intend to define a course project in which students implement a classical solution method for a stochastic or robust financial problem. This practice problem should be easy to understand and require a modest theoretical effort as the focus in on the implementation aspects (we will use GAMS). </p>
<p>Right now, I've benders decomposition in mind for the solution method (as it's is the basis of many methods in stochastic programming), but I'm looking for a good practice problem. My own search has led me to the portfolio optimization problem with cardinality constraints (the maximum number of holdings is limited). This could be solved by the generalized benders decomposition.</p>
<p>I would really appreciate it if you could point out other possible candidates for the practice problem. While I prefer benders decomposition as the selected solution method, I'm open to consider other methods. Please note that the practice problem could be rooted in either stochastic programming or robust optimization techniques as I'm covering both.</p>
<p>Thanks in advance for your sharing your expertise and advice.</p>EhsanTue, 02 Feb 2016 00:33:14 -0500http://www.or-exchange.com/questions/13310/suggestions-for-a-practice-problem-in-a-stochastic-programming-course-targeted-towards-financial-engineering-studentscoursestochastic-optimizationfinancerobust-optimizationHow do stochastic programming, markov decision processes, and simulation modeling relate?http://www.or-exchange.com/questions/13230/how-do-stochastic-programming-markov-decision-processes-and-simulation-modeling-relate<p>Hello guys,</p>
<p>I'm trying to relate stochastic programming (e.g. using recourse models), Markov decision processes, and simulation modeling together. I know what each is separately, but I don't know when each is more suitable. I expect there is an overlap. Any pointers to books or articles that discuss these topics together?</p>
<p>Regards, </p>anahanaThu, 14 Jan 2016 09:24:21 -0500http://www.or-exchange.com/questions/13230/how-do-stochastic-programming-markov-decision-processes-and-simulation-modeling-relatestochastic-optimizationmarkov-chainssimulationHow to dualizehttp://www.or-exchange.com/questions/13005/how-to-dualize<p>We have a minimization problem, which includes the following two constraints, where M constant, and and x,y,z variables:
M(1- z) >= x-y
-M(1-z) <= x-y</p>
<p>I want to dualize these two constraints using lagrangean multipliers u >=0 and v>=0.
So I add in the objective function the following:
u * [x-y-M(1-z)] + v [ y-x-M(1-z)].
Or should I do u * [ M(1-z) -(x-y) ] + v [M(1-z) -(y-x)] ?</p>
<p>Thank you</p>spyimpTue, 10 Nov 2015 06:23:10 -0500http://www.or-exchange.com/questions/13005/how-to-dualizeoptimizationstochastic-optimizationdualityconvex-optimizationlagrangian-relaxationLagrangian Relaxation Fisherhttp://www.or-exchange.com/questions/12991/lagrangian-relaxation-fisher<p>This is one of the most well-known papers on Lagrangian relaxation , produced by Fisher:<br>
<a href="http://yalma.fime.uanl.mx/~roger/work/teaching/mecbs5100/papers/lagrangean%20relaxation/int-1985-Fisher.pdf">http://yalma.fime.uanl.mx/~roger/work/teaching/mecbs5100/papers/lagrangean%20relaxation/int-1985-Fisher.pdf</a></p>
<p>If you see, on page 13 (the number of the page as it is written on the document), it says that the derivative of the Lagrangian Problem function is : 8x1 + 2x2 + x3+ 4x4 - 10. But if you calculate it , you find : -8x1 - 2x2 - x3- 4x4 + 10.
Did he do such a basic mistake, or am I so wrong here ?</p>spyimpSat, 07 Nov 2015 06:25:47 -0500http://www.or-exchange.com/questions/12991/lagrangian-relaxation-fisherdiscrete-optimizationstochastic-optimizationlagrangian-relaxationStochastic programming practical examples/resourceshttp://www.or-exchange.com/questions/12778/stochastic-programming-practical-examplesresources<p>Dear all,</p>
<p>I am a newbie to optimization and I am currently working on some stochastic (integer) programming problems. I found a huge number of books/resources on the theory of multi-stage stochastic programming with recourse but I did not manage to find any similar resources on solving these problems (how to fit distributions based on previous data, generate the scenarios and solve the resulting deterministic equivalent on some programming language). I would be grateful if you could provide me pointers to such resources.</p>lstavrWed, 02 Sep 2015 10:49:15 -0400http://www.or-exchange.com/questions/12778/stochastic-programming-practical-examplesresourcesinteger-programmingstochastic-optimizationlinearmixed-integer-programmingNested L-shaped decompositionhttp://www.or-exchange.com/questions/12511/nested-l-shaped-decomposition<p>I am relatively new to GAMS and I want to write the Nested L-shaped (or Benders) decomposition in GAMS for a three-stage stochastic programming problem. Anyone who's already done that and can help me?</p>
<p>Thanks :)</p>SMTSun, 14 Jun 2015 08:05:25 -0400http://www.or-exchange.com/questions/12511/nested-l-shaped-decompositionbenders-decompositiongamsstochastic-optimizationcombinatorial benders cuts in l-shapedhttp://www.or-exchange.com/questions/12510/combinatorial-benders-cuts-in-l-shaped<p>I am working on a two-stage stochastic program where the master is a mip model, and slaves only have continuous variables.
For each scenario, I pass three variables set (k, a, b: first stage decisions, all binary) from master to slave. Then stochastic parameters realize. The slave problem has M constraints between decision variables a,b and contiunous s,c,x variables (slave variables). The first stage variable k is not in any M constraint in the slave problem, but it constraints s,c variables.</p>
<p>I have two questions:
1. With these conditions, can I use combinatorial benders cuts in the form: a(sum where/=1)+(1-a)(sum where/=0)+b(sum where/=1)+(1-b)(sum where/=0) >= 1 for this problem?
2. I havent seen any papers using combinatorial benders cuts in stochastic programming to generate feasibilty cuts. Are there any other conditions? Since we generate feasibility cuts for each scenario subproblem, I believe we can use these cuts as feasibility cuts for l-shaped algorithm.</p>
<p>Thanks for the help in advance.</p>paretSat, 13 Jun 2015 08:43:08 -0400http://www.or-exchange.com/questions/12510/combinatorial-benders-cuts-in-l-shapedbenders-decompositionstochastic-optimizationcutdecompositionSolving a nonlinear optimization problem with unknown decision variableshttp://www.or-exchange.com/questions/11557/solving-a-nonlinear-optimization-problem-with-unknown-decision-variables<p>Hello,</p>
<p>I need to solve a stochastic nonlinear optimization problem of this form: Find the decision variables Q1,..., Qn and Q_b (order quantities) such that the objective function (G(Q1,.,Qn and Q_b)) is minimized.Here Q1,...,Qn are same type of decision variables whereas Q_b is different.For this, I have considered a two variable case with Q1 and Q_b to find relation between the two.I have taken the relationship as constraint equations. I use fmincon in matlab to solve this problem. However, the last decision variable Q_b is always the same as the given upper bound.I think something is wrong for which the last variable is not optimized. Does matlab have any features to solve this type of problem? Or any other solver is there which can solve this kind of problem.</p>pinkyWed, 04 Mar 2015 08:30:38 -0500http://www.or-exchange.com/questions/11557/solving-a-nonlinear-optimization-problem-with-unknown-decision-variablesnonlinearstochastic-optimizationnonlinear-optimizationReferences for open-loop feedback control for discrete-time discrete-state POMDPhttp://www.or-exchange.com/questions/11175/references-for-open-loop-feedback-control-for-discrete-time-discrete-state-pomdp<p>I am looking for the application of open-loop feedback control to determine sub-optimal policies for partially observed Markov decision process.</p>
<p>The applications normally focus on finite-horizon POMDP. I am looking for some text on the application of open-loop feedback control for infinite horizon POMDP along with some known structural results on the sub-optimal policies.</p>
<p>Thanks </p>anon123Mon, 26 Jan 2015 20:42:45 -0500http://www.or-exchange.com/questions/11175/references-for-open-loop-feedback-control-for-discrete-time-discrete-state-pomdpstochastic-optimizationmarkov-chainsConfusion on how to obtain expected costhttp://www.or-exchange.com/questions/10239/confusion-on-how-to-obtain-expected-cost<p>In a stochastic optimization problem, assume a scenario-tree with N nodes spread out in T stages.
Let prob(k) denote the probability that node 'k' will realize.
Let cost(k) denote the cost associated with node 'k'.</p>
<p>According to wikipedia, the total expected cost is the sum of cost(k)*prob(k)/T because it can be viewed as the weighted average cost, with the prob(k) being the weight.</p>
<p>Most of us know that the expected cost is just the sum of cost(k)*prob(k) across all k = 1..N.</p>
<p>Which interpretation do you use?</p>spyimpThu, 25 Sep 2014 19:32:44 -0400http://www.or-exchange.com/questions/10239/confusion-on-how-to-obtain-expected-costoptimizationstochastic-optimizationnonlinear-optimizationWhich research direction is better, data mining or traditional OR?http://www.or-exchange.com/questions/10105/which-research-direction-is-better-data-mining-or-traditional-or<p>Hi all,</p>
<p>First of all, thank you for reading this post. I do need some of your professional advice on my situation. I am a new PhD student in industrial engineering and I am so puzzled on the research direction I should pick.</p>
<p>The two available directions I am interested in now are 1) data mining/machine learning and 2) traditional OR. </p>
<p>On one hand, professor A focuses more on CS side and does a lot of machine learning application such as prediction and time-series analysis in healthcare. I believe this area is a lot hotter in the coming 10-20 years in the industry. On the other hand, professor B is experienced in traditional OR side. He is the expert in response surface modeling, statistics, and dynamic programming. He exactly knows what should be done for each project, and he knows well how to integrate and apply bunch of complicated theory and knowledge upon a specific problem.</p>
<p>As for the advisers themselves - Professor A is encouraging and perhaps a little bit pushy, demanding result from you all the time. Otherwise he will not spend time on you. Whereas for Professor B, he is very professional and experienced in leading and giving one research directions. </p>
<p>In this regard, I love Professor A's research topic and Professor B's work style. I do want to work for Professor B but I am just afraid that I cannot get a job after graduation easily as his research topics (response surface modeling, statistics, and dynamic programming) are not very hot indeed in the industry. Also the research difficulty for Professor B's topics is obviously a lot harder.</p>
<p>If you were to make a blunt decision of choosing one of these two advisers, which one would you choose? I realize I can have co-advising arrangement but this is something I don't prefer. I don't want to follow Professor A on the first year and then follow Professor B on the second year. It shows you are not loyal enough to someone. </p>
<p>Thanks, again, for reading this long post. I do look forward to your constructive comment!</p>
<p>Sincerely,
Mr.F</p>bookyeah1679Mon, 25 Aug 2014 00:01:50 -0400http://www.or-exchange.com/questions/10105/which-research-direction-is-better-data-mining-or-traditional-oroperations-researchstochastic-optimizationmachine-learningdata-miningA basic question on Stochastic gradient descenthttp://www.or-exchange.com/questions/10063/a-basic-question-on-stochastic-gradient-descent<p>Consider a stochastic gradient iteration: </p>
<p>\(\theta_$k+1$ = \theta_$k$ - \gamma_k F(\theta_k))\</p>
<p>where $F$ is a noisy estimate of the gradient $\nabla f$ </p>
<p>Now, a book says that it converges in the following sense : $f(\theta_k)$ converges and $\nabla f(\theta_k)$ converges to zero and then it says that it is the strongest possible result for gradient related stochastic approximation. </p>
<p>What is the meaning of it ? Why does not it shows the convergence of the iterates ?</p>soshaFri, 15 Aug 2014 09:27:34 -0400http://www.or-exchange.com/questions/10063/a-basic-question-on-stochastic-gradient-descentstochastic-optimizationhow to calculate optimal policy for a MDP with deterministic transitionhttp://www.or-exchange.com/questions/9868/how-to-calculate-optimal-policy-for-a-mdp-with-deterministic-transition<p>Suppose for a MDP all the transitions are deterministic. Then can there by any other easy (other than value iteration/policy iteration) algorithm to calculate optimal policy ?<br>
</p>soshaFri, 20 Jun 2014 11:35:19 -0400http://www.or-exchange.com/questions/9868/how-to-calculate-optimal-policy-for-a-mdp-with-deterministic-transitionstochastic-optimizationProspect of research in some stochastic optimization/approximation fieldhttp://www.or-exchange.com/questions/9851/prospect-of-research-in-some-stochastic-optimizationapproximation-field<p>This question is a not a technical one. Sorry for that. As I am new to the area of stochastic optimization/control, I want to know the active prospect of research in the following areas</p>
<p>1) Simultaneous Perturbation Stochastic Approximation</p>
<p>2) Reinforcement learning Algorithms in Partially Observable Environments</p>
<p>3) Risk Sensitive Stochastic Optimal Control</p>
<p>4) Rolling horizon procedure for approximate dynamic programming. </p>
<p>Any advise by the experience researchers in the fields will be helpful. </p>soshaThu, 19 Jun 2014 00:53:29 -0400http://www.or-exchange.com/questions/9851/prospect-of-research-in-some-stochastic-optimizationapproximation-fieldstochastic-optimizationDirectly solving Bellman's optimality equation by iterative methodshttp://www.or-exchange.com/questions/9848/directly-solving-bellmans-optimality-equation-by-iterative-methods<p>Consider Bellman optimally equation for value function. Now, if we use the iterative algorithm to compute the optimal value function will it converge to the same ? </p>
<p>I see that books talks about policy/value iteration algorithms which are alternate sequence of policy evaluation and improvement steps. This is shown to converge.</p>
<p>Why the books does not do the obvious thing. There must be some reason.</p>soshaWed, 18 Jun 2014 13:15:34 -0400http://www.or-exchange.com/questions/9848/directly-solving-bellmans-optimality-equation-by-iterative-methodsstochastic-optimizationHow to use POMDP optimal policyhttp://www.or-exchange.com/questions/9830/how-to-use-pomdp-optimal-policy<p>Suppose for a POMDP, we have calculated a deterministic optimal policy (i.e. mapping from belief states to action). Now, I want to apply it. So, I have an observation and I want to choose the optimal action. What should I do ? How shall I find belief state vector from the observation. </p>soshaMon, 16 Jun 2014 02:10:58 -0400http://www.or-exchange.com/questions/9830/how-to-use-pomdp-optimal-policystochastic-optimizationA basic doubt on policy iteration algorithmhttp://www.or-exchange.com/questions/9829/a-basic-doubt-on-policy-iteration-algorithm<p>consider the policy iteration algorithm for a finite state MDP. Suppose the initial policy is a stochastic policy. Now, can the optimal policy be deterministic after improvements ? Or, can we say that always the optimal policy will be a stochastic one ? Confused about this. Any ideas will be helpful. The reason I am asking this question is that in the absence of model i.e. when we need to need to use Monte Carlo methods then each of the improved policies must be a stochastic one to make sure action-value function estimates are near equal to the mean. </p>soshaSun, 15 Jun 2014 09:04:31 -0400http://www.or-exchange.com/questions/9829/a-basic-doubt-on-policy-iteration-algorithmstochastic-optimizationA basic doubt on belief MDP for a POMDPhttp://www.or-exchange.com/questions/9822/a-basic-doubt-on-belief-mdp-for-a-pomdp<p>The wikipedia page of POMDP <a href="http://en.wikipedia.org/wiki/Partially_observable_Markov_decision_process#Belief_MDP">http://en.wikipedia.org/wiki/Partially_observable_Markov_decision_process#Belief_MDP</a> says about belief MDP. My question is on its transition function. In order to be a MDP should not it be transition probability ? So, the transition probability function should be from the set \(B \times A \times B\) to \([0,1]\). From the wiki page we know how to update beliefs given the previous belief state, action and observation. As the observation set is finite the transition probability for a belief state will be distributed to some finite number of belief states according to the corresponding observation. Should not such "transition probability" (rather than "transition function") description is appropriate to describe belief MDP? Then value iteration, policy iteration will be just like general MDPs. Confused why wikipedia page writes like $that$.</p>
<p>BTW, how to write math formulas here. The usual style that works in math exchange/overflow is not working here.</p>soshaFri, 13 Jun 2014 09:28:42 -0400http://www.or-exchange.com/questions/9822/a-basic-doubt-on-belief-mdp-for-a-pomdpstochastic-optimizationReinforcement learning in partial observable domainshttp://www.or-exchange.com/questions/9810/reinforcement-learning-in-partial-observable-domains<p>I have googled some papers in this area. If someone has some information on this area, please share with me if possible.</p>soshaThu, 12 Jun 2014 08:31:24 -0400http://www.or-exchange.com/questions/9810/reinforcement-learning-in-partial-observable-domainsstochastic-optimizationA basic question on POMDPhttp://www.or-exchange.com/questions/9797/a-basic-question-on-pomdp<p>In order to specify a POMDP, we need observation probabilities given state and action. I don't understand the need for action here. Agent can't view the actual states but the tokens (probabilistic) emitted by it. What is the role of action here ? <br>
</p>soshaWed, 11 Jun 2014 20:29:07 -0400http://www.or-exchange.com/questions/9797/a-basic-question-on-pomdpstochastic-optimization