# Techniques for when the objective function is the output of a simulation

 4 3 What techniques are appropriate when the objective function cannot be expressed as a math program, but is the output of a simulation? Obviously, standard math programming algorithms can't be used. My background inclines me towards search heuristics, such as Genetic Algorithms and Tabu Search. These methods still work well when the objective function is a "black box". Are there any other techniques I should be aware of? Are there any software packages that solve this sort of problem (and how do they do it)? asked 27 Oct '10, 03:54 DC Woods ♦ 4.1k●2●25●46 accept rate: 5%

 2 You may try using some derivative-free solvers: NMSMAX, APPSPACK, NEWUOA, DFO ... http://www.mcs.anl.gov/~more/dfo/shootout.html answered 27 Oct '10, 14:33 anonymous 691●3●11 accept rate: 8%
 2 I will second the answer about DOE. I do simulation analysis pretty much full time, and we often encounter 5-20 independent variables. We usually perform a small simple experiment (a fractional 2^k, or plackett-burman etc) with fewer replications than would be needed for a precise CI on the response. subsequent anova identifies which decision variables (main effects) wield measurable influence over the response, and in most cases we reduce the size of the decision space significantly. A follow on experiment on the variables of interest may be a more complicated higher resolution design, with more replications per run. If your simulation is set up for random number synchronization i suggest implementing common random numbers using a baseline case as a control, and measure the difference between experimental runs and that control. In most cases this will drastically reduce the number of replications necessary for a desired precision and help keep your run times manageable. I've used Opt-Quest that comes with ARENA simulation software and my experience is that for more than just a few decision variables the amount of time required to approach optimality becomes prohibitive (but that experience is pretty limited--it may have been our particular problem). Best, Jon answered 29 Dec '10, 16:47 Jon Davis 156●4 accept rate: 16%
 1 I don't know how big of a problem you got but you might wanna check out Neural Networks. Since wikipedia has a better definition on them I will just quote.. "The utility of artificial neural network models lies in the fact that they can be used to infer a function from observations." You can still use your heuristics in the training of the net. However AFAIK you have to write the whole network algorithm yourself, but it's relatively easy. answered 27 Oct '10, 05:57 Buxley 564●1●6●14 accept rate: 9% Neural networks are indeed a good way of estimating a function from observations. Unfortunately they aren't suitable in this case. I don't HAVE observations, and the objective function isn't a "function" I need to estimate (in the sense of a combination of inputs), it's the output of a real-world process that I can simulate. (27 Oct '10, 07:33) DC Woods ♦
 1 I think the problem(blakbox) can be seen as a regression problem. Given some inputs we receive an output (in this case the output of the simulation) and there is no known equation between inputs and the output and we try to estimate the relation between inputs and the output. For this purpose, the outputs of a series of simulations can be considered as training and validation data. Then there are different methods and models to choose and neural network is one of them. answered 27 Oct '10, 09:03 Arman 329●2●7●14 accept rate: 0%
 1 I read an article about wing shape optimization for aircraft. The authors used Genetic Algorithm to determine the optimal parameter set. In an evaluation for each parameter set, aerodynamic simulation was executed. They used a lot of blade computers to execute the simulation.... answered 27 Oct '10, 12:12 yonezat 41●1 accept rate: 0% Yes, this is the type of problem, where even a tiny change in one parameter of the solution can result in the the result becoming very different. (27 Oct '10, 21:00) DC Woods ♦ I've also read about GAs and Genetic Programming being used to "reinvent" the dyson vacuum cleaner, and some other amazing things. (27 Oct '10, 21:01) DC Woods ♦
 1 You can find a blackbox optimization package here : http://www.gerad.ca/nomad/Project/Home.html answered 28 Oct '10, 14:54 Pascal 11●1 accept rate: 0%
 1 If your model is a discrete event simulation model, most of the leading discrete event simulation packages have optimization engine add-ons that you could use to optimize using the simulation outputs and inputs as your objective function and controls. For example, OptQuest for Arena/Promodel or SimRunner. answered 28 Oct '10, 17:46 dcope 31●1 accept rate: 0%
 1 Have you checked out How to Solve It: Modern Heuristics or a similar book? How to Solve It is a very good book in my experience, but perhaps a better one has been published since I last looked. Without knowing anything about the characteristics/nature of your problem it's hard to say which methodology makes the most sense. There are quite a few approaches though (it only takes a quick look at wikipedia for "Metaheuristic", "Particle Swarm Optimization", "Hill Climber" and following a few of the numerous links to appreciate how varied the options are). I've used simulated annealing (and hill climbing on simple cases) successfully in a past project. But really, it depends on the character of your problem. From what you've written in the comments ("a tiny change in one parameter of the solution can result in the the result becoming very different") I think an issue that you'll need to address is how define neighbors of your current solution in such a way that a "neighbor" doesn't have a dramatically different output. This may seem obvious, but if "neighbors" have wildly different results, maybe your simulation is somewhat volatile; if so, I would suggest increasing trial count and averaging. answered 06 Jan '11, 14:00 danhs 11●1 accept rate: 0%
 1 As part of Crystal Ball software suite, we routinely solve simulation optimization problems. The definition we use is broader than what you mention: we can have some of the constraints coming from simulation as well. We use OptQuest to solve the optimization problem (Crystal Ball provides the simulation results within the optimization framework), which internally uses tabu search for solving. So, again, a meta-heuristics solver is getting used. answered 26 Aug '11, 13:15 Samik R. 1.2k●1●9●20 accept rate: 2%
 toggle preview community wiki

By Email:

Markdown Basics

• *italic* or _italic_
• **bold** or __bold__
• image?![alt text](/path/img.jpg "Title")
• numbered list: 1. Foo 2. Bar
• to add a line break simply add two spaces to where you would like the new line to be.
• basic HTML tags are also supported

Tags: