The perils of tweaking

how to use macrodata to set parameters in complex simulation models

Brian Epstein

Patrick Forber

pp. 203-218

in: Christopher Pincock, Marion Vorms (eds), Models and simulations 4, Synthese 190 (2), 2013.

Abstract

When can macroscopic data about a system be used to set parameters in a microfoundational simulation? We examine the epistemic viability of tweaking parameter values to generate a better fit between the outcome of a simulation and the available observational data. We restrict our focus to microfoundational simulations—those simulations that attempt to replicate the macrobehavior of a target system by modeling interactions between microentities. We argue that tweaking can be effective but that there are two central risks. First, tweaking risks overfitting the simulation to the data and thus compromising predictive accuracy; and second, it risks compromising the microfoundationality of the simulation. We evaluate standard responses to tweaking and propose strategies to guard against these risks.