Trial and error is a modus operandi in science, and has become a popular way of doing business as well. But it hasn't made enough inroads into public policy, leaving a lot of potential for societal gains untapped. In Uncontrolled: The Surprising Payoff of Trial-and-Error for Business, Politics, and Society, Manzi seeks to rectify this by discussing the challenges that the soft sciences present and how they can be somewhat overcome.
Manzi starts out by going through the history of experimentation, and how it led to various gains in different fields. Unfortunately, the inter-field growth of experimentation as a core operating methodology kind of stalled out, as complex societies don't lend themselves to being boiled down to a few equations.
In many hard sciences, most variables can be controlled, leaving no doubt as to how an independent variable affects the environment. But in social sciences, most variables cannot be controlled or even accounted for, requiring the use of statistics as well as the recognition that the statistics chosen may not capture the important factors.
But it is still important to experiment. Manzi demonstrates this by doing a take-down of some popular theories. For example, Stephen Levitt's thesis that legalized abortion was a large contributor to the drop in crime rates was put under the microscope. By reasonably changing just a couple of assumptions, Levitt's thesis no longer becomes supported by his original arguments. Using observational data only, we really don't know if the assumptions were right, the factors capture what's needed, and therefore whether the conclusions are correct. Even if the conclusions happen to be correct, we have no idea whether they hold true *outside* of the environment in which the initial observations were made.
Of course, controlled experiments are impossible to run in societies, but Manzi has some suggestions which can help us move forward by trial and error. For example, by decentralizing policy decisions (from federal to state, state to local etc.) we can get a wider variety of attempts, and then learn and observe which worked out and which didn't.
If this type of discussion interests you, you will enjoy Uncontrolled: The Surprising Payoff of Trial-and-Error for Business, Politics, and Society.