Joe Nocera had an excellent column in the Times on the pseudo-scandal of Solyndra, the failed solar panel manufacturer that received a $535 million federally guaranteed loan. He notes that, while the Obama Administration may have fast-tracked the deal for political reasons, it’s counterproductive to claim a scandal just because an innovative young company has failed, for many of these companies are bound to fail. Here’s Nocera:

If we could just stop playing gotcha for a second, we might realize that federal loan programs — especially loans for innovative energy technologies — virtually require the government to take risks the private sector won’t take. Indeed, risk-taking is what these programs are all about. Sometimes, the risks pay off. Other times, they don’t. It’s not a taxpayer ripoff if you don’t bat 1.000; on the contrary, a zero failure rate likely means that the program is too risk-averse.

If anything, government investment in science and technology is too afraid of risk, too unwilling to fund innovative projects that can’t get private capital. In fact, there’s good evidence that scientific funding programs with a bigger appetite for risk (and thus a higher tolerance for failure) are also better at producing major breakthroughs.

One way to illustrate the importance of encouraging risk is to compare the research strategies of the National Institutes of Health (NIH) – the largest funder of biomedical science in the world — and the Howard Hughes Medical Institute (HHMI), a non-profit set up to “push the boundaries of knowledge.” The NIH evaluates grant proposals in an exceedingly rational manner. A team of experts analyzes and scores each proposal, ensuring that the project is scientifically sound and is supported by plenty of preliminary evidence. Their explicit goal is to not waste taxpayer money — nobody wants to fund a failure.

HHMI, in contrast, is known for supporting avant-garde projects. In fact, it explicitly encourages researchers to “take risks, explore unproven avenues and embrace the unknown — even if it means uncertainty or the chance of failure.” HHMI does this by focusing on individual scientists, not particular experiments. (Instead of requesting a detailed proposal of future research, HHMI asks for an example of past research.) The assumption is that a creative scientist should be able to pursue ideas without having to justify them to a panel of experts. Like Solyndra, these researchers are full of potential. But potential is not a promise.

A few years ago, a team of economists at MIT and UCSD analyzed the data from NIH and HHMI funded labs to see which funding strategy was more effective. The economists tried to control for every possible variable, such as outside scholarships and the quality of graduate students. Then, they compared the output of NIH researchers to HHMI investigators with similar track records.

The data was clear: In every biomedical field, the risky HHMI grants were generating the most important, innovative and influential research. Although HHMI researchers had similar qualifications to their NIH counterparts when they first applied for funding, they went on to produce twice as many highly cited research articles and win six times as many awards. They also introduced more new “keywords” into the scientific lexicon, which is a marker of highly original work.

The bad news, of course, is that all this creativity comes with a cost. This is why, according to the economists, the HHMI researchers also produced 35 percent more research papers that went largely uncited. (These papers were abject failures, the Solyndra of peer-review.) The moral is that these scientists weren’t producing better research because they were smarter or more creative or had more money. Instead, they had more success because they were more willing to fail.

Bob Dylan captured the paradox perfectly: “She knows there’s no success like failure/and that failure’s no success at all.” Failure will never be fun. But that doesn’t mean it’s a scandal. We talk the talk about restoring American innovation. But the only way to get major innovation is to make bets that might not pay off.

PS. If you’re interested in the subject, I highly recommend Tim Harford’s Adapt.

Image: Kincuri/Flickr