The start of 2011 marked the moment that “innovation” arrived as the buzzword on lips of everyone from the president of the Consumer Electronics Association to the president of the United States. Invoked in totemic tones in one speech after another, business leaders have stressed “innovation” (often in contrast to “regulation”) as the cure for the world’s economic and cultural ills.

But while innovation enthusiasm was rising among business and government leaders, it also became fashionable in certain quarters to fret that we’re in the midst of an enduring innovation drought. And if my conversations with venture capitalists and entrepreneurs are any indication, then even the cloud—supposedly the cutting edge of technological progress—is facing an uphill battle to innovate. The reason? A combination of easy money and cheap transistors (i.e., Moore’s Law). Even worse, these factors are also creating a kind of hidden barrier to cloud adoption that’s potentially larger than the standard “barrier” issues of security, reliability, regulatory compliance, and vendor lock-in.

Innovation and the talent shortage

In January of this year, right as the innovation-speak reached fever pitch, prolific blogger and economist Tyler Cowen launched the “innovation stagnation” conversation into the punditry mainstream with his best-selling Kindle Single, The Great Stagnation. In this weakly argued essay, which serves mainly to demonstrate Paul Krugman’s point about how little attention high-profile libertarians and other “freshwater” economist types pay to the opposing team’s work (I’m thinking specifically of Marxist economic historian Robert Brenner’s dense, data-rich oeuvre on innovation and “the long downturn”), Cowen begins with the observation that technological innovation has essentially flatlined since 1973. He then claims that America has eaten up all of the “low-hanging fruit”—like free land and a smart yet under-educated populace—that powered previous waves of innovation.

More recently, Cowen’s arch-libertarian fellow traveler Peter Thiel penned an essay in the National Review that also aimed to illuminate how and why the innovations of the past few decades have failed to measure up to those of the Cold War era and before. And now novelist Neil Stephenson has picked up the baton with an essay entitled “Innovation Starvation,” which, I kid you not, blames the problem in part on a dearth of good sci-fi.

All of these authors are focused on a perceived lack of fundamental, earth-shaking innovation of the Apollo 11, “one giant leap for mankind” variety. But for my part, I’m interested specifically in cloud computing, a field where change comes so quickly and where jobs are so specialized that it’s near impossible for one person to stay on top of everything important. The rhetoric around “cloud” might give you the impression that the term is practically synonymous with “innovation,” but in talking to those in the trenches, the reality is a bit different.

I recently had a sit-down chat with Ping Li, a venture capitalist at Accel Partners who does investments across the layers of the cloud stack. Over the course of our conversation, Ping expressed frustration about the difficulty of hiring and maintaining talent right now. “It’s this heated funding environment,” he said, going on to explain that all of the money sloshing around in the Valley had created a market for talent that’s just as tight as it was during the dotcom boom. What’s worse, he explained, is that the talent shortage is stifling fundamental innovation in the cloud space.

To do really fundamental engineering innovation of the kind that was done, say, in the early days of Google and VMware, you need to hire and retain teams of talented engineers. But in today’s go-go funding environment, top engineers are being enticed with truckloads of money to break off and form two- and three-person startups. This phenomenon, explains Li, is why “many of the really big innovations happen in less frothy times.” He did go on to clarify that “some great companies do get created in these times (like Amazon in the last bubble). It’s just harder given talent shortage.”

Li’s comments are by no means the first I’ve heard in this vein. I recently talked to another well-known Valley entrepreneur who told me that his startup had poached top talent from a rival, only to see that talent leave and form a new VC-backed startup. Then there’s the email I got a few months ago from a friend of mine and product manager at Apple, who was wondering if I knew any cloud computing hackers that they could hire. When we get to the point where Apple product managers on the client side are reaching out to their personal networks in search of cloud coding talent for the world’s largest tech company, you know it’s bad out there.

I’m facing this in my own project, a small book publishing startup where I moonlight as Chief Product Officer. We’re written in Ruby and hosted on Heroku, a pair of technical decisions that we made so that we could easily and painlessly scale, and so that we wouldn’t have to waste resources on any sort of sysadmin work. Back in the depth of the last downturn, we were fortunate to have found a team of contract developers who are very talented and who are now therefore very, very busy.

This is par for the course. We’ve heard other startups complain that all of the big Ruby shops are taking only large jobs right now, because they’re so maxed out that there’s no bandwidth for small startup projects or overflow. So if our team gets run over by a bus on the way to an off-site, or vacuumed up by a VC with a giant bankroll, then all of the cloud-based redundancy and scalability in the world won’t get new features pushed out on our platform.

Talent as a hidden barrier to cloud adoption

The tight state of the current talent market has another, second-order effect on cloud innovation that goes beyond the team size issue that Ping Li points out. The talent shortage is also a factor in the slow pace of cloud adoption in the enterprise, or, at least, that’s one way of reading the results of Symantec’s newly released State of the Cloud 2011 survey. The survey asked IT departments about their staff’s readiness to make the leap into the cloud, and here’s what it found:

About half of the organizations surveyed said their IT staff is not ready for the move to cloud. While a handful (between 15 and 18 percent) rated their staff as extremely prepared, roughly half rated their IT staff as less than somewhat prepared. Part of the reason for this hesitancy is their staff’s lack of experience. Less than 1 in 4 computer staffers have cloud experience. As discussed earlier, the adoption of cloud changes how IT works, so experience is absolutely crucial for IT.

The survey goes on to find that despite the excitement around cloud, most companies’ cloud initiatives are stalled in the discussion/trial phase.

Moore’s Law: too much of a good thing?

I said at the outset that there are two factors working against innovation in the cloud, but so far I’ve only talked about the first: easy money. The second factor is Moore’s Law. Advances in microprocessor technology are producing far more hardware than programmers can collectively or individually program.

On the enterprise side, the move to the cloud is driven partly by uncertain economic conditions and is based on an infrastructure layer that offers more performance per dollar with every upgrade cycle. So as economic fear continues to reign and commodity datacenter hardware gets cheaper in cost-per-MIPS, the pull of the cloud will get even stronger. And more dollars flowing into cloud hardware means that the demand will grow for cloud-savvy talent that can put the hardware to productive use.

Not only does the profusion of cheap computing power create an aggregate demand for more cloud programmers, but it also taxes individual programmers like never before. Coding, deploying, and maintaining highly parallel cloud apps is hard. It’s notoriously challenging to architect for parallelism at the design level, because it’s hard for mere mortals to reason about parallelized tasks, especially if they’re non-deterministic (and they often are). It’s also hard to meaningfully test software on today’s large, massively parallel clouds; the logistics are a huge challenge.

Are taxes the answer?

Clayton Christensen, author of The Innovator’s Dilemma, touches on some of thsee issues in a must-read interview on GigaOM. Christensen fingers the “hot money” that flows into sectors looking for a quick return as one of the factors slowing down innovation. He advocates eliminating capital gains taxes on investments of eight years or more, so that VCs will be incentivized to invest for the long term, thereby increasing the quality of innovation by shifting the focus away from things that can be done quickly.

Regardless of what you think of taxes—more of them or less of them—as an answer to our innovation dilemma, it seems clear that the current crisis in the cloud is the product of too many dollars and transistors chasing too few coders and sysadmins. It will take a while for the latter to catch up with the former… unless, of course, another major downturn strikes. It seems ironic that less money could equal more innovation, but it wouldn’t be the first time that a wave of downsizing and tight money boosted productivity.