Thursday, January 15, 2009

2009 Prediction - Rise and Fall of the Private Cloud

#6 in our series of 2009 predictions

2008 Recap

2008 saw massive hype around the concept of a “private cloud,” roughly defined as a adopting the technology and practices from public cloud providers for a single company behind the firewall. “Private clouds are the future of corporate IT” declared Gartner. “Private Clouds Take Shape,” gushed InformationWeek, citing the funding of companies like Elastra and Parascale. “Get off my cloud” said eWeek, questioning the security of public cloud environments compared to private clouds.

2009 Prediction

Here’s the rub: Private clouds are just an expensive data center with a fancy name. We predict that 2009 will represent the rise and fall of this over-hyped concept. Of course, virtualization, service-oriented architectures, and open standards are all great things for every company operating a data center to consider. But all this talk about “private clouds” is a distraction from the real news: the vast majority of companies shouldn’t need to worry about operating any sort of data center anymore, cloud-like or not.

The idea that somehow companies can use “private cloud” technology to offer their employees web services similar to Google, Amazon, or salesforce.com will lead to massive disappointment. Here’s why:
  • Private clouds are sub-scale: There’s a reason why most innovative cloud computing providers have their roots in powering consumer web technology—that’s where the numbers are. Very few corporate data centers will see anything close to the type of volume seen by these vendors. And volume drives cost—the world has yet to see a truly “at scale” data center.
  • You can’t teach an old dog new tricks: What do you get when you move legacy applications as-is to a new and improved data center? Marginal improvements on your legacy applications. There’s only so much you can achieve without truly re-platforming your applications to a cloud infrastructure… you can’t teach an old dog new tricks. Now that’s not entirely fair…. You can certainly teach an old dog to be better behaved. But it’s still an old dog.
  • On-premise does not equal secure: the biggest driver towards private clouds has been fear, uncertainty, and doubt about security. For many, it just feels more secure to have your data in a data center that you control. But is it? Unless your company spends more money and energy thinking about security than Amazon, Google, and Salesforce, the answer is probably “no.” (Read Craig Balding walk through “7 Technical Security Benefits of Cloud Computing”)
  • There’s no secret sauce: There’s no simple set of tricks that an operator of a data center can borrow from Amazon or Google. These companies make their living operating the world’s largest data centers. They are constantly optimizing how they operate based on real-time performance feedback from millions of transactions. (check out this presentation from Jeff Barr and Peter Coffee at the Architecture and Integration Summit). Can other operators of data centers learn something from this experience? Of course. But the rate of innovation will never be the same—private data centers will always be many, many steps behind the cloud.

There’s also something very suspicious in all this discussion of private clouds…. private clouds are advocated mainly by companies who make their money from selling or operating data centers, and risk losing their shirts as real cloud computing drives more and more computing onto shared infrastructure. I understand why these companies are reluctant to embrace true cloud computing: Imagine being the junior partner in IBM Global Services pitching a client to develop an application on Amazon, Google, or Salesforce. Not only are you taking money out of the pocket of your colleagues in hardware and software….. you are also taking money out of the pocket of your colleagues in professional services, since integration and app development are so much easier using on-demand platforms.

That’s not to say that there’s no place for the technology behind private clouds. In certain cases where it simply isn’t an option to utilize a public cloud, this technology can have a significant impact. But those use cases are few and far between, and the benefits to be achieved are insignificant relative to the benefits of moving to a public cloud. Here’s who should be thinking about private clouds:
  • Cloud Providers: This is an easy one… companies that plan on being in the business of providing cloud computing capabilities to others need to think about how to effective provide their own cloud. But we’d argue that very few companies actually need to be in this business (e.g., we believe most on-demand BI vendors should be running on public cloud infrastructure).
  • Highly regulated industries: Government regulation will always lag behind commercial application of technology. There will inevitably be instances where nervous politicians or policy makers write up requirements that can only be met through a private cloud.
  • Companies in the process of moving to a public cloud: Of course, no company of any significant size can move its IT infrastructure to the cloud all at once. In fact, Appirio specializes in helping companies figure out what the right first step is away from their on-premise infrastructure. For the IT infrastructure that hasn’t yet moved, it definitely makes sense to think about how to use “private cloud” technology. But that means the private cloud is a temporary stop-gap, not the “future of enterprise IT.”
Implications for customers.
Of course any customer with a data center should be thinking about how to use the technologies behind “private clouds” to improve their efficiency. But this should be a minor element of your long-term IT strategy. The most important thing any IT department can do in 2009 is chart out a thoughtful plan to migrate significant portions of your IT infrastructure to the public cloud. Don’t let “private clouds” be a distraction from that goal.

10 comments:

  1. As I mentioned in my "Are internal clouds bogus?" post, I don't agree that the number of orgs that could use an internal cloud is "few and far between." Our take is that if you have hundreds of servers you are probably keeping at least some of them in inefficient silos, tied to specific applications and that sharing resources for those apps in a "cloudy" way could be helpful to your cost structure -- and your ability to match your IT to what the business really, actually wants. But you are right on about one thing: the big vendors are not going to be too pleased with the economics that cloud computing will cause. But internal cloud computing could be bad for them, too: orgs will be able to use what they have already more efficiently. That blog post I referenced: http://datacenterdialog.blogspot.com/2009/01/are-internal-clouds-bogus.html

    ReplyDelete
  2. Hi Jay, thanks for the comment and a great post. Agree that organizations with hundreds of servers need help managing them more efficiently. But is this the "future of IT"? No way. That's our main point.

    ReplyDelete
  3. Gartner thinks internal clouds will be the future of IT ops. Of course, they've been wrong before. :^) My notes from Tom Bittman's keynote on this: http://datacenterdialog.blogspot.com/2008/12/from-gartners-data-center-conf-tom.html

    ReplyDelete
  4. The real reason why you're right has nothing to do with any part of your well written and informative post. Its all about economics. For an enterprise, it doesn't add up to have a mountain of unused capacity (i.e. vm's in a cloud just waiting to be use), lying around. While running datacenters isn't even a core competency of many businesses, its the fact that to be elastic and respond to demand, you just gotta have the excess capacity available. There's a good reason why Ely Lilly lets their scientists fire up EC2 instances to work on test cases, where as they'd be hesitant with internal resources. Moreover, when you need more power, you just buy it from your energy company--you don't build generators for the purpose of providing your own energy cloud.

    Its incumbent on the security professionals of the world to hold the enterprise's hand on the flight up to the cloud.

    ReplyDelete
  5. While I agree about some of your points, I don't know if Amazon will run a data centre the way you want it. You mention the constant optimisation by these companies but Google is optimising their data centres for their search application, and Amazon is optimising for their shop.

    In both of these cases, running an enterprise application on their computers will be a marginal usage case for them, less than 0.001% of the total usage of their centres, even if you're a pretty huge operation, so they'll spend no time at all optimising for your own usage.

    ReplyDelete
  6. Hi Ewan, thanks for the post. You bring up a great point, and the reason why we think the market will require multiple IT clouds will be required. Different clouds will be optimized for different applications (e.g., Force.com for business applications), and different clouds will allow optimization at different levels of the stack.

    Quick point on Amazon EC2-- at last week's TechCrunch event, Amazon shared that Amazon.com represents only a minority the traffic on EC2 and S3--- their stack definitely isn't just optimized for shopping.

    ReplyDelete
  7. Cloud has great potential, but it's not going to eliminate the need for datacenters (especially not overnight). Private clouds (or datacenters that are virtualized and therefore can share some properties of a cloud) should actually help the whole cloud movement.

    ReplyDelete
  8. Stuart, you make a great point. Private clouds are a big step forward from the current, inflexible and inefficient architecture of private datacenters and start to demonstrate the benefits of moving to the cloud. But the true benefits of massive scalability and paying for what you use are only realized if you're sharing infrastructure on a much broader scale.

    ReplyDelete
  9. Agreed 100%. Repeat after me: "the evolution of virtualisation is NOT cloud computing".

    If we get this wrong the trough of disillusionment will be longer and deeper as customers realise that the hype of "I can't believe it's not cloud" doesn't live up to the promise of RealCloud™.

    I guess then RealCloud™ will become "Cloud 2.0", and what's a meme without a 2.0? :P

    Sam

    ReplyDelete
  10. Perhaps you could link to Craig's original blog post at http://cloudsecurity.org/2008/07/21/assessing-the-security-benefits-of-cloud-computing/ rather than its evil twin at Sys-con

    ReplyDelete