For an Amazon Web Services Clone
Wednesday, January 17th 2007 | Ismael Ghalimi
In my Inferences for ‘07, I suggested that open-source, plug-compatible alternatives to Amazon’s EC2, S3, SQS will be released sometime this year. Here are a set of reasons why this would make sense, and why everybody would benefit from such a thing, including Amazon itself.
There are two primary reasons why I like Amazon Web Services: One, they define very simple interfaces for addressing very common needs you have when developing any kind of Web-based application. Two, they promote a no non-sense pricing model that makes them a viable option for most projects. Thing is, I am not the only one thinking that way, and I am hearing more and more organizations, small and large, starting to adopt Amazon’s services, which raises a very important question: can we rely on them?
As any utlity, Amazon Web Services are subject to failure, and recent outages experienced by developers are a painful reminder than when you put all your eggs in the same basket, and the basket falls on the ground, you better like your eggs scrambled. Which is why you should use multiple baskets.
Imagine for a moment that an open-source project would be created with the goal of developing all the software necessary for deploying clones of Amazon’s most popular Web services, including EC2, S3, SQS. Imagine that you could deploy this open-source software onto your own servers. Imagine that more elaborate versions would be developed on top of some kind of grid operating system — like 3TERA, allowing any ISP or Web hosting provider to provide plug-compatible Amazon-like services by using unused bandwidth, processing, and storage capacity. Finally, imagine that such an open-source project would also develop the client-side software you need to use Amazon’s Web services in combination with any other clone in order to get failover capabilities, without having to learn any new API. Do you think such a thing would be of value, to you or others?
I recently asked this very question to a group of five IT executives working for one of the ten largest companies in the world, and the answer was a resounding ‘yes’. I must assume that you could find many other companies, small and large, that would share the same opinion.
If such a thing existed, you would be able to use Amazon’s low cost services for your daily operations, and the services offered by more expensive providers whenever Amazon would go down, for some limited period of time. You would also be able to run your own version of Amazon’s services within your own data center as a backup, giving you the assurance that your data and your services remain available, even in the event of a catastrophic failure, or a decision to increase prices beyond reason.
Today, popular Web sites and mission-critical applications served over the Web rely on redundant connections to the Internet that are provided by multiple service providers, as a way to remove any single point of failure. Time has come to apply the same approach a level up in the stack, and I could not think of a better starting point than Amazon Web Services for it.
Initially, Amazon might see such a project as a competitive threat, but over time, it should realize that most developers will still use its original services as primary ones, for they are most likely to provide the highest level of availability, at the lowest price point.
Such an open-source project does not exist today, but I bet it will very soon.
Entry filed under: Office 2.0, Open Source, SOA
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Ismael,
It is very interesting that you come to the conclusion that no single point of failure (i.e. Non-Stop) is important. Grid computing, augmented by SOA, is a solution that truly satisfies the original requirements of the APARNET (now Internet). Packet-switching networks provide alternative paths to send data over the network without a fixed route. Workloads can be distributed among computers based on available capacities, regardless of models or operating systems, on the network.
I believe that it makes more sense to use one’s own computing facilities for the normal operations, and subscribe to Amazon-like services as a remote disaster recovery site. This provides business continuity if the primary site is down for a long duration, or cannot be recovered other than by building a new one from scratch. In essence, this is a good alternative, and cheaper than subscribing to hot or even cold backup sites.
Your vision will work when every organization, regardless of size, goes for 100% outsourcing. Nevertheless, many organizations regret that they lose the institutional knowledge after they outsource more than they should! I know of 3 clients that outsourced some of their production work, not just IT, in the early- and mid-1980s. They brought them back in-house when error rates became unacceptably high, or costs escalated exponentially after the intial contract expired!
Outsourcing is not new, it has been around for more than 20 years. Software as a Service (SaaS) is not new either, it is equivalent to the time-sharing service bureaus that we had back in the 60’s and 70’s, when hardware and software were expensive. It looks like we are recycling the same business models every now and then, with new names and technologies.
Cheers!
-Francis
Francis,
I agree with you, the pendulum is swinging back…
Best regards
-Ismael
The nice thing about outsourcing and sharing technology resources is that if Amazon, Salesforce.com, StrikeIron, or any of these SaaS providers go down, immediately hundreds if not thousands of users will be all over it (these users effectively become real-time system monitors), so the incentive to not go down is incredibly high, and if there is an outage, notification and acknowledgement is instantaneous, and a team of specialists immediately deployed. Over the long haul this will prove to optimize uptime much better than any one company could do on its own, without incredible resource expenditure.
In other words, it’s nice to have your neighbors call the phone company and get the problem resolved if your whole street has lost service, probably before you even notice. The open-source concept is interesting of course, but whose throat do you choke, and who would kick into action should shared resources be down?
Trackback this post | Subscribe to the comments via RSS Feed
Leave a Comment