Monday, November 29, 2010

Examining Compute Commodities

Over the last several weeks I've been doing quite a bit of studying of the commodities markets, mostly around the emergence of energy trading in the early 1990's. During my research a few things have become pretty clear to me. I thought I'd outline a few of the more interesting observations.

In looking at corollaries in treating compute resources as a commodity, the closest is probably that of energy creation and power plant financing. In the energy world, commodities trading desk are used to protect power companies from dramatic price shifts, using a so called "Hedge". These hedges are done in a number of very interesting ways. First of all, most of the major banks act as both a provider of capital for buyers and sellers of power plant assets and the companies themselves. The majority of the major finance players in the energy world also have commodities desks, allowing them to operate in the commodities market as well as in the more traditional loan / financing businesses. They can offer issuers (power providers) access to commodity markets where hedges can be created to protect an energy company from dramatic changes in prices of coal, natural gas or oil. Many utilities that provide electricity, for example, use coal to fire their plants are protected from price swings and more importantly the banks who are trading these commodities have greater influence and protection from these price spikes by sitting on both sides of the deal. In essence dovetailing leverage finance with commodities. In return, these banks are granted the ability to not only finance the development of the power plants, they also buy the future contracts on the energy itself which in turn gives the energy providers a guarantee of future revenue which then can be used as collateral for the development of their various energy assets and reduces the risk for all involved.

Let me explain this concept using the data center space as an example. Imagine being able to build a data center with a guarantee that a portion of your capacity will be bought at a certain price for an extended period of time before you even built your data center? This in a nutshell is the driver for the commoditization of computing resources. It has less to do with the actual computing resources so much as the ability to provide enhanced insight into future cash flows while reducing the risk surrounding the un-certainty of future utilization levels. Data centers are the new power plants.

What SpotCloud does is provide a general structured framework for the creation of a Compute Spot Market - an essential requirement before you can potentially have the ability to buy future capacity in bulk. Now imagine for a moment a SpotCloud Price Index (SCPi) where buyer are given a normalized average (a weighted average) of prices for a given class of compute services (based on a key hardware metric) in a given region, during a given interval of time. This is where things start to get interesting, an index is a requirement for any futures market. Without this critical statistic, it would be very difficult to compare how compute prices, taken as a whole, differ between time periods or geographical locations. With this data, hedges and arbitrage are now possible regardless of whether the quality of compute resources differs because you can use the average with traditional finance methodologies allowing for differences among providers answering the problem of not all compute resources are actually the same. Money itself being the great equalizer.

I readily admit that selling any debt in the current market is tough, but given the current trends in computing and the ever increasing need for computing resources around the globe, it's a fair bet to say that the demand for these types of resources will continue for the foreseeable future.

Sunday, November 14, 2010

Pork Bellies, Bandwidth and Cloud Computing

-- Update ---
I'm happy to announce that we've launched SpotCloud, the first Cloud Capacity Clearinghouse and Marketplace. Check it out at

In May 1999 Enron announced to the world that it was creating a new market for trading Bandwidth. A wired article from 2001 noted that it seemed to many that the then energy giant had found a new pot of virtual gold. Enron and a broader group of their experienced traders believed it was only a matter of time before bandwidth (as well as other virtual resources) would be bought and sold in much the same way that commodities markets trade everything from petroleum to pork bellies. Now, more then 10 years later this transition has yet to occur. In this post I will examine why the idea of trading bandwidth never took off and see if today might be the ideal time to try again.

Looking back at the previous attempts to create virtual commodities exchanges including Enron's failed attempt, it now appears that it was indeed a great opportunity, but just about a decade too early. In the case of Enron, they had a right vision, but suffered from the now obvious fact that it was born from an overwhelming greed. In other words, the right idea but the wrong people at the wrong time.

Today with the emergence of cloud computing looking at these past failures such as the failed bandwidth markets and as well as the successes of the energy markets of 1990's may represent a case study in how to we might go about creating a successful commodity compute marketplace.

One of the first problems in getting bandwidth trading off the ground was timing. The bursting of the dotcom bubble meant that there was a significant disconnect between an over supply of bandwidth versus the demand for it. Basically there weren't enough companies who wanted to buy and too many selling. Making the market go in one direction, down. This discouraged both buys and sellers from getting involved. The key to an active market and ecosystem is growth.

Secondly, as the wired article points out, the telecom firms that owned the fiber optic networks didn't like the idea of selling their services as a commodity. Some made the case that "not all networks perform equally well." Basically there were no measurement standards and therefore no easy way to determine the good from the bad. In addition, most telecom firms preferred to negotiate prices with customers, rather than be stuck with a one-size-fits-all pricing scheme. In a sense they would rather lose on the excess capacity and make up the difference by charging more for the capacity that was actually used. In contrast, the benefit to a commodity style approach, you may charge less overall but make more money because you have a higher utilization of your resources (volume).

Another major problem was at that time the adoption of broadband was at its infancy. Most Internet users in 1999 we're still using dial up connections. Compounding things was other than a few notable exceptions (Napster) the majority of web applications were static and light weight. Mobile apps, streaming media, social web applications, realtime web and cloud computing (Internet centric computing) had yet to be widely accepted. Fast forward to today and these applications have become the key drivers to a recent explosion of rich user generated content and the ever increasing need for realtime compute capacity to process it all.

Thanks in part to the increasing popularity of cloud computing, the idea of just-in-time compute capacity has helped lower some of barriers that limited the previous bandwidth markets from flourishing. For many the concept of distributed batch processing and compute elasticity have become critical parts of modern business IT strategies. These kind of flexible and elastic compute usage models are ideally suited to that of a spot market for commodity compute capacity (provided via a method that is quoted for immediate (spot) settlement for both payment and delivery. Also the announcement last month that Amazon Web Services would start offering excess EC2 capacity using a spot market approach has also helped legitimized the concept. The notion of selling your excess compute capacity now has a poster child (AWS), this may lead to increased acceptance of selling excess compute resources using a commodities approach. This is in much the same way that Amazon EC2 has encouraged companies to use cloud like strategies within their internal systems (private clouds). In a very real way, AWS is blazing a path for the broader industry.

I see tremendous opportunities for the trading of excess Cloud Computing resources or compute capacity and believe the most viable market example may be that of the energy marketplace. The energy market is similar to bandwidth and compute capacity in that the commodities are variable, transient and don't store well. The concept of selling excess capacity in cloud centric data centers may also make sense in that cloud providers must have significant additional capacity on hand just in case of demand spikes.

As I've said before, unused compute capacity = lost revenue. It's better to sell your excess then to have it disappear. For a lot of larger players such as Telecoms and large content providers this means un-utilized compute capacity is making you nothing. The notion of a public spot market may help address this problem.

The great example for a compute centric market may be based on that of the electricity wholesale markets. Like compute capacity, electricity is difficult to store because of it's transient nature. It needs to be available on demand, and unpredictable demand spikes may occur. Using the energy trading market as a model provides an existing proven context that may translate well into compute centric environments, not mention there are wide variety of trading platforms already built that may be easily modified to address the needs of a compute exchange market.

One of the more common energy trading models uses a automated central scheduler to balance supply and demand and calculate the market price. Another model is that of conducting auctions in various time scales, i.e. auctions for yearly and daily provision of power, with additional spot market that resolves the need for accommodating short-term demand spikes.

Before a widely accepted commodity compute trading market may form and begin trading, governments may also need to provide a common regulatory framework as well as standards and liability controls. Otherwise the market will be doomed to serve as a novelty or worst yet, limited to academic use only.

So what's next? First a trading organization must form, preferably in a transparent not for profit context, so to help avoid future Enron type scenarios. I'd also say the capital to develop such a trading platform, the will of the industry to help make this happen and some standard processes for the measurement of the cloud capacity itself. So will this happen? Certainly, but question of when is still up for debate.

Wednesday, November 10, 2010

SpotCloud Update (Free ECP SpotCloud Edition & Webinar)

It's been a crazy week at Enomaly after last week’s SpotCloud announcement. We'd like to take a brief moment to update you on some new and exciting opportunities that have emerged out of our discussions around the SpotCloud Marketplace (

Buyer / Seller Traction
Signups for both the buy and sell side for SpotCloud have been very strong with hundreds registering for the service. One of the more interesting statistics is the ratio of buyers to sellers is tracking at 5:1 for buyers. This shows there’s a lot of buy-side demand for the service. For those of you who have already registered, we are selectively adding new buyers and sellers while we work through the beta phase, so hold tight, we haven't forgotten about you. If you haven't registered yet, go ahead and add yourself today.  

Announcing the Free Enomaly ECP SpotCloud Edition
We're happy to announce that we are now offering a Free Enomaly ECP SpotCloud Edition (Shipping next week). This is a feature-limited version of the Enomaly Elastic Computing Platform specifically tailored for cloud providers, as well as private and public data centers looking to sell excess capacity on the SpotCloud market. By simply installing ECP SpotCloud edition on a few spare servers, providers are able to easily participate in the marketplace. No public cloud or payment systems are required. Just a few servers and the Free ECP SpotCloud IaaS software and you're ready to start making money. To get access to the ECP SpotCloud Beta, simply register as a seller at and check the box for the ECP SpotCloud edition. 

Third Party IaaS / Cloud Platform Support 
Due to strong demand from non ECP-based, cloud providers and IaaS platforms, we have decided to open up our marketplace to any and all platforms. In the next week we will publish seller integration instructions for the SpotCloud marketplace so platforms powered by other technologies can easily participate . If you haven't already done so, please sign up to be notified when the docs are ready. 

Joint Marketing and PR Opportunity
Responses to our initial SpotCloud announcement has been overwhelming. A number of buyers and sellers have indicated they would like to participate in a broader announcement highlighting our partners. If you'd like to participate in our upcoming PR announcement, please feel free to get in touch with us.

SpotCloud Webinar and Live Demo  Friday, Nov 19, 2010 
Interested in seeing how SpotCloud works or just have some questions? Join the SpotCloud Webinar next Friday.  

Topic: SpotCloud Demo 
Time: 12:00 pm, Eastern Standard Time (New York, GMT-05:00) 

Register for the SpotCloud Webinar at

Friday, November 5, 2010

Group Buying For Cloud Computing Capacity

As many of you know, I've been to China many times this year. The market for cloud computing is booming over there. But what you may not know is that these trips to China have been a key part of the inspiration for the creation of SpotCloud. Some of my inspirations has come from the popular concept of group buying know as tuangou in Chinese.

If you haven't heard of Group buying wikipedia describes it as follows:
Group buying, which refers to social buying or collective buying as well, is the buying an offer which has been significantly reduced, due to the fact that it is only valid if enough buyers are found. Recently, group buying has been taken online in numerous forms, although group buys prior to 2009 usually referred to the grouping of industrial products for wholesale (especially in China).
Group buys are a variation of tuangou buying that also occurs in China, in which an item must be bought in a minimum quantity or dollar amount, otherwise, the seller will not allow the purchase. Since individuals typically do not need multiples of one item or do not have the resources to buy in bulk, group buys allow people to invite others to purchase in bulk jointly. These group buys often result in better prices for the individual buyers or ensure that a scarce or obscure item is available for sale.
So now lets think about how this approach could be applied to the buying of cloud infrastructure resources through a marketplace such as SpotCloud. A simple example could be buying Amazon EC2 reserve instances. With Reserved Instances you pay a one-time fee and in turn receive a significant discount on the hourly usage charge for that instance over a 1 to 3 year term. Using this model of reserved Instances can save you up to 49% over the cost of On-Demand EC2 instances. But there is a small problem, you need to commit to at least 1 - 3 years with a fairly high utilization for the full savings to be realized.

Now apply group buying to EC2 reserved instances, say 100 or so capacity buyers who need cheap capacity for short periods of time. The brokerage (SpotCloud) essentially buys on behalf of the group and makes the the capacity available at a greatly reduced cost for all market participants. The group buying approach also leads to interesting arbitrage models for increased cost reduction by taking advantage of a price difference between two or more cloud resource providers (Amazon's Spot Instances for example), as well as potentially negotiating wholesale discounts on behalf of the collective buying group from other large cloud capacity providers.

Lots of interesting SpotCloud ideas. I'll keep posting as they come to me.

SpotCloud Launch Overview (Week 1)

So by now you've probably heard about the SpotCloud announcement. If you missed it, after more than a year of development, we finally took the covers off our SpotCloud Capacity Clearinghouse and Marketplace for service providers. The feedback so far has been tremendous, with hundreds of registrations for the SpotCloud service including a good portion of the major "Non ECP" based cloud providers signing up. One particularly interesting stat is the ratio of buyers to sellers registering for the service at a ratio of 5:1 for buyers, indicating significant interest from the demand / buy side. A key metric for a successful marketplace.

But what I really want to tell you is some of the opportunities that have emerged in my discussions with both providers and buyers. Most of which I hadn't even considered before this weeks launch.

The first usage example is with managed service providers. With the recent downturn in the economy a lot of dedicated and managed hosting providers are seeing large amounts of customers abandoning their dedicated servers for various reasons, causing a major influx of un-used racked servers. One director of IT for a large hosting firm indicated that last week one of his long time customers had defaulted on 100+ dedicated servers. He said that they had no plans to create a "public" cloud service but thought that the ability to offer these servers through the SpotCloud marketplace would be an ideal way for his company to cover their "carrying costs" while they attempted to resell / lease these servers to other hosting customers at a much higher margin. Turns out this concept has legs, I had the very same conversation this week at the CloudExpo in Santa Clara with no less then 20 different hosting companies all telling me the same story, not interested in public cloud, very interested in private capacity offering through an opaque market. I used the analogy of renting a new or used car before selling it.

Another recurring opportunity was that of what I have started describing as a "private cloud exchange" where a group of aligned companies or organizations share capacity amongst each other through a private version of the SpotCloud marketplace. A kind of capacity consortium. One particularly good example was from a major Asian Telecom with many various divisions around the globe, each with their own data centers and no effective way to share capacity between the business units who run independently from one another. Another private exchange idea was focused with in specific industry verticals such as Finance, healthcare, entertainment and eduction. Think of a group of private clouds with specific regulatory controls sharing or selling capacity with each other.

What's next for SpotCloud? Going forward we plan to release SpotCloud integration instructions for working with Non ECP based cloud providers and suppliers. This will allow SpotCloud to support the broadest group of capacity suppliers with out having to directly use ECP. We've also had a lot of interest in doing a Free Enomaly ECP SpotCloud edition, which we hope to have available in the near future. As soon as these are ready, I'll post the links on this blog.

We are hoping to do some public announcements around actual buyers and sellers in the coming weeks, ping me if you'd like to be included in this announcement.

Do you have an interesting use case for SpotCloud? Please let me know.

Monday, November 1, 2010

Introducing SpotCloud, The First Clearinghouse & Marketplace for Cloud Computing Services

Enomaly Inc., the leading vendor of Infrastructure-as-a-Service (IaaS) cloud computing software, is proud to announce that it has launched the beta of SpotCloud ( the first cloud computing clearinghouse & marketplace.

For cloud service providers
, the SpotCloud Marketplace Platform provides an easy way to sell unused cloud capacity. Cloud providers can use SpotCloud to clear out unused capacity and sell computing inventory that would otherwise go unsold, enabling increased utilization and revenue, without undermining their standard pricing.

In order to avoid directly competing with regular retail sales of cloud services, SpotCloud uses an "opaque" sales model, similar to sites such as The SpotCloud service meters, tracks, and bills capacity buyers, and pays capacity sellers directly.

For Cloud Capacity buyers
, SpotCloud bridges many disparate regional cloud providers, allowing buyers to find the best cloud providers at the best price. SpotCloud serves as the central place to discover and buy computing capacity, based on performance, cost and location parameters, through a simple and easy to use web dashboard and API. 

By 2014, IDC predicts, sales of cloud computing products or services will generate almost $56 billion in annual revenues. Gartner analyst Daryl Plummer has stated that by 2015 “cloud service brokers will be the largest revenue growth opportunity”, going on to say that “20 percent of cloud services will be consumed via a broker.“

“The market driven approach of SpotCloud is a game changer for both buyers and providers of cloud computing resources.” said Reuven Cohen, Enomaly founder and CTO.  “For service providers, SpotCloud enables the most important feature – the ability to make more money. Each service provider can define prices for the excess capacity offered through the service and adjust these based on time and utilization. For consumers, SpotCloud provides a secure central location to buy from a global pool of providers at highly competitive prices.”

How it Works
How it works

How it Looks
Signup for our Beta at 

#DigitalNibbles Podcast Sponsored by Intel

If you would like to be a guest on the show, please get in touch.