Wednesday, July 29, 2009

Cloud Computing as a Commodity

-- Update ---
I'm happy to announce that we've launched SpotCloud, the first Cloud Capacity Clearinghouse and Marketplace. Check it out at
I seem to keep coming back to the same question when discussing Cloud Computing. Can cloud computing be treated as a commodity that could be brokered and or exchanged? Recently a few have attempted to do this, notably a German firm called Zimory.

To give you a little background, before the development of the Enomaly ECP platform. I had the grand idea to create what I described as "Distributed Exchange (DX)" (circa 2004 - I've put the site online temporarily for demo purposes). This was actually one of the original motivations for the creation of the original ECP platform (aka Enomalism) The idea of DX was to create a platform and marketplace which would allow companies to buy and sell excess computing capacity similar to that of a commodities /exchange marketplace. Think Google Adwords & Adsense for compute capacity.

I actually put quite a bit of thought into the whole concept. Generally the concept was to use a commodity-based approach to manage computational resources for consumers in a peer-to-peer style computing grid. In turn buyers and sellers would have access to an electronic trading environment where they could bid for unused compute cycles from a Google Adwords style web interface. Consumers could bid on computing power while Computing Capacity providers offer cost-effective computational capacity on demand, thus creating a competitive computing marketplace.

Although the concept may have been fairly well thought out, it was way too early. For one thing, there was no demand for the service, and the second problem was compute capacity wasn't and still isn't an actual commodity which can be supplied without qualitative differentiation across the greater cloud computing market. Basically there is no basis for an apples to apples comparison.

Wikipedia describes a commodity as "a product that is the same no matter who produces it, such as petroleum, notebook paper, or milk. In other words, copper is copper. The price of copper is universal, and fluctuates daily based on global supply and demand. Stereos, on the other hand, have many levels of quality. And, the better a stereo is [perceived to be], the more it will cost."

Another major limiting factor for treating cloud capacity as a commodity is that there are no standards for cloud capacity and therefore there is no effective way for users to compare it among cloud providers. With out this standardization there is no way to determine optimal cloud capacity requirements for particular application demands and thus determine the optimal price & costing. In order to overcome this, I believe a Standardized Cloud Performance Measurement and Rating System (SCPM) will need to be created which would form a basis of measurement through an aggregate performance benchmark.

As an example a cloud provider may want to use some aggregate performance metrics as a basis of comparing themselves to other providers. For example, Cloud A (High End) has 1,000 servers and fibre channel, Provider B (Commodity) has 50,000 servers but uses direct attached storage. Both are useful but for different reasons. If I want performance I pick Cloud A, if I want massive scale I pick Cloud B. Think of it like the food guide on back of your cereal box. This may provide the basis for determine the value and therefore a cost for the cloud capacity.

This has been one of the motivations behind the creation of an open standard for cloud computing capacity called the Universal Compute Unit (UcU) and it's inverse Universal Compute Cycle (UCC). An open standard unit of measurement (with benchmarking tools) which would allow providers, enablers and consumers to be able to easily, quickly and efficiently access auditable compute capacity with the knowledge that 1 UcU is the same regardless of the cloud provider. (See my previous posts on the subject)

I also believe that the creation of cloud exchanges & brokerage services has less to do with the technology and more to do with the concept of trust and accountability. If I'm going to buy a certain amount of regional cloud capacity ahead of time for my Christmas rush. I want to rest assured that the capacity will be actually available with an agreed upon quality and service level. I also need to be assured that the exchange is financially stable / adequately capitalized and will remain in business for the foreseeable future.

I've never been a particularly big fan of regulation, but given the potential for fraud some oversight may be required to assure a fair and balanced playing field. If we truly want to enable a cloud computing exchange / marketplace another option might be to build upon existing exchange platforms with a proven history. A platform with an existing level of trust, governance and compliance.

#DigitalNibbles Podcast Sponsored by Intel

If you would like to be a guest on the show, please get in touch.