Wednesday, March 23, 2011

Real-time SpotCloud Market Data API Released

One of the most requested additions to the SpotCloud API is the ability to get realtime market data. I'm happy to announce that we've added this to the latest SpotCloud Provider API update V1.2.

These new API calls allow you to adjust your pricing to be based on market conditions or simply see what the most popular locations or instance type are at any given moment giving our providers an extra level of insight into the SpotCloud market.

You can grab the latest API docs from the following locations.


GET /api/v1/provider/market.{json|xml}

Response Fields

  • average_package_size - The average appliance storage size.
  • popular_cpu - A list of popular CPU counts used when creating instances.
  • popular_memory - A list of popular memory sizes used when creating instances.
  • popular_location - A list of popular geographic locations used to deploy instances.
  • average_memory_cost - The current average cost per GB of memory.
  • average_cpu_cost - The current average cost per CPU.

Friday, March 11, 2011

SpotCloud Update (New App Directory)

We’re happy to announce that we’ve launched a new Appliance Directory for SpotCloud. The integrated appliance directory allows you to more easily deploy from a directory of pre-built SpotCloud appliances to any of our providers from around the globe. To get started, login to the SpotCloud management dashboard as a buyer, select a providers and you’ll see a list of appliances in the drop down.

We are actively adding new appliances to the directory, so check back frequently. Some of the appliances we have available currently included:

The SpotCloud Webmin appliance is our generic Ubuntu 10.10 with a Webmin management console added to it. This allows you to visually configure all aspects of your server, including installing packages and managing users.

BOINC SpotCloud (Distributed Grid Computing Platform)
The BOINC appliance allows you to create clients anywhere in the world for handling distributed processing jobs. To operate the appliance, create a new instance with it. You can then ssh into the instance for configuration. The configuration script only asks for one item, and that is the IP address your BOINC management server that issues jobs.

Built using the SpotCloud Ubuntu OS, the SpotCloud Open VPN appliance allows peers to authenticate each other using a pre-shared secret key, certificates, or username/password. Essentially, using Open VPN on SpotCloud allows for on-the-fly, anonymous internet access using on-demand, regional spot instances.

Ubuntu 10.10
Ubuntu JeOS (pronounced "Juice") is an efficient variant of the Ubuntu Server operating system, configured specifically for use as virtual appliances. This is a specialized installation of Ubuntu Server Edition with a tuned kernel that only contains the base elements needed to run within a distributed SpotCloud environment.

Built on Ubuntu JeOS & Aflexi's CDN automation control panel is the first CDN software especially catered for cloud computing. The Aflexi software allows you to quickly and easily build your own global content delivery network using SpotCloud resources from around the globe.

Varnish Cache
This is a fully functional SpotCloud appliance built on Ubuntu JeOS and Varnish Cache. It includes a one time command line configuration example accessible via SSH console on any SpotCloud provider. Once configured, the appliance automatically locks itself down preventing further access.

Get Listed in our Directory
We’re looking for more reference SpotCloud appliances, If you have an application well suited for SpotCloud please get in touch with us.

Tuesday, March 8, 2011

The Amortization of Cloud Futures and Derivatives

Really interesting tweet from @jayfry3 earlier today. In it he noted a reference by @joeweinman that said Netflix treats reserved AWS instances as CapEx and depreciates it's cost over 3 yrs. This immediately got me thinking -- Wow, now that's an interesting concept.

Before I go any further I warn you I am not a tax expert, and if you are, please feel free to correct anything I say from this point forward. This is a random thought, more than a proof.

First of all, I don't think what they were referring to is actually depreciation since depreciation strictly refers to tangible assets. Instead I think what they're talking about is a very similar concept of amortization which is basically the same but for intangible assets.

A quick recap in tax law, according to Wikipedia "amortization refers to the cost recovery system for intangible property. Although the theory behind cost recovery deductions of amortization is to deduct from basis in a systematic manner over an asset's estimated useful economic life so as to reflect its consumption, expiration, obsolescence or other decline in value as a result of use or the passage of time, many times a perfect match of income and deductions does not occur for policy reasons."

With the launch of SpotCloud which essentially is a Spot Market for computing capacity, the concept of being able to reserve computing capacity in the form of a futures contract or derivative has been a popular topic of conversation during my various presentations and pitches lately. Previously I saw the opportunity for "cloud futures" from the point of view of the provider the capacity. Basically allowing the provider a greater level of insight into future capacity consumption, inventory and capacity planning. The missing part of the equation has been on the buy side, other then potentially locking in a future price, (as a hedge) the rationale for buying future computing capacity was fairly limited. With the introduction of amortization to the equation the concept dramatically shifts from not only a capacity planning exercise but also to a tax and accounting strategy for major buyers of computing capacity.

So much like traditional depreciation, amortizing future computing capacity bought as a "reservation" or derivative allows the compute asset to be deferred rather than treated as a current expense with the difference of the spot price of the compute asset as recorded on the Spot market at that the time of consumption defining a gain or loss. Taking this concept even further, if the 'depreciable' or amortized compute asset is not actually used, but instead re-sold on a computing Spot market such as SpotCloud, the business can again recognize a gain or loss based on net basis of the asset. (The net basis is cost less amortization) Yup, crazy, and it would seem -- totally do-able.

Are we on the verge of a cloud futures market? Maybe sooner than you think.

Sunday, March 6, 2011

The Rise of The Cloud Aggregator

Recently there has been a lot of renewed talk of "Cloud Aggregators" a term that has been thrown around quite bit of the the last few years. But what is a cloud aggregator really? How does one define this segement? How do you qualify a company or service as a cloud aggregator? It's time for my turn to attempt to define this some-what vague term.

First, more generally what is a aggregator? One of the best descriptions I could find is described as "a system or service that combines data or items with similar characteristics (geographic area, target market, size, etc.) into larger entities. Value is derived from cost savings, or the ability to reach a larger market and charge higher prices from bundling multiple goods or services."

I'm not sure I agree with the higher prices part, but I do think the key point is the value in assembling "a system or service" that brings together a group of formally distinct components or web services. For example Google at its heart is an aggregator, all those websites found on the Internet would and do exist regardless of Google, but easy access to all of them through a simple, power and effient interface had previously not existed (at least not in a way that was relevant). Then there are marketplace aggregators such as Ebay, in this case without the ebay platform each of these vendors could not exist beyond potentially a physical storefront. Ebay provided both the aggregation, facilitation (which makes tasks for others easy) and fulfillment (completing the transaction).

So using Google and Ebay as two ends of the aggregator spectrum. You have Google the market disrupter versus ebay the market maker. Each important, but important for very different reasons. Both share common traits in that they provide easy access to something that was previously not very easily accessible. Yet each derive their true value in completely different ways. Google brought order to chaos through the use of advanced algorithms and massive computational power and ebay created a structured marketplace that had never existed in an area that needed it.

 Now back to cloud aggregators. Here's my definition.

Cloud Aggregator  - a platform or service that combines multiple clouds with similar characteristics (geographic area, cost, technology size, etc.) into a single point of access, format, and structure. Value is derived from cost savings and greater efficiency found from the ability to easily leverage multiple services providers.

Thursday, March 3, 2011

Living on the Edge

There's somethin' wrong with the cloud today, I think I know what it is. We're seeing things in a different way, but should we be judging a cloud provider by the color of their -- logo?

It's seems that for many, the only basis of comparing cloud providers is based upon superficial aspects. I know of the company, recognize the logo, or read a random review. But the reality is we're moving away from the traditional vendor driven marketing fluff of a single provider world to a multi-cloud, federated ecosystem of capacity providers, where brand recognition is less important than performance and price. I'm talking about living on the edge, the edge of the network.

One of the more interesting recent announcements was Amazon's Japan availability zone, in describing their launch AWS spoke of latency for users within Tokyo being less than 10ms. Hitting directly at the heart of the opportunity. Yet on the flip side, they also mentioned that the Japanese zone was ideal for other nearby geographies, to which I say they're missing the point. Using Japanese resources in South Korea makes little sense given the rapid advancement and availability of cloud capacity in South Korea. The opportunity going forward isn't to address generalized areas of the world, but to address the specifics, not just on a country basis but on a city or even a neighbourhood basis. A single provider will never be able to get this level of granularity regardless of how much money they have. Economies of scale will always be limited by total market size, the more granular the market the less economies of scale work in favor the large provider. The only way to address this growing movement toward edge based, latency dependent application deployment is to federate many providers with many customers across many diverse geographies.

I admit that this is easier said than done, ask anyone who's attempted to use more than one provider in a federated, intercloud connection global cloud of clouds type of deployment and tell me what you think about your sit-u-a-tion. They'll tell you it's complication, and aggravation.

I know a few of you will point back to the inevitable economies of scale that an AWS or Google bring forth. Yes, they probably spent many millions building their Japanese cloud infrastructure. But did they have to? Economies of scale are important factors mostly for true for commoditized IT aspects such as bandwidth. But for areas such as ultra-localization computing, this is not practical for even the largest web companies. Sure there are many factors that cause a cloud providers average cost per compute unit to fall as the scale of output is increased, purchasing power is probably the most relevant. Essentially the biggest Internet companies can buy the most servers at a massive volume thus getting the lowest cost per unit of compute time and therefore achieving the best margins at the lowest cost. But even this equation has a practical limit when it comes to geography.

When it comes to ultra-localization the boundaries of the so-called provider economies of scale, commoditiziation and volume quickly breakdown. Now it becomes a question of federation and aggregation. Many providers connected through a normalized or structured market interface rather then one provider attempting to address all markets. I'm not just talking about what we're doing with SpotCloud, but what I believe to be the move toward a market centric economy of federated cloud ecosystems, a move I think is inevitable. Ultra-local capacity or edge based computing, or whatever you chose to call it in a nutshell is the opportunity moving forward.

The choice will quickly become one of choosing a single provider (one to many) or an aggregator (many to many). I believe the choice will quickly become obvious to anyone who has a geographic component to their applications. The opportunity is living on the edge.

Tuesday, March 1, 2011

SpotCloud Beta Update (March)

To say we’ve had a spectacular launch would be putting it lightly. Since our Valentines Day SpotCloud Launch, we’ve been inundated with interest in the marketplace. Buyers, sellers, investors and media have flocked to us in the thousands from around the world.

As of today we have dozens of active providers online from Hafnarfjörður (that’s in Iceland) to Boston to Brisbane to Hong Kong. There are also in the hundreds of providers who have signed up and are in various stages of preparation and set-up of their SpotCloud environments. We are actively on-boarding new providers as fast as we can. If you haven’t done so yet, go ahead and get started as a provider

We’re also seeing a lot of interest from the buyer side as well with a ratio of 5:1 buyers to sellers.  It’s been great to observe an increasing rate of buyers converting from the initial “tire-kicking” phase to becoming more active users.  The market is now open and free to browse. So go ahead and sign up as a buyer.

Some Various SpotCloud Market Updates

Instance Renewals
We just pushed an update to the SpotCloud marketplace that fixes a few bugs, but, most importantly, adds the ability for SpotCloud buyers to renew their instances at the current spot market price. Previously, at the end of the maximum duration, a buyer’s appliance would be automatically terminated (deleted). Now, if a SpotCloud seller enables this feature, a buyer will have the option to keep their instance at the end of the duration and pay the current spot price going forward. More importantly, they can keep their IP address and data. An email is automatically sent to the Buyer when their instance is close to being terminated to provide the option of having it renewed. 

As a seller, to enable this feature, you must login to the SpotCloud management interface as a seller at and select the check box on each hardware profile labeled "Instance renew allowed"

As a buyer, you’ll see the new option in your instance information or via an email sent to you before termination. We hope to include this in our API in an upcoming release.

New SpotCloud Apppliances
We’ve published several new sample spotcloud Appliance including a OpenVPN appliance and Aflexi CDN appliance. In the next few weeks we will be releasing a built-in App Directory in the SpotCloud market, to enable much easier deployment of SpotCloud Appliances. 

Get Featured in our upcoming App Directory
If you have an application that would make a great sample SpotCloud appliance, we’d love to include it in our upcoming App Directory, so please get in touch.

Create your own appliance using the SpotCloud CLI & appliance bundling tool
You can now easily convert an existing VM to the SpotCloud Appliance format using our SpotCloud CLI and Appliance Bundling Tool. This tool also provides an easy command line API access to the SpotCloud marketplace enabling a true hybrid cloud computing experience.

Selected Media Coverage

Economist Magazine - Cloud computing: A market for computing power
“LIKE oil or pork bellies, computing capacity is now a tradeable commodity. February 14th saw the launch of SpotCloud, the world’s first spot market for cloud computing. It works much like other spot markets”.

Eweek Review - SpotCloud Explores the Cloud's Utility Computing Future
“One of the most popular images used to illustrate the concept of cloud computing is the electrical grid. The “cloud” can be tricky to pin down, but everyone is familiar with plugging into a standardized socket and drawing the juice necessary to run many electrical devices from a network of large power plants.”

#DigitalNibbles Podcast Sponsored by Intel

If you would like to be a guest on the show, please get in touch.