Embracing Low Performance Computing (LPC)

Had an interesting time in Banff last week at the Summit09 conference. The conference was a combination of events including the Cybera/CANARIE Summit, OGF 27 and IEEE Grid 2009 which focused predominately on the emerging opportunity for Cloud Computing with in the intersection of High Performance, Grid and Distributed Computing realms. The Summit brought the leading figures from around the globe including organizations such CERN as well as various other academics.

A few of the more interesting tidbits included, the discussions around the future of the Open Grid Forum. It seems that the OGF is currently going through a major transition as the Grid world is quickly distancing itself from the stigma surrounding the use of the term Grid or High Performance Computing. There were several conversations discussing whether the OGF should even continue calling themselves the Open Grid Forum with a few even suggesting the Open Cloud Forum might be more suitable name. Also notible was most of the marketing materials at the Summit simply refers to the OGF.

What I also found quite interesting was that there is little discussion on the topic of traditional high performance computing. But instead what everyone seemed to want to know was this thing called Cloud Computing and how it could benefit or improve their existing grid deployments. It was clear that the vast majority of attendees seem to realize that cloud computing isn't just another way to describe grid or distributed computing, but instead the opportunity to reimagine how they could address the shift to globalized web centric computing.

For the most part this reimagining seem to mean the movement from the traditional aspects of HPC to the notion of Low Performance Computing or LPC. Several times I heard the analogy of "I'd rather have something done now that takes a few days to complete then wait two weeks for something that takes a few hours to complete." These comments seem to hit at the heart of the opportunity. The new reality facing traditional grid centric architecture is that of efficiency and adaptability. The old grid computing systems and platforms tend to focus on one or two specifically optimized operations, but do little else. With the introduction of virtualization and infrastructure as a service platforms there seems to be a renewed excitement in the ability to quickly re-provision and rapidly implement self service applications.

Another comment I heard several times was the less these grid folks needed to involve their system admins, the better. The idea of humanless computing was a topic that kept coming up. The idea that the more tasks that could be automated, the less chance for human errors to be introduced into the various work flows. Simply humans seem to be the biggest obstacle facing grid related technologies.

All in all a very interesting week in Banff.

Humanless Computing

I wanted to do a quick post before I take off for the next couple weeks. I'll be at my cottage next week and in Venice Italy the following week speaking at a private business summit at a monastery in the heart of the city. Rough life, I know. As part of this gig they even agreed to fly out my wife and 7 month old son.

A few random thoughts before I leave. As we move more towards autonomic computing future, I think the greater test of a efficient fault tolerant infrastructure will be to measure the human factor in it's management or possibly the lack of it. I know that some may not like a future where computers and infrastructure no longer need humans to manage it, but the reality is we as human operators are the biggest single limiting factor when it comes to computing. So while I'm away, ponder this. The next big thing in computing may very well be humanless computing.

I'll leave you with this great presentation by Lew Tucker at Sun Microsystem where he theorizes applications that are entirely self-sufficient, where humans will be able to set boundaries, of course, but will no longer be needed to turn servers, or anything else for that matter, physically on or off.

Waiting in the Cloud Queue

Which would you rather have? A compute job that gets done over a 12 hour period on a Supercomputer with the catch that you need to wait 7 days until the job actually runs? Or a job that runs over a 60 hour period on a lower performance public cloud infrastructure that can start immediately?

In a recent post Ian Foster asked just this saying "what if I don't care how fast my programs run, I simply want to run them as soon as possible? In that case, the relevant metric is not execution time but elapsed time from submission to the completion of execution. (In other words, the time that we must wait before execution starts becomes significant.)"

What I can't help wondering is whether cloud computing may be shifting the focus of high performance computing from the need for optimized peak utilization of a few very specific tasks to lower performance cloud platforms that can run a much broader set of diverse parallel tasks.

Or to put it another way, in those seven days while I wait for my traditional HPC job to get scheduled and completed, I could have been running dozens of similar jobs on lower performance public cloud infrastructures capable of running multiple variations of the original task in parallel.

In a sense this question perfectly illustrates the potential economies of scale cloud computing enables. (a long run concept that refers to reductions in unit cost as the size of a facility, or scale, increases) On a singular basis my job will take a significantly longer period of time to execute. But on the other hand, by using a public cloud there is siginificanly more capacity available to me, so I am able to do significantly more at a much lower cost per compute cycle in roughly the same time my original job was in the queue .

Hoff's Cloud Metastructure

Recently, Chris Hoff posted an interesting concept for simply defining the logical parts of a cloud computing stack. Part of his concept is something he is calling the "Metastructure" or "essentially infrastructure is comprised of all the compute, network and storage moving parts that we identify as infrastructure today. The protocols and mechanisms that provide the interface between the infrastructure layer and the applications and information above it".

Actually I quite like the concept and the simplicity he uses to describe it. Hoff's variation is the practical implementation for a meta-abstraction layer that sits nicely between existing hardware and software stacks while enabling a bridge for future yet undiscovered / undeveloped technologies.. The idea of a Metastructure provides an extensible bridge between the legacy world of hardware based deployments and the new world of virtualized / unified computing. (You can see why Hoff is working at Cisco, he get's the core concepts of unified computing -- one API to rule them all)

In a post back in February, I described the contrast of a Cloud "Metaverse" as a logical inverse to the traditional structured view of infrastructure. My idea was to describe everything that exists beyond the confines of a particular virtualized environment through the use of an extensible semantic API abstraction. At the heart of this theory there is the ability to define the relationship of how multiple distributed clouds describe their interrelations between themselves (who, what, where, when, and so on).

I'd also like to point out that my concept was inspired in part by the suggestion of Scott Radeztsky at Sun to look at the problem of cloud interoperability as a meta-problem, (problems describing other problems). Also my original Metacloud post was inspired by a multiverse post I wrote which was itself inspired by a post by Chris Hoff where he proposed an interesting use case for IPv6. So we seem to have come full circle.

In order to solve abstract problems, we need abstract solutions to these problems. This fit perfectly into my Semantic Cloud Abstraction thesis loosely described as an API for other API's.

A Standardized Cloud Performance Rating System

Ever get one of those random phone calls in the midst of your work day that makes you think, huh -- interesting idea? Well earlier today I had one from a guy looking to learn more about cloud computing platforms. Although it ended up he wasn't specifically looking for an elastic computing platform, he did ask a few very thought provoking questions.

What he asked was if there is a simple way to compare the performance, security and quality of various cloud computing providers? He went on to say that when comparing traditional hardware vendors it was easy for him to understand the standardized specifications (GHZ, GB, etc) as well as determine quality based on brand recognition, but in the cloud world there was no easy way for him to compare "apples to apples". In his words, "there is Amazon and then there is everyone else". Although overly simplified, he was kind of right. For a lot of people looking to get into the cloud, it's a bit of a mystery.

This got me thinking. With all the talk lately of cloud standards, is there an opportunity to create a common or standard Cloud Performance Rating System? And if so, how might it work?

Unlike CPU or Storage, Cloud Computing is significantly more complex involving many different moving parts (deployment approaches, architectures and operating models). Defining one common standardized basis of comparison would be practically impossible. But within the various aspects of cloud computing there certainly are distinct areas that we may be able to quantify. The most likely starting point would be infrastructure related offerings such as compute and storage clouds.

The next question is what would you rate? Quality, performance, security? And how might these be actually quantified?

I'm going to leave those answers for another time. But it does make you think. So thank you random guy for brightening up an otherwise rainy day.

Elasticvapor is 1 year old!

I'm happy to say that ElasticVapor is now one year old. And what a year it has been.

Here is an overview of my RSS readership via Feedburn. These stats doesn't include the nearly 200,000 people who have read the site directly through a browser.

Since April 2008.

• 164,927 views of 472 items
• 18,086 clicks back to the site on 472 items

Top 30 Posts by Popularity (RSS Readers)

Name	Views	Clicks
Semantic Cloud Abstraction	1183	323
Cloud Computing is for everyone -- except stupid…	1132	112
Re: Microsoft Moving Toward an Open Pro…	1033	97
Cloud Control with Google Talk (Botnets)	811	303
The untold (back) story of Amazon EC2	762	302
Inside access to Microsoft VSTS Invitat…	949	44
Live From Stockholm, Ruv I Molnet	921	38
Cisco's Grand Unified Computing Vision	761	195
Browser Based Distributed Computing	823	120
The Universal Amazon EC2 API Adapter	742	147
Cloud Attack: Economic Denial of Sustain…	806	75
New Design for ElasticVapor.com	778	84
Is Cloud Computing Becoming A National …	717	145
The Case Against Commodity Cloud Exchange…	797	59
(Update) Wall Street Cloud Interoperability…	781	70
Navigating the Fog -- Billing, Metering…	746	104
Cloud Warfare & Proactive Network Defences…	733	100
Sun Announces Open Cloud Platform & API	737	90
Breaking News: Denial of Service, The P…	712	110
CloudCamp Federal @ FOSE (Washington DC…	755	65
Appistry Bridging Application Clouds	737	80
Enomaly ECP 2.2 Released	633	182
Small Web Hosts turning to 'Mini Clouds'	735	79
CCIF Skype Discussion Group Created	742	72
The Hybrid Cloud Multiverse (IPv6 VLANS)	706	91
The State of the Cloud Computing Intero…	715	80
10 Steps To Unified Cloud Computing: Cl…	684	105
CCIF Wall Street. RSVP	702	84
IBM & Juniper team up for Cloud Migrati…	720	61
Forget Cloud Standards, First Think Clo…	718	62

Draft Agenda for DC Cloud Computing Interoperability Workshop

March 23, Hyatt Regency Crystal City, Virginia

Morning Session: Standards Groups (8:00 - 12:00)

8:00 - 8:30 NIST - Tim Grance (NIST Cloud Program Manager), Peter Mell (Senior Computer Scientist)

8:30 - 9:00 Cloud Computing Interoperability Forum - Reuven Cohen Leader, Founder & Chief Technologist, Enomaly Inc)
“Defining an End-to-End Cloud with Web Architecture"

9:00 - 9:30 Open Cloud Consortium - Robert Grossman (Leader)

"An Overview of the Open Cloud Consortium"

9:30- 10:00 Open Grid Forum - Craig Lee (President)
"Distributed Computing Scenarios: What this Means for Interoperability"

10:30 - 11:00 Network Centric Operations Industry Consortium -Krishna Sankar (Leader of Cloud Computing Team)

”Vectors in Federal Cloud Computing”

11:00 - 12:00 Interactive Panel and Q&A with all Standards Group Speakers
=======================

Afternoon Session: Companies (1:00 - 6:00)


1:00 - 1:30 Cisco - Krishna Sankar (Distinguished Engineer)

"A Hitchhiker’s Guide to InterCloud”

1:30 - 2:00 IBM - TBD

TBD

2:00 - 2:30 Microsoft - Susie Adams (CTO of the Federal Sales organization)
"Interoperability: A Necessary Condition for Success in the Cloud"

2:30 - 3:00 Salesforce.com - Dan Burton (Senior Vice President, Global Public Policy)
"Salesforce.com and the Interoperable Cloud"

3:30 - 4:00 Sun - Scott Radeztsky (Chief Architect for Americas Systems Engineering)

"Real Clouds for Real People"

4:00 - 4:30 Elastra - Stuart Charlton (Chief Software Architect)
"Defining an End-to-End Cloud with Web Architecture"

4:30 - 5:30 Interactive Panel and Q&A with all Company Speakers

5:30 - 6:00 Wrap-up discussions