I don’t know about you, but Amazon Web Services’ announcement last week that it is launching a spot market for its Elastic Cloud Computing service (EC2) struck me as a kind of watershed event.
Spot markets pop up in mature capacity industries where buyers bid on the market price of something at any given moment in time based on supply, demand and constraints to production. Spot markets are a sure sign of a community’s understanding of how constraints affect a constant price and you see it in wizened oil or livestock traders all the time.
Amazon Spot Instances (as they’re formally called), create the first such market in computing infrastructure that I’m aware of. That’s why we might want to preserve the date somewhere as a minor infrastructure milestone.
All you really need to remember is that, for Amazon, idle CPUs are the devil’s plaything and that a spot market creates a legitimate market for traders. AWS customers who don’t buy on demand by the hour (costliest option), or reserve future CPU usage (less costly) can test the spot market for pricing at a threshold (likely the least costly).
The hitch with Spot Instances is that your job runs as long as your bid remains above the spot price. When the spot price exceeds the bid (someone outbids you or you hit a peak usage period), the service stops. When the spot price falls back below your bid, the task resumes. For that reason, Spot Instances are best suited to folks with flexible workloads who are not on deadline.
Amazon CTO Werner Vogels says in his blog that likely spot users have an ongoing process, e.g. Web crawling or database processing. And you don’t buy spot for projects that cannot be restarted in midstream.
But I digress. The details of Spot Instances aren’t as cool as the idea that supply, demand and capacity are opening new markets with dynamic pricing and bidding. If you get enough interest and bidders, you’re making a real market with visibility and understanding that negotiates fair market value, not something based on a salesman’s handshake.
Saugatuck put out a research note (registration required) to say the new offer will confuse and frustrate new users who don’t understand how it works. I’d agree and hope that there are likely customers out there who remember bits of their Econ 201 class and understand why Amazon is willing to drop price for their data centers over a slow weekend or on cold nights when energy costs go down. Those savings all add up.
It’s familiar turf to traders, but not to technologists. What’s next, a true futures market? CPU derivatives? I’d better hold that thought before I go down the road to AAA+ rated read/write debentures, but darned if you can’t see it on the horizon somewhere.