Cloud Index

Interested in deploying applications on spot servers (like Amazon spot instances) but detered by the uncertainty of variable pricing? Inspired by the economic and financial market indices, we design a mechanism to reliably estimate the cost efficiency of flexible applications on EC2 spot markets.

Why an Index?

In contrast to the current practice of predicting spot market behavior solely based on the historical price traces, we take first principles approach to observing the infrastructure-level realities in public cloud datacenters. Based on these observations, we infer that spot markets are reliably predictable at aggregate levels than at individual level. Then, towards collectively modeling a group of spot markets, we introduce a market index for the cloud.

The resulting cloud index not only provides reliable cost estimates but also reflects the fair market value of the idle cloud capacity. Our insights from applying the index to the global spot markets are shared here.

Index-tracking by Server Hopping

Our system design goal is simple: run a given application on variable-price spot servers such that it incurs a predictable expense. In doing so, we observe that investors face similar issues in financial markets when making investment decisions. While broad market indices like S&P 500 and Dow Jones Industrial Average provide a high-level benchmark, investors employ active trading to adjust their portfolio so that it continues to meet their investment targets despite the volatility of financial markets. Enabled by recent advances in container virtualization, datacenter networking and per-second billing models, we design an index-tracking mechanism that utilizes server hopping technique. We describe in more detail the properties of index-tracking, policies for cost-availability tradeoff, and our system evaluations in the papers below.

Publications

IndeX Marks the Spot: Cost-Predictive Server Hosting in Spot Markets
Supreeth Shastri and David Irwin
In Submission   PAPER  

Towards Index-based Global Trading in Cloud Spot Markets
Supreeth Shastri and David Irwin
HotCloud 2017   PAPER   PRESENTATION

Contact Us

We would love to hear from you. Please send a note to Supreeth or David for any questions or feedback.

Based on a template from Matei Zaharia.