An Interview With The Architects of The Cloud Servers API

There’s good news for the Rackspace Cloud developer community: the wait is over.

Today, The Rackspace Cloud launched our new API for Cloud Servers. The API provides a flexible, programmatic way to Control Cloud Servers. Now, from within your own applications, you can write code that launches or reboots Cloud Server instances, creates custom metadata for your servers, injects files into the servers’ filesystems, creates high availability configurations using shared IP groups, and a variety of other tasks. You can learn more about Cloud Servers and the features of the API on our site.

Last Friday, I was able to sit down with the project’s lead architects, Erik Carlin and Jason Seats, to talk about designing the API, responding to community feedback, and industry standards around the cloud. Below are the answers they provided.

So a couple of the new features in the API allow you to customize your Cloud Server instances as you launch them. How do you envision those features being used?

Erik: Really, the idea is to provide layers of customization. First, you can create custom server images, and then you can further customize the instances you launch with metadata and file injection. Elasticity and the dynamic nature of cloud computing demand these kinds of real-time configuration options.

Jason: With metadata, you can provide arbitrary key/value pairs that are queryable through the API and associated with a running instance, providing a unique fingerprint for each one. With file injection you can put a file directly into the filesystem so that it’s accessible inside the instance. It could be a license file, it could be an SSH key—whatever you need.

What were the tough decisions in designing the API?

Erik: We wanted to adhere as strictly as possible to RESTful practice. We iterated several times on the design to make it more and more RESTful. We actually did an update this week where we made some final changes because we just didn’t feel like it was RESTful enough.

Jason: Another goal was conforming to open, accepted web standards that exist. For example, we worked to make our use of HTTP error codes precise and accurate. We are using headers the way they should be according to our interpretation of those standards. We really tried to not invent anything new but rather to build things on well-defined standards.

I think any developer or architect who’s tried to take a service and map it on to a RESTful interface would understand the pain of doing it. You can spend a lot of time debating how things should be modeled. Turning a real-world, sometimes messy thing into what’s really an idealistic, simple view is hard. It’s a long struggle, but when you come out the other end, it ends up looking very concise and clear and it’s easy to understand.

Erik: We spent a lot of time thinking through the API so that it’s very consistent—consistent with itself and with the Cloud Files API. Across the AWS services, Amazon doesn’t necessarily use the same web service interface types. So EC2 has no purely RESTful interface. They have a SOAP interface and what’s called a Query interface, which is REST-like, but not truly RESTful.

A while ago, a draft version of the API was released for public feedback. Can you give a few examples of the feedback that came back and how it drove the API’s development?

Erik: Feedback from the draft was really important. The metadata option and the server data injection feature were directly born of suggestions from the developer community. The feedback we received also drove the API’s support of XML and JSON—we got lots of requests for JSON. A few developers told us that our first take at JSON wasn’t as efficient or as elegant as it could be, so we went back and reworked that piece.

Jason: One thing we got real strong feedback on from the early viewers of the spec was visibility into the caching mechanism. There are web services out there that involve caching and it’s never entirely clear what the age of the cache is. We thought through those things and tried to use well defined ways to display how the caching layer works.

Request limits are something we actually expose through the API and your application or service can dynamically and programmatically ask about what its limitations are. We also have very clear ways of telling you when your application has exceeded its available requests and is being throttled or served cached results because of that.

There’s been a lot of debate in the cloud space concerning open standards and how they should be defined. What is it that makes developing standards across cloud providers so difficult?

Jason: As a company, we’ve certainly committed to supporting open standards but I think that we’re very realistic about recognizing the difficultly in establishing open standards. And it really is a reflection of different providers who have different underlying technology; different limitations. Those differences are reflected in these APIs. To have one overarching standard would mean the standard would be necessarily more complex than each of the implementations. A more generalized, more complex standard takes a long time to erupt naturally.

Erik: Rackspace is on the leadership board of the DMTF cloud incubator group and we’re working on a number of standards around cloud. We intend to open source the API specification as well so you can distribute, modify, or reuse the API freely. We tried to open it up as much as we can, recognizing that our specific design and the things we need to do are unique. If there was an existing API standard that we could have embraced I think we would have done it.

So this is the first public beta release of the Cloud Servers API. How will it evolve going forward?

Erik: Well first, we tried to think ahead: simple things like versioning the API so that we could introduce new features without breaking the API. You can query the API without the version and see all the versions that are available. You can query an API version and get a link directly to the API docs or WADL for a human readable or computer readable version of the specification.

Jason: In future versions of the API, we’re looking at tightening the link between Cloud Servers and Cloud Files.  Our expectation is that we’ll extend the API and the Cloud Server service so that we’ll have launch-able images on Cloud FIles. That’s probably the main thrust of the next rev that we’re looking at right now.