Building a RESTful API in a Rails Application

1 Introduction

There comes a time in the development lifecycle of most web applications when a third-party integration becomes necessary. One of the simplest ways to do so is to expose a REST API for consumption. This article will walk you through a possible approach to designing and implementing a REST API in an intentionally simplistic task management web application, and will cover some best practices to ensure maintainability of the code.

2 Requirements, Assumptions

This article is going to assume some familiarity with the Rails framework, the Ruby programming language in general, and at least some familiarity with the Rails ecosystem by way of Devise, RSpec, and Capybara. When this does not significantly impact DRYness (DRY stands for "Don't Repeat Yourself," a mantra of software development), the more verbose syntax was intentionally chosen for readability.

2.1 The theoretical application

We will build out an API which corresponds to a task management application. It will contain a User model to represent users with access to the system, a Project model to represent projects, and a Todo model representing specific tasks to be done within a project. As such, a User will have many Projects, and a Project will have many Todos.
3 Theory. What makes an API RESTful?

• Statelessness: Client state should not be stored on the server between requests. Said another way, each individual request should have no context of the requests that came before it.

• Resource identification per request: Each request should uniquely identify a single resource. Said differently, each request that modifies the database should act on one and only one row of one and only one table.Requests that only fetch information should get zero or more rows from one table.

• Representational state transfer: The resource endpoints should return representations of the resource as data, usually XML or JSON. The returned information should be sufficient for the client to uniquely identify and manipulate the database row(s) in question.

While there are other criteria which need to be fulfilled for a theoretically pure REST API, these suffice in practice.

4. A basic REST API in Rails

4.1 Routes

Rails provides a fantastic tool for defining endpoints in the form of routes:

rubyApiDemoApp::Application.routes.draw do 
    scope '/api' do 
        scope '/v1' do 
            scope '/projects' do 
                get '/' => 'api_projects#index' 
                post '/' => 'api_projects#create' 
                scope '/:name' do 
                    get '/' => 'api_projects#show' 
                    put '/' => 'api_projects#update'                         scope '/todos' do 
                        get '/' => 'api_todos#index' 
                        post '/' => 'api_todos#create' 
                        scope '/:todo_name' do 
                            get '/' => 'api_todos#show' 
                            put '/' => 'api_todos#update'
                        end 
                    end 
                end 
            end 
        end 
    end 
end

4.1.1 Naming

While Rails will let you use PUT and POST more or less interchangeably, your API will be consumed by other developers. As such, it is best practice to go for the principle of least surprise – POST for create, PUT for update, PATCH for upsert (update and insert). Another reason to use the correct methods is that your application will be maintained by developers who are not you, possibly long after you are gone. Using methods which conform to standard practice (index, show, create, and update) ensures that this is a simpler task.

4.1.3 Versioning

Once your API is exposed, you need to assume that somebody is consuming it. Therefore, an existing API should never be modified, except for critical bugfixes. Rather than changing existing endpoints, expose a new version. One way to do this might be to create versioned controllers and routes (as above). With comprehensive test coverage, backwards compatibility can be ensured. A corollary of the above is that one should only expose an API that has gone through rigorous internal testing.

4.1.4 Route Parameters

While it is possible to identify routes based on the id column of the targeted resource, this does presume a higher degree of knowledge by the consuming application. A deliberate tradeoff — that needs to be made on a case by case basis — using unique database ids in the route chain allows users to access short routes, and simplifies resource lookup, while exposing internal database ids to the consumer and requiring the consumer to maintain a reference to ids on their end. Using public, but possibly not unique, identifiers like name reduces the amount of system internals that needs to be exposed, while allowing the client to easily lookup any data needed to make a request. The downfall is longer nested routes.

4.2 Controllers

4.2.1 BaseController and Authentication

A base API controller is useful to handle authentication and extract common API functionality. There are many possible schemes, but a common approach is to require reauthentication on a per-request level. This is probably the simplest way to ensure statelessness.

javascriptclass BaseApiController < ApplicationController    
    before_filter :parse_request, :authenticate_user_from_token! 
    private def authenticate_user_from_token! 
        if !@json['api_token'] 
            render nothing: true, status: :unauthorized 
        else 
            @user = nil User.find_each do |u| 
                if Devise.secure_compare(u.api_token, @json['api_token']) 
                    @user = u 
                end 
            end 
        end 
    end 
    def parse_request 
        @json = JSON.parse(request.body.read) 
    end 
end

4.2.1.1 Security

Devise.secure_compare helps avoid timing attacks. While the comparison algorithm used by Devise is not strictly speaking constant time as it uses newly allocated memory and is capable of invoking the Garbage Collector as a result, it is much nearer constant time then custom comparison routines. Similarly, the Users loop does not break, thus preventing an attacker from establishing api token validity based on response time.

4.2.2 ProjectsController

javascriptclass ApiProjectsController < BaseApiController 
    before_filter :find_project, only: [:show, :update] 
    before_filter only: :create do 
        unless @json.has_key?('project') && @json['project'].responds_to?(:[]) && @json['project']['name'] 
            render nothing: true, status: :bad_request 
        end 
    end 

    before_filter only: :update do 
        unless @json.has_key?('project') 
            render nothing: true, status: :bad_request 
        end 
    end 

    before_filter only: :create do 
        @project = Project.find_by_name(@json['project']['name']) 
    end

    def index 
        render json: Project.where('owner_id = ?', @user.id) 
    end 

    def show 
        render json: @project 
    end 

    def create 
        if @project.present? 
            render nothing: true, status: :conflict     
        else 
            @project = Project.new
            @project.assign_attributes(@json['project'] 

            if @project.save 
                render json: @project 
            else 
                render nothing: true, status: :bad_request 
            end 
        end 
    end 

    def update 
        @project.assign_attributes(@json['project']) 
        if @project.save 
            render json: @project 
        else 
            render nothing: true, status: :bad_request 
        end 
    end 

    private 

    def find_project 
        @project = Project.find_by_name(params[:name]) 
        render nothing: true, status: :not_found unless @project.present? && @project.user == @user 
    end 
end

The todos controller will look similar, even having to find a project in order to fulfill the RESTfulness requirement (one uniquely identifiable resource per endpoint).

4.2.2.1 Defensive Programming

Defensive programming is a software design principle that dictates that a piece of software should be designed to continue functioning in unforeseen circumstances. Because your API will be exposed to third-party developers, allowing them to submit arbitrary inputs, it is important to apply this practice in API design.

This is why has_key? is used above rather than simple existence checking. If @json['project'] came in as false, or null, a simple existence check would return false, even though we can not say with any degree of accuracy that no JSON request body will ever have a false node at a depth of one.

The more significant defensive choice was the inclusion of responds_to?(:[]). This is due to the fact that, if the request body were say {"project": "noteIAmNotAnObject"}, @json['project']['name'] would result in a server error. In the spirit of defensive programming, when building APIs, ask yourself, "what is the least expected, most random, or most malicious input a user can submit?" Then write code to handle it.

4.2.2.2 HTTP Status Codes

While it is tempting to simply return 200 (OK), 404 (Not Found), and 500 (Internal Server Error), HTTP status codes — very much like HTTP verbs — have well-defined meanings. For the benefit of other developers, use status codes that make sense. For example, if we have a uniqueness constraint on Project name, we should return a conflict status on creation attempt for name clashes to let the developers understand where they went wrong.

The exception to the rule is when an item exists but the API user does not have the right access privileges to act on it. While it might be tempting to return a 403 (Forbidden), this in and of itself provides an attacker with information about the existence of the item. Instead, return a 404 if the user cannot access the record, no matter what the reason.

4.2.2.3 Code DRY?

While the above does achieve the desired result, it is far from DRY. There are repeated JSON validations and a duplicate database read, once for read only on update, and once for update rejection on a non existing item for create. Additionally, any nested route, such as /api/v1/projects/:name/todos will require the same find methods so as to uniquely identify the correct resource.

5 A better way

First, we'll extract functionality common to all API endpoints into the BaseApiController.

javascriptdef validate_json(condition) 
    unless condition 
        render nothing: true, status: :bad_request 
    end 
end 

def update_values(ivar, attributes) 
    instance_variable_get(ivar).assign_attributes(attributes) 
    if instance_variable_get(ivar).save 
        render nothing: true, status: :ok 
    else 
        render nothing: true, status: :bad_request 
    end 
end 

def check_existence(ivar, object, finder)   
    instance_variable_set(ivar, instance_eval(object+"."+finder)) 
end

This turns the create and update methods into calls to update_values, while the Project JSON validations call validate_json:

javascriptbefore_filter only: :create do |c| 
    meth = c.method(:validate_json) 
    meth.call (@json.has_key?('project') && @json['project'].responds_to?(:[]) && @json['project']['name']) 
end 

before_filter only: :update do |c| 
    meth = c.method(:validate_json) 
    meth.call (@json.has_key?('project')) 
end 

before_filter only: :create do |c| 
    meth = c.method(:check_existence)    
    meth.call(@project, "Project", "find_by_name(@json['project']['name'])" 
end 

def create 
    if @project.present? 
        render nothing: true, status: :conflict 
    else 
        @project = Project.new 
        update_values :@project, @json['project'] 
    end 
end

The other, possibly more significant improvement is introducing a multi-tiered inheritance hierarchy into your controllers. A good way to break this out is based on route nesting. Todos are nested within Projects; therefore, any Todo will need to find a Project.

javascriptclass ApiProjectRouteController < BaseApiController 
    private 

    def find_project 
        @project = Project.find_by_name(params[:name]) 
        render nothing: true, status: :not_found unless @project.present? && @project.user == @user 
    end

    def find_todo 
        @todo = #... end #other finders also go here. 
    end
end

ApiProjectsController now needs to inherit from ApiProjectRouteController rather then BaseApiController.

javascriptclass ApiProjectsController < ApiProjectRouteController 
    #... 
end 

class ApiTodosController < ApiProjectRouteController     
    #...
end

6 Custom behavior

Unfortunately, APIs do not exist in a vacuum. The system will have its own set of predefined behaviors, and on occasion, despite the best of intentions it will be necessary to expose behavior through the API that is not available otherwise, or may even clash with the application's existing behavior. A specific, fairly common example would be the relaxing of validations.

For the sake of clarity, let us walk through an example. Let's assume that our system's Project model has a priority column that is required.

javascriptclass Project < ActiveRecord::Base 
    has_many :todos 
    belongs_to :user 

    validates :priority, presence: :true 
    validates :name, presence: :true 
end

Now let us assume that a major customer would like to integrate an in-house solution that allows Project Managers to bulk create Project entries without priority, as priorities have not yet been decided. However, the regular UI users should not gain these increased privileges. How could this be achieved? In short, we need the model to be aware of the request source, and apply one set of validations in case the request came from the API, and another otherwise.

The first thing to do, is to add this concept at the controller level. Adding a method to the BaseApiController:

javascriptbefore_filter :indicate_source 
def indicate_source 
    @api = true 
end

The second thing to do is to allow the model to validate conditionally.

javascriptclass Project < ActiveRecord::Base 
    after_initialize :set_ivars 
    has_many :todos 
    belongs_to :user 
    validates :priority, presence: :true, if: lambda
    { |model| model.instance_variable_get(:@strict_priority_validation)}
    validates :name, presence: :true 

    private 
    def set_ivars 
        @strict_priority_validation = true 
    end 
end

Now every time a model is instantiated it will set its validations to be strict, and apply the priority validation if strictness is true.

The final piece of the puzzle is sharing state between the controller and the model to override model level instance variables. To do this, we can define a PORO (Plain Old Ruby Object):

javascriptclass ProjectValidationPicker 
    def pick_a_validation(project, api=false) 
        if api 
            project.instance_variable_set(:@strict_priority_validation, false) 
        else     
            project.instance_variable_set(:@strict_priority_validation, true) 
        end 
    end 
end

And for the final piece of the puzzle, this object needs to be used on update/create. Handily, we already have a function on BaseApiController to do just that. Lets change this method to pull in a validation picker if it exists and to set the api appropriately.

javascript
def update_values(ivar, attributes) 
    instance_variable_get(ivar).assign_attributes(attributes) 
    validation_picker = check_validation_picker_existence(ivar) 
    validation_picker.send(:new).pick_a_validation(instance_variable_get(ivar), @api) 

    if validation_picker 
        if instance_variable_get(ivar).save 
            render nothing: true, status: :ok 
        else 
            render nothing: true, status: :bad_request 
        end 
    end 

    def check_validation_picker_existence(ivar)    
        Module.const_get (ivar.to_s[1..-1] + "_validation_picker").camelize rescue false 
    end
end

7 Debugging

Unfortunately, no code is ever perfect the first time it is written. While a detailed analysis of either of these tools is outside the scope of this article, it is worthy of note that both the Unix curl command, and the Google Apps Postman REST client allow crafting of custom requests to arbitrary endpoints.

8 Testing

With the code above, we have created a flexible, DRY, RESTful API. It allows us to have behavior that is unique to the API, while maintaining the same Database Models. But how can this be tested? Since the bulk of the lifting is done at the controller level, and since every request is a stateless transaction, request level specs are the most obvious fit. The simplest way to achieve this is controller tests, which do not differ significantly from the usual model for controller endpoint tests. Fundamentally, we craft JSON requests, and send them to the endpoints. A simple example might look something like this:

javascriptdescribe ApiProjectsController do 
    before do 
        @base_json = { api_token: @user.api_token } 
        @project_json = {project: {priority: "4", name: "foo"}} 
        @new_project_json = @base_json.merge(@project_json) 
    end 

    describe "actions" do 
        describe "#create" do 
            before do 
                @request.env['RAW_POST_DATA'] = @new_project_json.to_json 
                lambda do 
                    post :create 
                    end.should change(Project, :count).by(1) 
            end 
            it "returns ok" do 
                expect(response.status).to eq(200) 
                expect(Project.all.last.priority).to eq(@new_project_json[:priority]) 
                expect(Project.all.last.name).to eq(@new_project_json[:name]) 
            end 
        end 
    end 
end

9 Other approaches and considerations

It is worth mentioning that the code outlined above represents only one possible approach to building a RESTful API in a Rails app. In fact it has made two, relatively significant tradeoffs.

The first, is that every request requires its own authentication. It is possible to avoid making this sacrifice using session cookies which Devise can also generate. I would argue that this complicates third party integration as it requires the developer to keep track of the session cookie, passing it in with every request instead of the api_token, as well as handling session expiration. In addition, it leaves the system exposed to attack, if the cookie origin is not scrutinized sufficiently, or even if the API consumer stands up to go to the restroom.

The second is that the in-house UI does not by default hit the same API endpoints. One solution would be to have Devise generate and expire a secondary, UI-only auth token, which does not trigger the API validation (maintains @api as false in BaseApiController), this may be stored in another column, or a different datastore entirely. Non-persistent in-memory datastores like Redis are particularly well suited to this application.

Finally it is worth noting that while this demo simply returns the entire resource, it is best practice to use a serializer to limit the subset of keys returned to what it is safe and/or necessary to expose.

10 Summary

We have covered how to build a REST api in an existing Rails application from the ground up, how to expose the endpoints, how to route to them, and how to allow custom behavior. We have touched on testing and debugging. While the example was fairly trivial, it is my hope that you can use it as a template for building scalable, reusable APIs.

11 Acknowledgements

A large thank you to my friend Joshua Yanovski who proofread this article and noticed the problem with Devise.secure_compare. He can be found at https://github.com/pythonesque