Adding a "live" data visualization to Rails with d3.js

I’ve wanted to add an admin dashboard to a Rails project so I could easily see a summary of some key datapoints. There’s a few options for adding data visualizations to a Rails app, but I wanted something that would continuously update. So, if the there are changes in the app’s database, I want the graphs to automatically update without the need to refresh the page.

One way to achieve this is with the JavaScript library d3.js and AJAX. It took me some time to get this working, so I figured I’d share the process in a barebones Rails project in case someone else is trying to something similar. All of the code is available in a GitHub repository. The master branch contains the end product, and other branches in the project correspond to each of the steps in the following walkthrough.

By the end, we’ll have a simple graph that updates not only when we change the data, but when other people do, too.

example gif of d3 plot

You can see a live, hosted version here.

I’m assuming you are familiar with the basics of a Rails project and d3.js. Let’s get to it!

Step 1: Build the humble voting app

Before adding any graphs, let’s start with minimal Rails project: a simple voting application. There is just one page with two buttons (red and blue), and clicking a button creates a vote with a corresponding color attribute. After each click, the vote counts are updated. This version is available in the step-1-rails-app branch in the repository.

Our humble voting app, sans graph

Model

The app has a single model, Vote, with one attribute, color. Vote has a single class method Vote.totals, which returns a two-element array containing the count of the red and blue votes, respectively.

app/models/vote.rb

class Vote < ActiveRecord::Base

  def self.totals
    [Vote.where(color: 'red').count, Vote.where(color: 'blue').count]
  end
end

Database schema

Our database is simple: just a votes table with a color (string) attribute. I left timestamps in there, in case I’d want to graph some time-related data at some point. I’ve used PostgreSQL so I could easily push the project to Heroku later.

db/schema.rb

ActiveRecord::Schema.define(version: 20160501181648) do

  # These are extensions that must be enabled in order to support this database
  enable_extension "plpgsql"

  create_table "votes", force: :cascade do |t|
    t.string   "color"
    t.datetime "created_at", null: false
    t.datetime "updated_at", null: false
  end

end

Controller

The votes controller has two actions: index and create. Calling index will assign the return value of Vote.totals to an array @votes, which will be used to show the vote totals in the view. The create action will (1) create and save a new Vote instance to the database, (2) reassign the vote totals to @votes, and (3) call the JavaScript in views/votes/create.js.erb. More on that last part later.

app/controllers/votes_controller.rb

class VotesController < ApplicationController

  def index
    @votes = Vote.totals
  end

  def create
    Vote.create(vote_params)
    @votes = Vote.totals 
    respond_to do |format|
      format.js
    end
  end

  private

  def vote_params
    params.require(:vote).permit(:color)
  end

end

Routes

We only define routes for the root path and the index and create actions described above.

config/routes.rb

Rails.application.routes.draw do

  root 'votes#index'
  resources :votes, only: [:index, :create]

end

View

The index template only has two important parts so far: a counter section (#vote-counter) that displays the vote totals, and the voting buttons (#voting-box) that trigger the create action. The buttons aren’t really buttons, but styled hyperlinks that send a POST request and a color parameter with AJAX. This prevents the page from being refreshed after each request. The styling isn’t shown here, but you can see it in app/assets/stylesheets/votes.scss.

app/views/votes/index.html.erb

<div class="main-container">
  <h1>Red vs. Blue</h1>
  
  <div id="vote-counter">
    <h3>Vote count</h3> 
    <p>Red: <span id="red-count"><%= @votes[0] %></span></p>
    <p>Blue: <span id="blue-count"><%= @votes[1] %></span></p>
  </div>

  <div id="voting-box">
    <%= link_to 'Vote Red', votes_path(vote: {color: 'red'}), {remote: true, method: :post, class: 'vote-button red' } %>
    <%= link_to 'Vote Blue', votes_path(vote: {color: 'blue'}), {remote: true, method: :post, class: 'vote-button blue' } %>
  </div>
</div>

Lastly, the respond_to block in the create action will call a bit of JavaScript that updates the vote counts in the view after every create action (or button click).

app/views/votes/create.js.erb

$("#red-count").html("<%= @votes[0] %>");
$("#blue-count").html("<%= @votes[1] %>");

At this point, we have:

  • A page with buttons
  • The buttons create database records (votes)
  • Some JS that updates the page when records are created

Now let’s use d3.js to add a bar plot that visualizes the vote totals.

Step 2: Add a static bar plot

Before we can draw anything with d3.js, we have to add the library to our project. I found the example at Overfitted very helpful when doing this the first time, so you might want to check that out, too.

Start by getting the d3.js code and copying it to app/vendor/javascripts/d3.js. Next, add a require d3 statement to app/assets/javascripts/application.js like so:

app/assets/javascripts/application.js

//= require jquery
//= require jquery_ujs
//= require turbolinks
//= require d3
//= require_tree .

Now we can reference d3.js functions in our application’s JavaScript files.

Before drawing the bar plot, I added a placeholder SVG to the view, below the buttons. The d3.js code will modify this SVG to draw the plot.

app/views/votes/index.html.erb

<div id="plot-container">
  <svg id="plot"></svg>
</div>

To draw the plot, we need to:

  1. Render the data as JSON
  2. Fetch the JSON when the page loads
  3. Hand the JSON to d3.js
  4. Draw some elements on the page based on the JSON

To render the JSON, we’ll modify the index action in the controller.

app/controllers/votes_controller.rb

class VotesController < ApplicationController

def index
  @votes = Vote.totals

  respond_to do |format|
    format.html
    format.json { render json: @votes }
  end
end

Adding the respond_to block allows us to handle two kinds of requests. The format.html line specifies that if the client requests HTML, then we should return the HTML that matches the current action (index.html). This just repeats the default response of the controller. The format.json line specifies that if the client requests JSON, then we should render some JSON and return that, too. The block specifies what that JSON will be: @votes.

Now that our controller will have the data ready as JSON, we need to fetch it when the page loads. One way to do that is through an AJAX request (just as outlined in the Overfitted posted mentioned above). We can make this AJAX request with jQuery in votes.js:

app/assets/javascripts/votes.js

// ajax call to fetch json
var loadData = function(){
                $.ajax({
                  type: 'GET',
                  contentType: 'application/json; charset=utf-8',
                  url: '/votes',
                  dataType: 'json',
                  success: function(data){
                    drawBarPlot(data);
                  },
                  failure: function(result){
                    error();
                  }
                });
              };

function error() {
    console.log("Something went wrong!");
}

// draw bar plot
function drawBarPlot(data){};

// fetch data on page load
$(document).ready(function(){ 
  loadData(); 
});

The loadData function contains an AJAX request with some important details: the type of HTTP request (GET), the URL (/votes), the datatype (JSON), what to do if the request succeeds (call drawBarPlot with the result), and what to do if the request fails (call error).

drawBarPlot will contain our d3.js plot-drawing code, which will now have access to the data. Let’s fill that out now:

app/assets/javascripts/votes.js

// set plot parameters
var barWidth = 20;
var colors = ['red', 'blue'];
var plotHeight = 300;

// draw bar plot
function drawBarPlot(data){

  // define linear y-axis scale
  var yScale = d3.scale.linear()
                 .domain([0, d3.max(data)])
                 .range([0, (plotHeight - 50)]);

  d3.select("#plot")
    .selectAll("rect")
    .data(data)
    .enter()
    .append("rect")
    .attr("width", barWidth)
    .attr("height", function(d){ return yScale(d); })
    .attr("fill", function(d, i) {
        return colors[i];
    })
    .attr("x", function(d, i){
        return (i * 100) + 90; // horizontal location of bars
    })
    .attr("y", function(d){ 
        return plotHeight - yScale(d); // scale bars within plotting area
    });
}

I set some basic plotting parameters (barWidth, colors, and plotHeight) outside of the drawBarPlot function because we’ll want to access them elsewhere later. Inside drawBarPlot, there’s the definition of our y-axis scale (yScale), and the d3 call that selects the SVG element (#plot), binds the data, then sets various attributes.

The result is the simplest of barplots: a red bar and a blue bar with heights proportional to the vote counts, scaled to the size of the SVG. The $(document).ready at the bottom of votes.js ensures that plot is drawn as soon as the page loads. For the full project code at this stage, see the step-2-static-d3-plot branch in the repository.

Our humble voting app, with a static bar plot

So now we have a plot of the data, but it doesn’t respond to new votes. We’ll have to refresh the entire page to update the plot. So first let’s add some more JavaScript so the graph changes whenever we cast a vote.

Step 3: Add an update function

Each time we cast a vote and change the database, we want to re-fetch the data and update our barplot. To do the re-fetching, I added an updateData function to votes.js:

app/assets/javascripts/votes.js

// ajax call to get fresh json
var updateData = function(){
                  $.ajax({
                    type: 'GET',
                    contentType: 'application/json; charset=utf-8',
                    url: '/votes',
                    dataType: 'json',
                    success: function(data){
                      updateBarPlot(data);
                    },
                    failure: function(result){
                      error();
                    }
                  });
                };

This is almost identical to the loadData function—the only difference is what happens if the request is successful. Here, we’ll call a new function updateBarPlot instead of drawBarPlot, because we only want to change the existing plot, rather than re-drawing the entire thing. So let’s define updateBarPlot:

app/assets/javascripts/votes.js

// update height of bars
function updateBarPlot(data){
  
  var yScale = d3.scale.linear()
                 .domain([0, d3.max(data)])
                 .range([0, (plotHeight - 50)]);

  d3.select("#plot")
    .selectAll("rect")
    .data(data)
    .transition()
    .attr("height", function(d){ return yScale(d); })
    .attr("y", function(d){
        return plotHeight - yScale(d);
    });
}

This is similar to the drawBarPlot function, but it only modifies the height of the bars and leaves the remaining attributes as they are. After entering the new data, the bars will smoothly transition to their new heights.

Lastly, we need to make sure that updateData is called whenever we cast a new vote. I added a call to updateData in create.js.erb, which is already handling the updating of the vote counters after each vote.

app/views/votes/create.js.erb

$("#red-count").html("<%= @votes[0] %>");
$("#blue-count").html("<%= @votes[1] %>");
updateData(); // re-fetch JSON and update barplot

Now the bars change as soon immediately after each vote, and we don’t need to refresh the page! If we are the sole voter, then we could stop here. But what if this was a live application and someone else was voting while we were watching the page? Their page would update, but ours would not. We’d have to vote again or refresh the page to update the plot with other users’ votes. Try it yourself by hosting the project locally and opening the page in two separate browser tabs.

Again, you can see the code for this stage at the step-3-update-function branch.

Step 4: Add background updating

One way to synchronize the plot across clients is to regularly update the data, regardless of whether any client voted. We can set a loop to re-fetch the data every few seconds and update relevant page elements: the plot and the vote counters above the buttons.

First, let’s add a function to update the vote counters:

app/assets/javascripts/votes.js

// update vote counters 
function updateVoteCounters(data){
  $("#red-count").html(data[0]);
  $("#blue-count").html(data[1]);
}

Just below that, add a function updatePage that will take care of updating the counters and the plot:

app/assets/javascripts/votes.js

// update page (plot and counters)
function updatePage(data){
  updateBarPlot(data);
  updateVoteCounters(data);
}

Now we’ll want to call updatePage after we re-fetch the data, so we’ll have to modify our updateData function. The only thing we need to change here is the call to updatePage after a successful request.

app/assets/javascripts/votes.js

var updateData = function(){
                  $.ajax({
                    type: 'GET',
                    contentType: 'application/json; charset=utf-8',
                    url: '/votes',
                    dataType: 'json',
                    success: function(data){
                      updatePage(data); // used to be updateBarPlot(data)
                    },
                    failure: function(result){
                      error();
                    }
                  });
                };

And for the final step, create a loop that will call updateData every 3000 milliseconds:

app/assets/javascripts/votes.js

$(document).ready(function(){ 
  loadData(); // initial fetching of data 

  setInterval(function(){
    updateData();
  }, 3000); // call updateData every 3000 ms
});

Now if you open the project in two browsers, you’ll see that they sync up every three seconds. You can see the full code on the step-4-update-loop branch.

Conclusion

That’s it! In case you’d just like to see a live example of this without writing it yourself, I pushed the master branch to Heroku: Red vs Blue. Open the page up in separate tabs or devices to see the updating in action.

Now I’ll admit that I fudged this a little. The visualization is not truly live—it’s making HTTP requests every few seconds, not constantly streaming the data in. But this approach will work just fine for some purposes - like an admin dashboard where I’m the sole user, but the underlying data may be changing.

My d3 plot was as barebones as possible to keep things simple, but if you are interested in learning more about d3.js, I found these sources very helpful: