Integrating Controllers, jQuery, JSONP, and Remote Domain Requests with Rails

July 2nd, 2009
By: Steven Haddox
No Comments »

I’m developing an application that needs to have a normal web presence (typical HTTP/HTTPS) as well as a very easily consumable 3rd party API. With these requirements in mind I easily decided to run with Ruby on Rails (.to_json anyone?) I’ve been working on it for over a month now and have encountered a situation where my controllers are getting a little large due to having to handle logic on whether the request via the API url was a Javascript AJAX requests or a regular HTTP request. Generally this is easily solved in the respond to block with something like:

respond_to do |format|
  format.html # reports/show.rhtml
  format.js { render :partial => 'reports/show' } # reports/_show.rhtml
end

This example is just rendering a partial that is also used in my show.rhtml to save updating more than one file at a time. However, my current project started turning into things like this (due to the API methodology we had decided upon with the client in advance that got extended into doing things we hadn’t anticipated):

# show method
respond_to do |format|
  format.html # reports/show.rhtml
  format.json { render :json => @report } # json format of the report data
  format.js { render :partial => 'reports/show' } # reports/_show.rhtml
  format.js_vdata {
    render :template => 'reports/show.visual_json', :layout => false, :content_type => 'application/json'
  }
  format.table {
    render :template => 'reports/show.html_table', :layout => false, :content_type => 'application/json'
  }
  ... [more formats for special MIME types here with customized views all from the same report data] ...
end

As you can see this code was being extended to generate multiple kinds of reports for loading via an API with AJAX requests. The extra formats are all being returned as JSON in order to allow for JSONP and remote-access from other domains and then the user can display the requested data / customize it as they desire. But how did I go about providing those same views to users on the web application itself? Create another format just to render the same view for regular HTTP requests? Definitely not very DRY. So it occurred to me that the following might be a possible solution:

#Controller show method
respond_to do |format|
  ...
  format.js_vdata {
    if request.xhr?
      render :template => 'reports/show.visual_json', :layout => false, :content_type => 'application/json' and return
    else
      render :template => 'reports/show.visual_json', :layout => true, :content_type => 'text/html' and return
    end
  }
  format.table {
    if request.xhr?
      render :template => 'reports/show.html_table', :layout => false, :content_type => 'application/json' and return
    else
      render :template => 'reports/show.html_table', :layout => true, :content_type => 'text/html' and return
    end
  }
end

#View template for reports/show.visual_json
< %- if request.xhr? -%>
  < %= @report.visual_data.to_json %>
< %- else -%>
  
  < %=debug @report.visual_data %>
< %- end -%>

Although this was a basic proof of concept there was already a problem with it. Any request coming from a remote domain would send requests for ‘jsonp’ and would not be recognized by request.xhr? (at least with jQuery 1.2.6 on the server we were consuming this API). I’m not sure why this was, but it seemed to be the situation I had to work with. This meant those requests were getting the HTML version of the page (not fun for parsing via Javascript). The solution I found was this (please note this is a very initial stab and is more than open to refactoring suggestions):

#Controller
before_filter :set_request_type

#show method
respond_to do |format|
  ...
  format.js_vdata {
    begin
      render_dynamic_template('reports/show.visual_json')
    else
      render_dynamic_template('reports/show.nil')
    end
  }
  format.table {
    begin
      render_dynamic_template('reports/show.html_table')
    else
      render_dynamic_template('reports/show.nil')
    end
  }
end

private
  def set_request_type
    @ajax_request = true if request.xhr? or (params[:callback] && params[:callback].include?('jsonp'))
  end

  def render_dynamic_template(template)
    template = 'reports/show.nil' if template.nil?
    if @ajax_request
      layout = false
      content_type = 'application/json'
    else
      layout = true
      content_type = 'text/html'
    end
    render :template => template, :layout => layout, :content_type => content_type
  end
#View template for reports/show.visual_json
< %- if @ajax_request -%>
  < %= @report.visual_data.to_json %>
< %- else -%>
  
  < %=debug @report.visual_data %>
< %- end -%>
#View template for reports/show.nil
< %- if @ajax_request -%>
  < %= [nil].to_json %>
< %- else -%>
  
  < %= 'There is no data at this time.' %>
< %- end -%>

Also, the best tutorial I could find for implementing jsonp support with a quick (like 2 minute) Google search was over at Sitepoint

And that should allow you to render both html and json/jsonp results based upon the kind of request coming in. Obviously it makes an assumption that remote jsonp requests will provide a parameter of ‘callback’ that includes ‘jsonp’ in the string, but given that it is your API that’s not a huge requirement to impose and you can customize it to how you see fit.

If you know of a better way to go about approaching this kind of task I’m more than open to being educated, but given the short time-frame I had and the few tutorials specific to jsonp I found this to be a pretty good first stab. Hopefully this post will help someone else if they find themselves looking for some way to accomplish similar tasks.

Rails and large data strings in MySQL (longtext)

June 6th, 2009
By: Steven Haddox
1 Comment »

During my day-job this week I came across a situation where we had nice functioning code very near to our launch of a Rails application. We had just completed migrating over some historical data from another database (have I mentioned how much I love rake in the past, if not I may have to do a blog post about that too). Anyway, the data all migrated over in a matter of minutes and after invoking a few more custom rake tasks that were simple to write we had every record in the exact same position as it was originally.

The cozy aura of accomplishment had settled upon our little team when we soon discovered that all was not well in Rails-land. You see, we are processing data at defined intervals (on a per-record basis by an attribute on the object). We have a cron job setup to invoke a rake task that checks and sees if each record needs to be updated or not (I love .minutes.ago and built-in timestamp fields btw). Then the rake task fires off the method call to update with the new data as often as needed. After the update is complete we parse the new data and create a JSON string that is stored in the database (for performance over parsing every time we want to display it). We also do a combination of all the past data sets we’ve received (one for 24 and 72 hours each) and store those in JSON format to the database as well. This worked out great on all our sample instances where our objects didn’t have hundreds of data sets being merged and combined into one long JSON string.

However, once we get some live data our application started blowing up rather quickly. We came across an ‘rbuff’ error (or something weird and very vague in its meaning) and spent a little bit of time realizing that the string we were trying to write to the database was more than 64KB. We had set our fields to ‘text’ and it was truncating our data (and hence blowing up all our JSON.parse methods to get our strings back into Ruby hashes). The fix seemed fairly easy, we just had to change our column types from ‘text’ to ‘longtext’.

If you do a Google search you’ll find several ways to have MySQL create a column in the table with a longtext type in Rails. However, none of them worked for me (we tried :text, :limit => huge_number as well as :text, :limit => 64K+1, and several other methods we found online). I couldn’t find any documentation to indicate that Rails had added in support for :longtext as an actual column type as it seems to be MySQL specific. My pair suggested (for the 10th time) that we just put :longtext in our migration file instead of :text and see if it worked. I’m not as much of an off-the-cuff kind of guy as he is and I was skeptical it would work as so many blogs said to do it the ways we had been trying. Well I finally gave in and said, “Okay, well try it your way, but I bet you’re wrong.” We edited the migration to:

change_column :table, :column_name, :longtext

rake db:migrate and all of the sudden our MySQL table was showing a column type of ‘longtext’ as we had wanted all along. Needless to say I was rather embarrassed that this wasn’t better documented and I lost the bet to my pair. My question now is, does anyone know if :longtext as a column type is supported on other database types or is tied directly to MySQL with Rails? I can’t seem to find any documentation for this as a column type anywhere so I’m not sure if it is new to Rails 2.3.2 or if it just has been overlooked in the docs. As the searches I performed hadn’t brought anything back of any use for migrating my column to longtext I figured I should try and make sure I at least get a blog up about it to help anyone else who may be in a similar situation.

Katie’s Amazing Writing, Publication, and Some Family Love

March 7th, 2009
By: Steven Haddox
No Comments »

Katie has had a busy week.  She’s had to deal with a financial roller coaster from a publisher, creating several felt fruit projects, mentally prepare for turning 21, take care of a very emotionally uncertain 17-month old, and discovered that her font was actually published along with her layout.  We also got to take some time and do things together as a family including a fun walk around the neighborhood. Read the rest of this entry »

Twitter

February 28th, 2009
By: Steven Haddox
No Comments »

So I’ve had several friends and coworkers join Twitter or ask about Twitter as of late. As a result I decided to write a quick blog from my #iPhone while I’m watching an episode of #SVU and laptopless.

First things first, several different sites use different terminology about Twitter. From what I can tell the most commonly used terminology is as follows:

Tweet: A new message posted to Twitter (eg: ‘That tweet by @sunnythaper was hilarious!’)
Retweet (RT): Prefixing ‘RT’ or ‘Retweet’ in front of a tweet of importance to you allows you to give credit to the original author (eg: RT @sunnythaper: Nobody’s beard is as cool as mine. Not even @santaclaus).
Overheard (OH): If something is particularly hilarious to you that someone else said you can overhear it (eg: OH: @sunnythaper, unless it’s about the KKK on twitter I’m good!)
@username: Prefixing a username with the @ symbol (usually at the beginning of the tweet) is a public reply in response to another person’s tweet (eg: @sunnythaper Are we still on for sushi next Tuesday?)
Direct Message (DM): A direct message is used to send a private message to another user. To send a direct message to a user prefix the username with a ‘d’ and a space (eg: d sunnythaper Do you mind if I use your Twitter handle for examples in a blog post about Twitter?)

Once you get the basic usage down you’ll find that Twitter can be very useful, relaxing, hilarious, time-consuming, productive, or any combination of the above. No matter what how you use Twitter you may want to find easier ways to interface with it than just twitter.com/home. Personally I use Twitterific on my Macs and iPhone (although I use TwitFon for my second phone application and I’ve heard Tweetie is amazing on the iPhone). There are tons of desktop clients but Tweetdeck seems to be the most common for cross-platform use as it runs on Adobe Air and is very customizable. You’ll probably want to look around and see what works best for you.

So what’s the point in investing all this time into learning all of these use-cases, learning the new terminology, and picking up yet another social networking tool? That’s entirely up to you. Twitter is what you make of it. For me Twitter is a way to keep in touch with my friends and find new ones (real or virtual). It’s a way to share and get ideas for activities and projects. It’s a way to follow great thinkers in web development and hence the newest trends. It’s a way to help others and in turn seek help on that stupid CSS bug I’m banging my head against the desk on.

What’s with the #hashtags earlier in the post you ask? Well hashtags are just one of the cool things about Twitter. They aren’t actually part of Twitter itself, but like all good things on the web as of late, they extend Twitter’s API over at www.hashtags.org and show you all tweets from users that have the same hashtag. Many third party Twitter clients support hashtags and automatically link to the appropriate URL to show you other tweets.

If you’re a developer you can find out how to extend Twitter by looking at their API.

Hopefully this post has helped to serve as a very basic introduction to Twitter and how it can be of use to you. If you haven’t yet, you’re more than welcome to follow me at: http://www.twitter.com/stevenhaddox

Also a special thanks to Sunny (http://www.twitter.com/sunnythaper) for letting me use his handle in my examples. If you’re looking for someone humorous to follow, he’s even more funny than I am serious ;)

2, 12, 15, 20, 26, 75

December 7th, 2008
By: Steven Haddox
1 Comment »

Some of you may know that I’m a huge fan of LOST (thanks to Katie).  If you don’t, you do now.  I was sitting here trying to figure out a quick way to blog about the month of December as I know I don’t have time to blog as much as I should and I realized that numbers are pretty much a representation of how this month is for me.  So here comes the explanations:

  • 2: The number of years Katie and I have been married (as of December 2, 2008).
  • 12: The number of hours I am trying to work on average each day to earn some savings for possible unemployment come January 1, 2009.
  • 15: Paisley’s age in months currently.
  • 20: Katie’s age in years currently.
  • 26: My age in years (as of yesterday).
  • 75: The number of lightbulbs we replaced on our Candy Land tree in order to get one of the two missing strands of lights to work again this year.

I’ll take a quick moment to go over some of these in more details.  Read the rest of this entry »