Remote JSON - JSONP

原文:Remote JSON - JSONP

 

The browser security model dictates that XMLHttpRequest, frames, etc. must have the same domain in order to communicate. That's not a terrible idea, for security reasons, but it sure does make distributed (service oriented, mash-up, whatever it's called this week) web development suck.

There are traditionally three solutions to solving this problem.

Local proxy:

Needs infrastructure (can't run a serverless client) and you get double-taxed on bandwidth and latency (remote - proxy - client).

Flash:

Remote host needs to deploy a crossdomain.xml file, Flash is relatively proprietary and opaque to use, requires learning a one-off moving target programming langage.

Script tag:

Difficult to know when the content is available, no standard methodology, can be considered a "security risk".

---

I'm proposing a new technology agnostic standard methodology for the script tag method for cross-domain data fetching: JSON with Padding , or simply JSONP .

The way JSONP works is simple, but requires a little bit of server-side cooperation. Basically, the idea is that you let the client decide on a small chunk of arbitrary text to prepend to the JSON document, and you wrap it in parentheses to create a valid JavaScript document (and possibly a valid function call).

The client decides on the arbitrary prepended text by using a query argument named jsonp with the text to prepend. Simple! With an empty jsonp argument, the result document is simply JSON wrapped in parentheses.

Let's take the del.icio.us JSON API as an example. This API has a "script tag" variant that looks like this:

http://del.icio.us/feeds/json/bob/mochikit+interpreter :

if
(typeof(Delicious) == 'undefined'
) Delicious = {};

Delicious.posts = [{

    "u"
: "http://mochikit.com/examples/interpreter/index.html"
,

    "d"
: "Interpreter - JavaScript Interactive Interpreter"
,

    "t"
: [

        "mochikit"
,"webdev"
,"tool"
,"tools"
,

        "javascript"
,"interactive"
,"interpreter"
,"repl"


    ]

}]

In terms of JSONP, a document semantically identical to this would be available at the following URL:

http://del.icio.us/feeds/json/bob/mochikit+interpreter?jsonp=if(typeof(Delicious)%3D%3D%27undefined%27)Delicious%3D%7B%7D%3BDelicious.posts%3D

That's not very interesting on its own, but let's say I wanted to be notified when the document is available. I could come up with a little system for tracking them:

var delicious_callbacks = {};

function getDelicious(callback, url) {

    var uid = (new Date()).getTime();

    delicious_callbacks[uid] = function () {

        delete delicious_callbacks[uid];

        callback();

    };

    url += "?jsonp="
 + encodeURIComponent("delicious_callbacks["
 + uid + "]"
);

    // add the script tag to the document, cross fingers

};



getDelicious(doSomething, "http://del.icio.us/feeds/json/bob/mochikit+interpreter"
);

The fetched URL from this hypothetical experiment would look something like this:

http://del.icio.us/feeds/json/bob/mochikit+interpreter?jsonp=delicious_callbacks%5B12345%5D

delicious_callbacks[12345]([{

    "u"
: "http://mochikit.com/examples/interpreter/index.html"
,

    "d"
: "Interpreter - JavaScript Interactive Interpreter"
,

    "t"
: [

        "mochikit"
,"webdev"
,"tool"
,"tools"
,

        "javascript"
,"interactive"
,"interpreter"
,"repl"


    ]

}])

See, because we're wrapping with parentheses, a JSONP request can translate into a function call or a plain old JSON literal. All the server needs to do differently is prepend a little bit of text to the beginning and wrap the JSON in parentheses!

Now, of course, you'd have libraries like MochiKit , Dojo , etc. abstracting JSONP so that you don't have to write the ugly DOM script tag insertion yourself, etc.

Of course, this just solves the standardization problem. Your page is still toast if the remote host decides to inject malicious code instead of JSON data. However, if implemented, it'd save a lot of developers some time and allow for generic abstractions, tutorials, and documentation to be built.

15 Comments »

1. I implemented something similar in Tagneto (the SvrScript.js library). The spec I used for the implementation allows for an success and failure callback as well as allowing “multipart” SCRIPT SRC requests when the request is bigger than what will fit in one GET request. See tagneto.org/how/reference/js/DynamicScriptRequest.html

   Comment by James — 2005-12-12 @ 9:46 pm

2. It looks like Yahoo! agrees with your (sage, IMO) suggestion: http://ws1.inf.scd.yahoo.com/common/json.html#callbackparam

   Mike

   Comment by Mike Shaver — 2005-12-18 @ 9:56 am

3. Hi Bob,

   Great idea! A few days after your post Yahoo made public something quite similar to (but maybe less generic than) what you’re proposing above as part of their Web Services.

   http://www.theurer.cc/blog/2005/12/15/web-services-json-dump-your-proxy/

   = tmk =

   Comment by tmk — 2005-12-18 @ 8:34 pm

4. The callback implementation that Yahoo! deployed is *almost* JSONP, but not quite. The difference is that callback filters out anything past the first identifier given in callback, so you don’t have the opportunity to throw in an “del.icio.us style” assignment. Originally I had thought to do exactly what Yahoo! did, but I didn’t see a reason not to just let arbitrary text get prepended because it allows for compatibillity with legacy API, and simplicity in some situations.

   Comment by bob — 2005-12-26 @ 6:28 pm

5. You describe ActionScript (the development language used for the Flash VM as:

   >a one-off moving target programming langage

   Just an fyi, but ActionScript is based on ECMA script. The current version is not 100% compatible (although very close). The next version aims to be 100% compatible with the next version of ECMA script.

   Just an fyi…

   mike chambers

   mesh@adobe.com

   Comment by mike chambers — 2006-01-12 @ 11:09 am

6. I like the idea very much, just added support for it to an RDF store (http://www.appmosphere.com/en-arc_rdf_store).

   cheers,
   bnj

   Comment by Benjamin Nowack — 2006-02-20 @ 9:25 am

7. This is a much needed feature.

   I was looking at ways to integrate my del.icio.us (which has some JSON support) categories in my blogger blog, and I had the exact same idea.

   Now being a beginner in JS, I have what might be a stupid question : isn’t it possible to build a bridge server that supports JSONP and forward the JSON calls?

   regards,
   nicolas rolland
   http://technofinance.blogspot.com

   Comment by nicolas rolland — 2006-05-26 @ 8:33 am

8. Yes, of course it is possible to build a proxy.

   Comment by bob — 2006-05-26 @ 11:52 am

9. I am not completely clear about this. How does it stay compatible with existing APIs? You seem to use it with del.icio.us without any updates.

   And the point of the parentheses is so that the same API can be called via regular XMLHttpRequest, and the data saved via assignment, or from an offsite page via a script tag, and the data saved via the callback? I guess the parentheses is the thing I really don’t get.

   Would it not be more safe to have the server call a pre-defined function name only? I am thinking about a situation where an attacker embeds some malicious call-back code in a GET url and includes it in an image tag or something like that. Maybe this is no different than including any old offsite .js file.

   Anyway…

   Comment by Evan — 2006-06-07 @ 10:55 am

10. It’s a new API, that del.icio.us does not support. It was a hypothetical example.

    The point of the parentheses is that it’s not valid JavaScript without them, and you need valid JavaScript for a script tag. XMLHttpRequest is irrelevant, because you would use regular JSON for that.

    I think you’re confused about the security implications. It’s definitely not an attack vector for the server because all it does is concatenate strings. If there’s already malicious code on the client, this is totally irrelevant.

    The only attack vector is stealing cookies from the JSONP-hosting server, which is why yahoo’s version doesn’t allow anything but a single identifier to be used as the callback.

    Comment by bob — 2006-06-07 @ 1:47 pm

11. Completely unmoderated javascript code injection (providing code containing any characters) is not a good idea, but what level of restraint would be? There is call some level of callback layout restrictions, and I would like to standardize a good practice, which is also ideally trivial to implement for JSONP providers.

    Forbidding parentheses goes a long way, but it still allows you to steal document.cookie by storing it in a variable. Forbidding = too slightly harms the most trivial JSONP consumption model (callback “my_variable=”). Yahoo seems to have set the bar for themselves on limiting to US-ASCII alphanumerics, underscore, period and square brackets. There is some merit to that; for one thing it skips scenarios with unbalanced quotation marks in the callback name. The outcome is at least useful enough.

    I would find it useful if JSONP appeared as an informal standard (IETF RFC, informational, for instance), stating good practices, conformance criteria, and so on, simply to have a document to refer implementors and service providers to, without forcing a high level of understanding of the domain on them. I think I more or less volunteer to writing it, but I would like you to be on board, so to speak, as inventor, originator or similar, even if you do not pull the actual weight.

    It would mostly be something to cover up the hole until Crockford’s JSONRequest is somewhere near as deployed as XMLHttpRequest is today, but that will take some time, and JSONP works now. It could just use a little help along the way, here and there.

    Comment by Johan Sundström — 2007-02-12 @ 5:54 am

12. While the call back param option is really nice, we need to be really careful while operating of sensitive JSON data for mash-ups. This is exactly what caused the gmail contact list XSRF hole. Isn’t it? http://jeremiahgrossman.blogspot.com/2007/01/gmail-xsrf-json-call-back-hackery.html

    Comment by Kishore Senji — 2007-06-04 @ 12:42 pm

13. No, it’s completely different than the XSRF hole in gmail.

    Even if it was, the whole point of JSONP is cross-site requests anyway… so it’s not really forgery nor a hole.

    Comment by bob — 2007-06-04 @ 1:43 pm

14. I have written json file which i want to pass to a javascript function on client side.
    Basically i am trying for callback of the function in my url at client side.
    It is not working can u please help me with the settings for callback.

    Comment by derek — 2008-10-15 @ 5:30 am

15. alert(document.cookie)

    Comment by Sundar — 2009-02-02 @ 8:09 pm