So it turns out my first attempt to a proposal for safe jsonp failed. Not so much failed as that vendors won't consider it, but more failed in the sense that I had trouble explaining it to my peers. I haven't even bothered with vendors yet. So, let me try this again :) My discussions about this proposal at least haven't changed my mind. I still think it's a good idea and I still believe it will result in a safe, clean and simple way of doing jsonp.
I propose a new host object with a very simple api supplied by the browser (hence, host object) that allows you to fetch jsonp (not json) with the same cross domain model as script tags but without sending cookies (maybe only if CORS would allow it). An example is this:
loadJsonp(
'http://example.domain/some/jsonp/source',
function onLoad(json){},
function onError(message){}
);
This new host object (naming is up in the air, I don't care about that) has only one purpose;
make the current "jsonp mechanism" safe.
When the function is executed, the browser will fetch the document given by url. It will apply the same cross domain restrictions as it would to a script tag and cookies would not be sent with the request (optional: cookies may be allowed if CORS headers would...). The fetched document should adhere to this production, using ES5 definitions:
JSONPText ::
Identifier ( JSONText ) ;
When evaluating
JSONPText, the
Identifier is stripped and ignored and
JSONText is ran through
JSON.parse. If parsed successfully the
onLoad callback is called with the object as argument. In any other case (whether it be a bad JSON, bad JSONP or unable to fetch the document) the
onError fires with some kind of error message.
The rest of this blog will try to explain the terms and reasoning of my proposal.
What is "JSON"?
JSON, JavaScript Object Notation, is a simple protocol using js object literal notation to transfer data as text. The js environment can simply do an eval on the text to get the object back. However, using eval is a very unsafe way of parsing JSON as you put your complete application environment at the merci of the content provider. Recently, ES5 introduced JSON.parse in an attempt to mitigate this problem. You can feed it a string containing the JSONText and it will return you an object. If the string was not proper JSON, an error is thrown. Nothing gets executed; win.
{"a":1, "b":2}
To get a JSON string from your own domain into your js environment as an object all you have to do is make an XHR call and run the resulting string through JSON.parse. However, you cannot do XHR across domains without CORS headers. Therefore it is impossible to transfer unpadded JSON strings from another domain to be objects in your js environment. Yes you can download the file and yes you can "execute" that file. But on its own a JSON string is an invalid script. And even if it wouldn't be, you have no way of saving the object because you have no control over the script tag. It doesn't return anything and you can't catch any errors thrown by it (from the outside).
What is "JSONP"?
JSONP, JSON with Padding, is a term to describe the method of transfering JSON objects across domains. Since cross domain XHR was impossible (until at some point CORS was introduced) and people wanted to share data anyways they started using script tags. Script tags are (virtually) not restricted like XHR is. You can include a script from any domain and run the returned script as if it was local. To work around the problem of actually getting the JSON string, people started using callbacks. This is where the P comes in. In retrospect, it maybe should have been called JSONC, but it's a little too late for that now. So a jsonp file may look something like this:
someCallback({"a":1, "b":2});
The caller would define a global function
someCallback and would receive the requested JSON string that way. The callback would be responsible for converting the string back to an object, either through JSON.parse or through eval. That's it. The major problem here is obviously that you're executing scripts from another domain. Sometimes the domain is yours, sometimes it isn't. When it's not, you are asuming the source will be safe. But it may not. Yet, this point is ignored. So every now and then you hear about issues regarding this.
What is "CORS"?
CORS, Cross Origin Resource Sharing, was recently introduced to allow cross domain transfer of arbitrary data, like unpadded JSON strings. It is in fact still a
working draft. It relies on the server of the content provider serving out a certain set of headers (defined in the protocol) which tell the browser whether or not a certain domain has access to this resource. The browser will, before requesting the resource, send an OPTIONS request to check these headers. If no headers are returned or if they do not match the request, everything is aborted and you're screwed.
At the time of writing, CORS support is present in all recent non-IE browsers. When I say support, I mean that the protocol is supported, but error messages are spotty at best. It is hard to determine what's going on. As long as CORS is set up properly, you have no problem. But as soon as you need to determine the problem, you have to figure this out on your own.
So CORS has a setup cost. On top of that, it adds to the complexity of the stack of your website. You have to add the headers for any resource you want to be fetchable cross domain. You have to determine whether it is a strict set of domains (it uses a whitelist approach), but you can optionally include any through *. Which headers may be sent (some browsers enforce certain headers). Whether cookies may be sent (if you want cookies you cannot use * for allowed domains so your complexity increases). And so on.
What is the problem?
The problem I am trying to fix is that JSONP is and remains a very popular way of transferring data cross domains from the client. The setup is very simple on either side and the stack doesn't get much more complicated. Compared to that, setting up CORS is expensive in short and the long term (setup and maintainability). As long as script tags are unrestricted, JSONP will remain to be the popular choice because of that. On top of that, CORS is still a working draft, meaning the syntax might still change.
JSONP is inherently insecure. By design, some might say. You fire a script tag, essentially executing an XSS, and hope your callback gets fired. And nothing else.
So why is your proposal safe?
Glad you asked! My proposal does not execute any arbitrary code from external domains. It will strip the callback (and ignore it) and run the argument through
JSON.parse. The returned object is fed to the
onLoad callback supplied to the host object (
loadJsonp, in my example) of the api. So as long as
JSON.parse is safe, so is my method.
Why require JSONP? Why not just JSON?
I don't want to introduce _anything_ new. I just make an existing mechanism safe. Without CORS, it is currently impossible to fetch JSON strings across domains and get them as an object in your environment. I want to keep it that way. Let's not mess with that model. On the other hand it is already possible to get JSONP. So the assumption is that allowing cross domain JSONP only lets you do something you could already do, albeit safe.
This seems to be a very difficult point to make clear.So you're not introducing anything new?
No. That's the point!
Are there other proposals?
Yes. Two currently stand out.
Kyle Simpson (@getify) has a proposal up on
json-p.org. He wants to keep using the script tag with a new type value, "application/json-p". The script tag should reject and not execute the content if it does not adhere to the syntax of json-p, which is almost equal to my proposal. I think the main problem with this is that you are still executing "hostile" code. He intends to harden the mechanism by black-listing certain methods from being invoked. I don't think black-lists are safe or future-proof. If a browser introduces something new, you're screwed. Such a standard would never hold for long. Also, feature detection is going to be difficult. And you are still going to have to define a global for the callback which is not exactly considered a best practice these days.
Douglas Crockford made
an effort in 2006 to get JSONRequest in. It didn't make it and I don't know why exactly. So why do I think my proposal has a chance? Because it's much simpler and doesn't introduce a new tool. Douglas' proposal would allow you to download JSON cross domain, which may have been fine back then, but that doesn't seem to be the case now. If the proposal would talk jsonp, I have no problem with it.
So, that's it :)