Web Services + JSON = Dump Your Proxy
n my post on "How to build a Maps Mash-up"I mentioned that there are different ways to overcome the browsersecurity restrictions to retrieve data from another domain(cross-domain restriction). The previous sample used the software proxymethod to make the Web service requests and this post talks about a wayto make a request without a proxy. It’s the dynamic script tag method.Today Yahoo! added a new output option for part of their Web services called JSON.This makes it possible to make the JavaScript WS request without usingthe XMLHTTPRequest object. It is a great way to pull data from anotherdomain because you can dump your proxy and all the data will not routethrough your server anymore. I will talk about the pros and cons ofboth these approaches later, but first I want to give an overview ofwhat JSON is, how it works and show some sample code.
What is JSON? On Doug Crockford’s pageit reads like that: "JSON (JavaScript Object Notation) is a lightweightdata-interchange format. It is easy for humans to read and write. It iseasy for machines to parse and generate." And that’s how it look like:
{"ResultSet":{"Result":[{"precision":"city","Latitude":"34.05217","Longitude":"-118.243469","Address":"","City":"LOS ANGELES","State":"CA","Zip":"","Country":"US"}]}}}
The string above is returned by Y! Geocoder for the query “LA”.JSON is a serialized JavaScript object, which JavaScript can turn backinto an object. For Yahoo! WS the structure of the JSON string issimilar to the XML result but the difference between and attribute andelement can’t be made. The following is a comparison of the XML resultfor the same call.
<ResultSet ... >
<Result precision="city">
<Latitude>34.05217</Latitude>
<Longitude>-118.243469</Longitude>
<Address></Address>
<City>LOS ANGELES</City>
<State>CA</State>
<Zip></Zip>
<Country>US</Country>
</Result>
</ResultSet>
One way to get from JSON to a JavaScript object is to call eval(),with the string as argument. The following sample uses the Geocoderresult to display LA’s Latitude and Longitude in an alert box. This isjust static.
eval.html
<HTML>
<BODY>
<script language"javascript">
var location = eval({"ResultSet":{"Result":[{"precision":"city","Latitude":"34.05217","Longitude":"-118.243469","Address":"","City":"LOS ANGELES","State":"CA","Zip":"","Country":"US"}]}});
alert("Lat:" + location.ResultSet.Result[0].Latitude + " Lon: " + location.ResultSet.Result[0].Longitude );
</script>
</Body>
</HTML>
This is nice but doesn’t do too much in the real world. The problemwas to get the data from a Web service that is located on anotherdomain imported without using a proxy.
The secret sauceAdding the <Script> tag dynamically in the DOM tree of thebrowser is the answer and the JSON response helps to get the data in aformat that is easy to digest for JavaScript. When a Script tags getsdynamically added to the DOM tree the code (script URL) gets executedon the fly. The trick is that instead pointing to a JavaScript library,we include a Web service request in the tag that returns data in theabove mentioned format. The Yahoo! Web services that offer the JSONoutput option also supports a parameter called ‘callback’ and all itdoes is wrap the return data in a function with the name of thecallback value. http://api.local.yahoo.com/MapsService/V1/geocode?appid=dantheurer&location=la&output=json&callback=getLocationwould result in something like thisgetLocation({"ResultSet":{"Result":[{"precision":"city",….) which triesto call the getLocation function (callback) that needs to beimplemented to deal with the data.
Below is a sample that takes a location as an input parameter, thencalls the Y! Geocoder WS and displays Long / Lat in the page.
geocodeJson.html
<script type="text/javascript" src="jsr_class.js"></script>
<script type="text/javascript">
//<![CDATA[
var appid = "dantheurer";
//That is the callback function that is specified in the request url and gets executed after the data is returned
function getLocation(jData) {
if (jData == null) {
alert("There was a problem parsing search results.");
return;
}
//get the values out of the object
var lat = jData.ResultSet.Result[0].Latitude;
var lon = jData.ResultSet.Result[0].Longitude;
//build some html
var smart = "Long: " + lon + "<br />" + "Lat: " + lat;
//add it in the page DOM
document.getElementById(’result’).innerHTML = smart;
}
function addGeocode() {
var location = document.getElementById("geoquery").value;
// Build the Yahoo! web services call
request =‘http://api.local.yahoo.com/MapsService/V1/geocode?appid=’ + appid +‘&location=’ + location +‘&output=json&callback=getLocation’;
// Create a new script object
aObj = new JSONscriptRequest(request);
// Build the script tag
aObj.buildScriptTag();
// Execute (add) the script tag
aObj.addScriptTag();
}
//]]>
</script>
Jason, my cube neighbor, wrote a really nice class that deals withthe dirty work of adding, removing and making sure the tags are unique.If you open up the file, it even has a security warning from theinventor of JSON in there. Below is the code snippet:
jsr_class.js
function JSONscriptRequest(fullUrl) {
// REST request path
this.fullUrl = fullUrl;
// Keep IE from caching requests
this.noCacheIE = ‘&noCacheIE=’ + (new Date()).getTime();
// Get the DOM location to put the script tag
this.headLoc = document.getElementsByTagName("head").item(0);
// Generate a unique script tag id
this.scriptId = ‘YJscriptId’ + JSONscriptRequest.scriptCounter++;
}
// Static script ID counter
JSONscriptRequest.scriptCounter = 1;
// buildScriptTag method
JSONscriptRequest.prototype.buildScriptTag = function () {
// Create the script tag
this.scriptObj = document.createElement("script");
// Add script object attributes
this.scriptObj.setAttribute("type", "text/javascript");
this.scriptObj.setAttribute("src", this.fullUrl + this.noCacheIE);
this.scriptObj.setAttribute("id", this.scriptId);
}
// removeScriptTag method
JSONscriptRequest.prototype.removeScriptTag = function () {
// Destroy the script tag
this.headLoc.removeChild(this.scriptObj);
}
// addScriptTag method
JSONscriptRequest.prototype.addScriptTag = function () {
// Create the script tag
this.headLoc.appendChild(this.scriptObj);
}
Here is what the script does in some bullet points
All this is not just a hack that might not work again tomorrowbecause of a browser upgrade. Most of the dynamic ads use the dynamicscript tag to display themselfes depending on the context. There are ofcourse ups and downs for both technologies and not everyone agrees, buthere are some points to think about.
Proxy method
n my post on "How to build a Maps Mash-up"I mentioned that there are different ways to overcome the browsersecurity restrictions to retrieve data from another domain(cross-domain restriction). The previous sample used the software proxymethod to make the Web service requests and this post talks about a wayto make a request without a proxy. It’s the dynamic script tag method.Today Yahoo! added a new output option for part of their Web services called JSON.This makes it possible to make the JavaScript WS request without usingthe XMLHTTPRequest object. It is a great way to pull data from anotherdomain because you can dump your proxy and all the data will not routethrough your server anymore. I will talk about the pros and cons ofboth these approaches later, but first I want to give an overview ofwhat JSON is, how it works and show some sample code.
What is JSON? On Doug Crockford’s pageit reads like that: "JSON (JavaScript Object Notation) is a lightweightdata-interchange format. It is easy for humans to read and write. It iseasy for machines to parse and generate." And that’s how it look like:
{"ResultSet":{"Result":[{"precision":"city","Latitude":"34.05217","Longitude":"-118.243469","Address":"","City":"LOS ANGELES","State":"CA","Zip":"","Country":"US"}]}}}
The string above is returned by Y! Geocoder for the query “LA”.JSON is a serialized JavaScript object, which JavaScript can turn backinto an object. For Yahoo! WS the structure of the JSON string issimilar to the XML result but the difference between and attribute andelement can’t be made. The following is a comparison of the XML resultfor the same call.
<ResultSet ... >
<Result precision="city">
<Latitude>34.05217</Latitude>
<Longitude>-118.243469</Longitude>
<Address></Address>
<City>LOS ANGELES</City>
<State>CA</State>
<Zip></Zip>
<Country>US</Country>
</Result>
</ResultSet>
One way to get from JSON to a JavaScript object is to call eval(),with the string as argument. The following sample uses the Geocoderresult to display LA’s Latitude and Longitude in an alert box. This isjust static.
eval.html
<HTML>
<BODY>
<script language"javascript">
var location = eval({"ResultSet":{"Result":[{"precision":"city","Latitude":"34.05217","Longitude":"-118.243469","Address":"","City":"LOS ANGELES","State":"CA","Zip":"","Country":"US"}]}});
alert("Lat:" + location.ResultSet.Result[0].Latitude + " Lon: " + location.ResultSet.Result[0].Longitude );
</script>
</Body>
</HTML>
This is nice but doesn’t do too much in the real world. The problemwas to get the data from a Web service that is located on anotherdomain imported without using a proxy.
The secret sauceAdding the <Script> tag dynamically in the DOM tree of thebrowser is the answer and the JSON response helps to get the data in aformat that is easy to digest for JavaScript. When a Script tags getsdynamically added to the DOM tree the code (script URL) gets executedon the fly. The trick is that instead pointing to a JavaScript library,we include a Web service request in the tag that returns data in theabove mentioned format. The Yahoo! Web services that offer the JSONoutput option also supports a parameter called ‘callback’ and all itdoes is wrap the return data in a function with the name of thecallback value. http://api.local.yahoo.com/MapsService/V1/geocode?appid=dantheurer&location=la&output=json&callback=getLocationwould result in something like thisgetLocation({"ResultSet":{"Result":[{"precision":"city",….) which triesto call the getLocation function (callback) that needs to beimplemented to deal with the data.
Below is a sample that takes a location as an input parameter, thencalls the Y! Geocoder WS and displays Long / Lat in the page.
geocodeJson.html
<script type="text/javascript" src="jsr_class.js"></script>
<script type="text/javascript">
//<![CDATA[
var appid = "dantheurer";
//That is the callback function that is specified in the request url and gets executed after the data is returned
function getLocation(jData) {
if (jData == null) {
alert("There was a problem parsing search results.");
return;
}
//get the values out of the object
var lat = jData.ResultSet.Result[0].Latitude;
var lon = jData.ResultSet.Result[0].Longitude;
//build some html
var smart = "Long: " + lon + "<br />" + "Lat: " + lat;
//add it in the page DOM
document.getElementById(’result’).innerHTML = smart;
}
function addGeocode() {
var location = document.getElementById("geoquery").value;
// Build the Yahoo! web services call
request =‘http://api.local.yahoo.com/MapsService/V1/geocode?appid=’ + appid +‘&location=’ + location +‘&output=json&callback=getLocation’;
// Create a new script object
aObj = new JSONscriptRequest(request);
// Build the script tag
aObj.buildScriptTag();
// Execute (add) the script tag
aObj.addScriptTag();
}
//]]>
</script>
Jason, my cube neighbor, wrote a really nice class that deals withthe dirty work of adding, removing and making sure the tags are unique.If you open up the file, it even has a security warning from theinventor of JSON in there. Below is the code snippet:
jsr_class.js
function JSONscriptRequest(fullUrl) {
// REST request path
this.fullUrl = fullUrl;
// Keep IE from caching requests
this.noCacheIE = ‘&noCacheIE=’ + (new Date()).getTime();
// Get the DOM location to put the script tag
this.headLoc = document.getElementsByTagName("head").item(0);
// Generate a unique script tag id
this.scriptId = ‘YJscriptId’ + JSONscriptRequest.scriptCounter++;
}
// Static script ID counter
JSONscriptRequest.scriptCounter = 1;
// buildScriptTag method
JSONscriptRequest.prototype.buildScriptTag = function () {
// Create the script tag
this.scriptObj = document.createElement("script");
// Add script object attributes
this.scriptObj.setAttribute("type", "text/javascript");
this.scriptObj.setAttribute("src", this.fullUrl + this.noCacheIE);
this.scriptObj.setAttribute("id", this.scriptId);
}
// removeScriptTag method
JSONscriptRequest.prototype.removeScriptTag = function () {
// Destroy the script tag
this.headLoc.removeChild(this.scriptObj);
}
// addScriptTag method
JSONscriptRequest.prototype.addScriptTag = function () {
// Create the script tag
this.headLoc.appendChild(this.scriptObj);
}
Here is what the script does in some bullet points
- Build request URL with input parameter and callback function
- Build the script tag that contains the request URL
- Add the new tag to the DOM tree
- As soon as the tag gets addes, the WS request gets executed andwhat gets returned is the JSON response wrapped in a function call. Thename of the function got specified in the callback parameter.
- The response, which is a function call now calls the matchingfunction with the JSON data as parameter. This is where the data canget extracted.
- The script tag gets removed again
All this is not just a hack that might not work again tomorrowbecause of a browser upgrade. Most of the dynamic ads use the dynamicscript tag to display themselfes depending on the context. There are ofcourse ups and downs for both technologies and not everyone agrees, buthere are some points to think about.
Proxy method
- More robust, error handling is easier
- More control on the server side
- It has some security implications on the server side as the proxy could be abused.
- The server side can have additional functionality implemented that is hidden to the caller e.g. login, exchange secrets…
- No XML parsing necessary
- Performance win
- No traffic gets routed (and counted) on your end.
- JSON converters don’t know that they should define an array if theyis only one nested element in a tag, even if the Schema allows 1..n
- More cross-browser issues
- Positive impact on rate limiting if it’s done per IP
- No need to set up a proxy