Category Archives: JSON

Unexpected token error when parsing JSON data

I have a bunch of files each having a json object. I wanted to write some server side javascript code to process this data. The files can be very large (can be as big as 10 MB) and so initially thought of using V8 javascript engine. That’s when I realized that I can’t write pure JavaScript code to get things done when there are requirements like reading from a local file. But I knew about Node.js and so quickly started looking into the documentation and started writing the javascript code.

The code is very simple

var fs = require('fs');
var json = fs.readFileSync("./json.txt","utf8");
var x = JSON.parse(json);

But this threw an error

SyntaxError: Unexpected token '
at Object.parse (native)
at Object. (/private/tmp/d/test.js:3:14)
at Module._compile (module.js:449:26)
at Object.Module._extensions..js (module.js:467:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Module.runMain (module.js:492:10)
at process.startup.processNextTick.process._tickCallback (node.js:244:9)

Apparently the strings in a JSON object are supposed to be double quoted. Since I don’t have control over the json files, I was trying to find ways to fix this. Finally, I settled for the following which may be a bit slower but does what I need it to do.

var fs = require('fs');
var json = fs.readFileSync("json.txt","utf8");
//var x = JSON.parse(json);
eval("var x = "+json);

Note that using

var x = eval(json)

didn’t work either (it gives the same Unexpected token error for ‘:’).

I see two performance issues with this approach

1) Unnecessary concatenation of the string “var x = ” to a 10mb string (in my case)
2) eval is lot more generic that can parse any javascript string which can be slow compared to a well optimized JSON parser.

Well, sometimes you have to give up performance for ease of coding.


Filed under javascript, JSON, Node.js

Providing Context to a JSON Callback

My twitter search engine prototype is going smoothly. But as I wanted to add more complex features, I ran into an issue. Twitter Search API provides JSON support with a callback. I like this option very much, as it allows doing all the work on the browser. There is no need for browser to my web server to twitter web server communication. Instead, the user’s browser directly interacts with twitter’s server and can process the results in the browser itself.

So, the way to specify a callback api is to pass the name of the function to the Twitter API, something like

Then, the response would be


So far so good. Now I wanted to keep track of some context within which the search request is being made. So, I first created an object to represent the context, defined the callback routine within that object. Now, since I am going to have several requests made, I need to have different context objects. So, I created a context object array and then I created a request URI like

the next time a request needs to be made, it would be cArray[1].foo and so on.

However, the response that was sent is invalid. It’s like


I have no idea how it ended up like this.

So, I immediately looked at Google’s own API to see what it does. Interestingly, in their search API they take two separate parameters, one for callback and one for context. Their example is

I wanted to see if my way of using a callback function which is actually member function of an array object worked or not. I tried

and the response is

{“responseData”: null, “responseDetails”: “bad or missing callback or context”, “responseStatus”: 400}

To me this feels completely wrong. The workaround I am using for now in my twitter prototype is to create one variable for each context at the document level scope (by using eval) and then passing that in the callback. Something like

where c0 is a new variable created for the first request. Next request would introduce c1 and so on.

Now the 20 lines question is, shouldn’t the callback name be allowed to be something like contextObjArray[someindex].callback as per JSON spec? Or, are these all wrong implementations?

Leave a comment

Filed under javascript, JSON