Category Archives: javascript

Unexpected token error when parsing JSON data

I have a bunch of files each having a json object. I wanted to write some server side javascript code to process this data. The files can be very large (can be as big as 10 MB) and so initially thought of using V8 javascript engine. That’s when I realized that I can’t write pure JavaScript code to get things done when there are requirements like reading from a local file. But I knew about Node.js and so quickly started looking into the documentation and started writing the javascript code.

The code is very simple


var fs = require('fs');
var json = fs.readFileSync("./json.txt","utf8");
var x = JSON.parse(json);

But this threw an error


undefined:1
{'Search':'AjaxSearch2','Query':'AjaxSearch2
^
SyntaxError: Unexpected token '
at Object.parse (native)
at Object. (/private/tmp/d/test.js:3:14)
at Module._compile (module.js:449:26)
at Object.Module._extensions..js (module.js:467:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Module.runMain (module.js:492:10)
at process.startup.processNextTick.process._tickCallback (node.js:244:9)

Apparently the strings in a JSON object are supposed to be double quoted. Since I don’t have control over the json files, I was trying to find ways to fix this. Finally, I settled for the following which may be a bit slower but does what I need it to do.


var fs = require('fs');
var json = fs.readFileSync("json.txt","utf8");
//var x = JSON.parse(json);
eval("var x = "+json);

Note that using


var x = eval(json)

didn’t work either (it gives the same Unexpected token error for ‘:’).

I see two performance issues with this approach

1) Unnecessary concatenation of the string “var x = ” to a 10mb string (in my case)
2) eval is lot more generic that can parse any javascript string which can be slow compared to a well optimized JSON parser.

Well, sometimes you have to give up performance for ease of coding.

2 Comments

Filed under javascript, JSON, Node.js

Moving from Prototype/Scriptaculous to JQuery

Recently I had to do the migration of some javascript code written using Prototype and Scriptaculous to JQuery. The process wasn’t straightforward but wasn’t very difficult either. I jotted down a bunch of differences I had to deal with and thought of putting them together here.

Feature Prototype/Scriptaculous JQuery
ID Selector id #id
appending a node appendChild append
cloning array Array.clone Array.slice
Setting css style values elem.style.prop=value elem.css(prop,value)
Getting parent node elem.parentNode elem.parent()
Utility methods for an element Element.Methods.getHeight for example elem.height()
Removing a node node.parentNode.removeChild(node) node.remove()

Leave a comment

Filed under JQuery

Getting Access Is Denied with JQuery Ajax in IE 9

I recently moved from Prototype and Scriptaculous to JQuery. The transition is not very smooth but wasn’t very painful either. Actually much of it is straightforward except for Drag and Drop. Anyway, everything was done and it worked fine in Firefox, Safari and Chrome. But not so with IE.

There is an ajax call which kept giving “Access is Denied” error with IE. Searching on the web indicated this is something to do with Cross Domain AJAX call (some reference to CORS). But in my case the call is to the same domain. I was scratching my head on what’s wrong and when I changed the minified javascript to the regular script the debugging became easy. The error was in buildFragment. So, once I started searching for the same Access Is Denied along with buildFragment, I got a link to this exact issue which actually has been fixed. The issue can be looked at here. In my case, I just downgraded from 1.8.0 to 1.7.2 and for now it’s fine. May be will upgrade in the future.

2 Comments

Filed under AJAX, JQuery

Providing Context to a JSON Callback

My twitter search engine prototype is going smoothly. But as I wanted to add more complex features, I ran into an issue. Twitter Search API provides JSON support with a callback. I like this option very much, as it allows doing all the work on the browser. There is no need for browser to my web server to twitter web server communication. Instead, the user’s browser directly interacts with twitter’s server and can process the results in the browser itself.

So, the way to specify a callback api is to pass the name of the function to the Twitter API, something like

http://search.twitter.com/search.json?callback=foo&q=some-query

Then, the response would be

foo(results-represented-as-JSON-object)

So far so good. Now I wanted to keep track of some context within which the search request is being made. So, I first created an object to represent the context, defined the callback routine within that object. Now, since I am going to have several requests made, I need to have different context objects. So, I created a context object array and then I created a request URI like

http://search.twitter.com/search.json?callback=cArray%5B0%5D.foo&q=some-query

the next time a request needs to be made, it would be cArray[1].foo and so on.

However, the response that was sent is invalid. It’s like

cArray([0].foo(JSON))

I have no idea how it ended up like this.

So, I immediately looked at Google’s own API to see what it does. Interestingly, in their search API they take two separate parameters, one for callback and one for context. Their example is

http://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=Paris%20Hilton&callback=foo&context=bar

I wanted to see if my way of using a callback function which is actually member function of an array object worked or not. I tried

http://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=Paris%20Hilton&callback=myarray%5B0%5D.foo

and the response is

{“responseData”: null, “responseDetails”: “bad or missing callback or context”, “responseStatus”: 400}

To me this feels completely wrong. The workaround I am using for now in my twitter prototype is to create one variable for each context at the document level scope (by using eval) and then passing that in the callback. Something like

http://search.twitter.com/search.json?callback=c0.foo&q=some-query

where c0 is a new variable created for the first request. Next request would introduce c1 and so on.

Now the 20 lines question is, shouldn’t the callback name be allowed to be something like contextObjArray[someindex].callback as per JSON spec? Or, are these all wrong implementations?

Leave a comment

Filed under javascript, JSON

Iframe reloads when moved around the DOM tree

I had a great idea. And to implement it, I needed to do DOM manipulation of an element that contains an IFRAME. It’s simple right? Just do something like

document.getElementById(“to”).appendChild(document.getElementById(“elementwithiframe”));

This actually worked. It even works if the iframe is generated using document.write that is part of an external javascript. I was almost done with my project. When I was testing out another javascript that creates an iframe, I ran into the first issue. The elementwithiframe moved by the iframe was blank.

I observed that in one case the javascript creates an iframe that has the src attribute and loads the iframe content from a URL. In another case, it creates the iframe and opens the iframe’s document and writes the content and closes the iframe. The second case is where I had the problem. This happens in both Firefox and Safari.

I wasted a lot of time on this issue. I even came up with a workaround for the second case. The workaround is to first get the head and body elements of the iframe before it’s moved and after the move, copy it back. In fact it worked perfectly fine. But I was still curious why the behavior is different in both case. I even downloaded WebKit source and browsed through it, didn’t find the exact issue, but the dom API has it’s issue which I will tell in a few minutes.

Then, after searching on Bugzilla@Mozilla, I found a bug that gave more details. Apparently, when an iframe is moved around the DOM tree, it’s refreshed. So, if an iframe has an src element, it’s just refreshing the frame again. This means the document is loaded twice. But when the iframe is created by opening the document and writing the content, that content is lost when the DOM is moved. It doesn’t seem to execute the script again (which is probably what is desired as otherwise, there would then be two frames). More importantly, the script element that generated the iframe should not execute the second time because the iframe is generated using document.write. So, once a document is loaded, calling document.write can have unwanted consequences. Anyway, I wasn’t observing this, so hopefully moving script element with src attribute around doesn’t execute it again.

Net net, moving an iframe around the DOM model makes it refresh and in case where the document content is filled using javascript, the iframe is going to lose it’s content unless the hack I mentioned above is done.

Looking at the above Mozilla bug and also the WebKit code I looked at made me realize that part of the problem is with DOM specification which states that since a node can have only one parent, moving around requires removing the node from the parent and then adding it back. But to remove the node, usual logic used is removeNode api. So, in case of an iframe element, calling the removeNode seems to have the adverse impact of clearing up the content (why this happens only for iframe but not other types of elements like div beats me, but may be that has to do with the fact that iframe represents a complete document altogether and not cleaning up immediately when the removeNode is called can have memory issues).

Had the apis to move the node (appendChild, insertBefore, insertAfter) were not required to do the remove as per the spec, then this would have been a simple move without destroying the iframe content. But again, I don’t know if I should say that the DOM api is wrong because as I mentioned, this works for other types of elements, not just iframe.

The fact that this bug was filed on 2004-08-03 and it’s open to this day reflects the complexity of fixing this issue which I don’t hope anytime soon. Someone actually provided a patch in 2009, but it’s not incorporated yet because of concerns on what it might break in several other areas.

After giving this some more thought, I abandoned my project which I thought I got a master idea :).

3 Comments

Filed under DOM API, IFRAME, javascript

DHTML: removeChild or div.innerHTML?

I ended up with a strange issue today, a difference in behavior between IE and FF (tell me something new huh?).

I have some dom nodes and they are dynamically inserted into a popup div as needed. So, before they are inserted, I first need to clear the popup div itself. I was doming (made a typo for doing but thought this is appropriate as well, “doming”) like

div.innerHTML = ”;
// add the existing nodes by context

This worked in Firefox. In IE, it did work, but subsequent popups started not showing the inner nodes being added. So, I changed the above to

for(var i=div.childNodes.length-1;i>=0;i–) div.removeChild(div.childNodes[i]);
// add the existing nodes by context

And this started working. So, it looks like when div.innerHTML is set to blank, or may be some other value for that matter, the existing nodes in the div and their content is being cleared up by IE, while firefox doesn’t clear up the contained nodes.

1 Comment

Filed under DHTML, javascript

JavaScript Associative Arrays (HashMaps) & Prototype.js & Object

Today I spent a lot of time figuring out what’s going on with the for loop over an Array. Finally realized what’s going on and hence this post.

In JavaScript, if you need to create associative arrays (hashmaps), then there are two ways to do it. Actually, there is only one way, but I will get to that in a minute.

var hmap = new Array();
hmap[0] = “hello”;
hmap[1] = “world”;

Here, the hmap array is used as a normal array.

var hmap = new Array();
hmap[“abc”] = “hello”;
hmap[“def”] = “world”;

is perfectly valid. Here, the hmap is used as an associative array. So far so good.

I had a bit of code where I loop through the keys of the hashmap and need to make use of both the keys and the values. So, the code was something like

for(var key in hmap) {
// do something with key and hmap[key]
}

I remember doing this several times in the past without any problem. But today, when using this pattern along with prototype.js, I ended up getting, in addition to the keys I put in, additional keys starting with “each”. Where the heck is this “each” and a whole bunch of other keys coming from? After a bit of digging I found Array @ prototypejs.org article that the prototype.js script actually adds several useful methods to the Array. And the right way to access the elements in the Array is to use the hmap.each method that enumerates all the elements. But in my case, I want both the key and value, so I couldn’t use this method.

After some more reading, I finally realized that it’s actually the Objects that are associative arrays in JavaScript. That means, rather than using Array for the hmap, I could simply use Object. So, by changing the code from

var hmap = new Array();

to

var hmap = new Object();

I could use the regular

for(var key in hmap) { /* do something with key and hmap[key] */ }

without any problem inspite of using prototype.js

Interesting how things could go wrong with prototype based languages, no pun intended.

1 Comment

Filed under javascript

Stack overflow at line: … in IE

I recently came across a strange issue with IE. All of a sudden one of my pages started giving “Stack overflow at line: 0” and I had no idea why that was happening. This didn’t happen in Firefox though. After a bit of searching, didn’t really come across any valid solution other than some microsoft support page talking about too many modal windows causing such an issue.

He is what I finally found with my page. I have the following code

<body onload="onload(event);">

In one flow, this dynamically generated page has the onload javascript function and that works fine. However, in another flow of the same page, the onload javascript function is not implemented. So, in case of IE, it looks like it’s ending up in recursive loop on itself and hence the stack overflow. Had I written the code to

<body onload="onloadfunc(event);">

then in the second flow, it would have simply thrown a regular javascript error that the function doesn’t exist. Hopefully now this mystery is solved for you!

1 Comment

Filed under IE, javascript

Bringing A Marker To The Top

In maps and charts, where there are several markers, there is a likelihood of a overlap depending on how dense the data is. In such cases, it would be good to be able to bring the marker that has been selected using the mouse to the top and display the details and push the rest of the neighboring points to the bottom. And then when a mouse is placed over one of those neighboring points, then the current point should go down and the other point should come up.

So, how to achieve this in HTML using Javascript? Below is a way to do this.

Say each marker is a div element. You can do the following. First have a depth variable.

var depth = 1001;

Now, have the following code for the div.

div.onmouseover = function(event) { div.style.zIndex = depth++; }

That’s it! What this does is, every time the mouse is placed over a div, it sets it’s depth to be one more than the current maximum depth ensuring that the element is the top-most element. Isn’t this simple and elegant?

Leave a comment

Filed under DHTML, javascript, Maps

User Input Sanitizing Before Using In Regular Expression

Say you want to take the input from a user and filter any set of text (titles/descriptions) and show the reduced list to the user. Assuming the users are not sophisticated enough to write regular expressions, the requirement is simply to ensure that whatever the user typed in is available some where within the text.

A simple way to do this is just look for the exact substring and see if the index is greater than or equal to 0 (something like str.indexOf(input) >= 0).

Say, you want to return a match that is case-insensitive. Then the substring approach will not work. So, you need to jump to using regular expressions. In JavaScript, this would become,

var re = new RegExp(input,"i"); // the flag i indicates case insensitive
if(re.test(str)) { /* do-some-thing-here */ }

So far, so good. Now, what happens if the user types in some special characters that are typically used in regular expressions as some special control characters? For example,

‘(‘ and ‘)’ are used for grouping and variable capturing
‘[‘ and ‘]’ are used for character set
‘{‘ and ‘}’ are used to indicate the cardinality
‘\’ is escape character
‘*’ is used to indicate 0 or more
‘+’ is used to indicate 1 or more
‘?’ is used to indicate 0 or 1

In this scenario, if someone types in a string with any of the above characters, then the above javascript will fail. So, in order to fix this, the user input string should be first fixed to make it a valid regular expression. This can be done using

var rere = new RegExp("[({[^$*+?\\\]})]","g"); /* you need 2 '\' s to mean 1 '\' and another '\' to treat ']' as special character instead of the characters ending bracket */
var reinput = input.replace(rere,"\\$1"); /* replace the special characters with a \ before them */
var re = new RegExp(reinput,"i");
if(re.test(str)) { ... }

Now if you are generating the above JavaScript code in perl, it gets a bit more complicated. Why? Because, ‘\’s are themselves have escape semantics. Also, $ symbol has special meaning in perl. So, for each ‘\’ above, it would be doubled and also, $ would be escaped as well.

Yes, it gets confusing, but it’s doable. You can see this in action at Flat Panel Plasma/LCD HDTVs. It contains a search box on the top of the image cloud and lets the user to input a string such as Panasonic, Samsung, Sony etc or even product numbers with special characters like TH-42PZ700U to find the corresponding product on the image.

Leave a comment

Filed under javascript, perl, regular expressions, Tech - Tips