Unexpected token error when parsing JSON data

I have a bunch of files each having a json object. I wanted to write some server side javascript code to process this data. The files can be very large (can be as big as 10 MB) and so initially thought of using V8 javascript engine. That’s when I realized that I can’t write pure JavaScript code to get things done when there are requirements like reading from a local file. But I knew about Node.js and so quickly started looking into the documentation and started writing the javascript code.

The code is very simple


var fs = require('fs');
var json = fs.readFileSync("./json.txt","utf8");
var x = JSON.parse(json);

But this threw an error


undefined:1
{'Search':'AjaxSearch2','Query':'AjaxSearch2
^
SyntaxError: Unexpected token '
at Object.parse (native)
at Object. (/private/tmp/d/test.js:3:14)
at Module._compile (module.js:449:26)
at Object.Module._extensions..js (module.js:467:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Module.runMain (module.js:492:10)
at process.startup.processNextTick.process._tickCallback (node.js:244:9)

Apparently the strings in a JSON object are supposed to be double quoted. Since I don’t have control over the json files, I was trying to find ways to fix this. Finally, I settled for the following which may be a bit slower but does what I need it to do.


var fs = require('fs');
var json = fs.readFileSync("json.txt","utf8");
//var x = JSON.parse(json);
eval("var x = "+json);

Note that using


var x = eval(json)

didn’t work either (it gives the same Unexpected token error for ‘:’).

I see two performance issues with this approach

1) Unnecessary concatenation of the string “var x = ” to a 10mb string (in my case)
2) eval is lot more generic that can parse any javascript string which can be slow compared to a well optimized JSON parser.

Well, sometimes you have to give up performance for ease of coding.

3 Comments

Filed under javascript, JSON, Node.js

Moving from Prototype/Scriptaculous to JQuery

Recently I had to do the migration of some javascript code written using Prototype and Scriptaculous to JQuery. The process wasn’t straightforward but wasn’t very difficult either. I jotted down a bunch of differences I had to deal with and thought of putting them together here.

Feature Prototype/Scriptaculous JQuery
ID Selector id #id
appending a node appendChild append
cloning array Array.clone Array.slice
Setting css style values elem.style.prop=value elem.css(prop,value)
Getting parent node elem.parentNode elem.parent()
Utility methods for an element Element.Methods.getHeight for example elem.height()
Removing a node node.parentNode.removeChild(node) node.remove()

1 Comment

Filed under JQuery

Getting Access Is Denied with JQuery Ajax in IE 9

I recently moved from Prototype and Scriptaculous to JQuery. The transition is not very smooth but wasn’t very painful either. Actually much of it is straightforward except for Drag and Drop. Anyway, everything was done and it worked fine in Firefox, Safari and Chrome. But not so with IE.

There is an ajax call which kept giving “Access is Denied” error with IE. Searching on the web indicated this is something to do with Cross Domain AJAX call (some reference to CORS). But in my case the call is to the same domain. I was scratching my head on what’s wrong and when I changed the minified javascript to the regular script the debugging became easy. The error was in buildFragment. So, once I started searching for the same Access Is Denied along with buildFragment, I got a link to this exact issue which actually has been fixed. The issue can be looked at here. In my case, I just downgraded from 1.8.0 to 1.7.2 and for now it’s fine. May be will upgrade in the future.

3 Comments

Filed under AJAX, JQuery

Taking Advantage Of Free Upgrade And Getting A Discount

After upgrading to Mountain Lion from Snow Leopard, I realized I can’t run the old PowerPC based programs any longer since the Rosetta technology is no longer supported from Lion itself. Luckily all my old software are for both PowerPC and Windows. So, I decided to go the route of virtualization and evaluated VirtualBox and VMWare Fusion. I settled for VMWare Fusion because it’s better for games and graphics.

I evaluted VMWare Fusion 4. I was ready to buy VMWare Fusion but I thought I will wait a few more days and see if they come up with a new release. After waiting a few weeks, I decided to just go and get VMWare Fusion 4 and placed an order. That evening, my trial version prompted me that there is a new VMWare Fusion version for upgrade. I immediately went to cancel my VMWare Fusion 4 order. However, it wasn’t possible even though it was not shipped yet.

Then I was upset about it and searched on the web if they have any sort of free upgrade for people like me who bought the old version within a day of the new release. Apparently they did have a free upgrade. However, to qualify for it, you should have purchased VMWare Fusion 4 between July 25th to Sep 30th. So, there is still some time!

Now here is the interesting thing. VMWare Fusion 5 currently retails for $49.99 but it’s possible to get Fusion 4 for less than that. That means, you can buy a newer version till September 30th for a lesser price and upgrade it to to VMWare Fusion 5. Note that you need to avail the upgrade by October 31th. Check the prices of VMWare Fusion 4 and Fusion 5 and decide for yourself.

In my case, I got Fusion 4 itself for $49.99 and on top I returned it with a shipping fine of $5 because I wanted to get a physical box of Fusion 5 instead of an electronic upgrade. So, you need to decide if you are OK with electronic upgrade and ready wait for some time to be able to get that savings.

Just thought it’s interesting to observe how it’s possible to get a discount due to this sort of a loophole and hence the post.

Leave a comment

Filed under virtualization

How I got 18 boxes of a snack

Last week I was looking for buying a snack in bulk. Amazon was offering a pack of 8 at 0.24 cents/oz and with subscription it even comes down to 0.23 cents/oz. Being a prime member, it was almost a no brainer for me to buy it but I felt that the price was still expensive. So, after a bit of search I found that Walmart.com actually had it at 0.172 cents/oz and also sold individual packs ($2.58 per pack). But the catch is, I have to pay for shipping. But they also had free-shipping for orders above $45. Since I was considering this in bulk, I just put an order for 18 of them which cost $46.44 with free shipping. Obviously this is far better deal than what I was getting at Amazon even with subscription.

I got an order confirmation from Walmart and in a few hours I got an email that the order was shipped with the tracking numbers. Those of you who use Gmail, it scans for tracking numbers and displays them on the side. I saw 3 such numbers and thought that probably they couldn’t put all the 18 packs in the same box and so sending me in 3 boxes. So far so good.

Today the order actually arrived home. To my surprise, the order arrived in 18 separate boxes! Yes, that’s 18 of them individually packed with air pouches. I think the delivery person must have thought I was crazy :). I was even teased by family members.

The interesting thing for me is, did Walmart make any money off my order at all? Did they end up paying more money for shipping all those 18 individual boxes than the cost of the packs? I don’t know but as a consumer if I were to similarly ship those 18 boxes to someone else, I know that I would end up paying a lot.

So, what went wrong? I think it’s probably a software drawback. The software that drove the order fulfillment wasn’t smart enough to figure out that it should try to ship them in fewer boxes and reduce the cost of shipping. What about the humans who oversee this? I guess, they were just busy neck deep with all the work to even think anything otherwise and question the computer.

I wonder if the enterprise software systems developed in the last two to three decades are smart enough for the Internet.

One last thing, I went back to Gmail to see why I saw only three tracking numbers on the side. I don’t know for sure but looks like Gmail only shows the first 3 and ignores the rest. I hope they should have had a fourth line indicating “3 out of 18 displayed” and then I would have prepared myself and alerted my family to be ready for 18 boxes 🙂 of the snack.

1 Comment

Filed under Amazon.com, ERP, Walmart.com

Desktop Web Application

Are you frowning reading the title of this post? I recently finalized on developing my next commercial application. These days everyone is jumping to SAAS but my application has to be a desktop app running on the users machine. It is going to have a lot of reporting and charting. So, the best option is to make it as a web application and get all the power of HTML and JavaScript for presentation.

In the past, when I wanted to develop such an app, I used Java, Tomcat and Derby for Programming Language, Web Server and Database respectively. Both the web server and database were used in embedded mode.

Fast forward a few years, now I am moving to a different tech stack. I wanted to use sqlite3 which is more flexible and that rules out Java (I didn’t wanted to rely on any non-native jdbc drivers). So, I had to decide between Perl which I am quite familiar with and Python which is all new to me. Given the raising (and may be lowering) popularity of Python and decline of Perl, I decided to go with Python.

So, my current tech stack looks like this:

Programming Language : Python
Database: Sqlite3
Web Server : CherryPy
Templating Engine: Mako (I evaluated a few, this is one of the 3 recommended by CherryPy and found the syntax to be less verbose)
Javascript Libraries: JQuery

While researching on some of the Python Libraries realized that Python 3 is not backward compatible with prior 2.x releases but I decided to go with Python 3 (how can you not have nonlocal when you need to exchange info between a callback function and the code that invokes a function that triggers the callback?)

Still need to research on JavaScript libraries for Graphs (using Canvas of HTML5 that do client side rendering). I also need to decide on the type of data presentation for large tables. I want to be able to load the data on-demand as the user scrolls down and not all at the beginning. As I learn from all this, I will update any additional components I might decide to use.

My end goal is to make the app usage as well as app installation as smooth as possible. At this time I don’t know how to make the installation smooth since there are some python libraries that are not part of the standard libraries that I am using. While these 3rd party libraries have licenses that permit including them into my software, I need to figure out if it’s best to get the end users install them from the respective websites or I should provide a way to do it via my software.

I am excited about this new project and more so doing it by learning a new programming language and tech components.

Update 1: For client side charting, I looked at multiple alternatives and finally decided to go with Google Chart Tools.

3 Comments

Filed under Desktop Application, Python

How I Miss Google’s Login To Multiple App Domains Simultaneously

Over the last few years, I ended up with multiple domains all registered with Google Apps. One of the things that I was amused with Google Apps was their ability to allow me to sign-in simultaneously to more than one account. I could sign in to the regular gmail account and one or more of my domain related mail accounts hosted by Google Apps. But something happened a few months back and now I can only sign-in to one account at a time. I don’t know what prompted them to take away this feature. Around the time this started, they also used to show some intermediate page about business account vs personal account and I don’t really understand what they are trying to say, sort of. It probably could have been presented much better.

If you are a web developer, you would know that typically cookies are used to support session and typically one flow is supported per session. There are very few RIA frameworks that can support multiple flows in the same session. They do this by using different flowids and then tracking the backend model at the flow level rather than session level. Infact, J2EE specification only has 4 scopes : request, page, session and application. Flow scope is not something easy to implement and only a few frameworks support them.

So, obviously I was quite excited by the fact that Google supported the ability to login to multiple Google Apps accounts at the same time all accessed via their domain. But that’s gone now and don’t know if it would ever come back.

1 Comment

Filed under Google Apps