Category Archives: Java

Increasing stack size for a C program

In java, when there is a problem, usually with heap, you simply throw in -Xmx and for stack, it would be -Xss. But what about a C program? Recently I ran into this problem. I had a C program which worked fine on one enterprise class Linux machine but didn’t work on my personal computer based on Mac OS X (Snow Leopard). First I didn’t understand what was going since the error wasn’t obvious but soon I realized that the problem was with the stack as it happened for larger values of input. When you do

ulimit -s

you would notice something like

8192

Then setting

ulimit -s hard

and then checking

ulimit -s

you would notice something like

65532.

So, with this increased stack size, I was able to run the program for even higher input values but soon ran into the same issue as the input value increased even further. Then, searching the web indicated that it’s possible to actually provide a hint at the time of compilation of how much stack size the program would need. So, all I had to do was to compile the program as follows

g++ -O3 -Wl,-stack_size,0xf000000 -o my_program my_program.cc

The 0xf000000 above is the stack size. It’s probably way more than I need. This solved my problem. While this is a good workaround, I think the Java solution is better as it doesn’t require recompiling the code. But then, java is an interpreter, so it has that extra flexibility.

Advertisements

Leave a comment

Filed under C/C++, Java

JavaMail, GMail, POP3 vs IMAP

I have been experimenting with JavaMail api to fetch Google Apps Email as part of a product idea. Initially, I used POP3 protocol and tried to get the Inbox and that worked just fine. Then I wanted to get the remaining Folders. Well, Google Apps EMail (which is based on their GMail software), organizes by Labels and doesn’t use folders (except may be the Inbox, I don’t know all the details). When I used POP3, the only folder that I managed to get is the INBOX folder. Then I switched to using IMAP and interestingly, it returned not just INBOX bot also the the rest are the labels that I defined. The remaining such as “Drafts”, “Sent Mail” etc provided by default by Google were not present. However, there was a special folder called “[Gmail]” which contains the standard default folders. When debugging was enabled, it indicated that the “[Gmail]” folder had something like

* LIST (\Noselect \HasChildren) “/” “[Gmail]”

while rest of them had
* LIST (\HasNoChildren) “/” “[Gmail]”

Interestingly, with POP3, even the “[Gmail]” is not present. Not sure if this is related to Google’s implementation of the protocol or some limitations in POP3 vs IMAP.

Leave a comment

Filed under Gmail, Google Apps, JavaMail

Java String Passwords

JPasswordField and Console classes return the password as an array and a good practice is to blank out the array characters after making use of the password so that a memory dump won’t reveal the password. This is good. But then, today I am experimenting with JavaMail api and it has a PasswordAuthentication interface that has “public String getPassword()” method. Hmm, so what’s the point in ensuring that secure APIs are used only to make them non-secure in between? Sort of like loading a webpage using https but the form submitted getting posted to http.

This is my third of those rarely “hate java” moments. I documented one in 2008, then another in 2009 and now in 2010. Surely, it’s rare. Isn’t it? :).

On a side note, seems James Gosling said in a 2001 interview about Java

“One of the things that forced Strings to be immutable was security. You have a file open method. You pass a String to it. And then it’s doing all kind of authentication checks before it gets around to doing the OS call. If you manage to do something that effectively mutated the String, after the security check and before the OS call, then boom, you’re in. But Strings are immutable, so that kind of attack doesn’t work. That precise example is what really demanded that Strings be immutable. ”

Update: Now after all the effort I put in to still use System.console().readPassword() because useless on my Mac (OS X 10.6.3) because the “without echo” is not working (meaning the letters are echoing on the screen). Could be a problem with Apple’s version of Java or may be in the underlying OS X call. Don’t know, but now I am really annoyed.

Leave a comment

Filed under Java

System.out and UTF8

Remember the post Why I (rarely) Hate Java? Well, this is another of those moments. I was trying to output a string encoded in UTF8 and it was displaying as ??? So, how am I supposed to fix this?

Well, I think, when Java started off, the designers perhaps didn’t bother about encoding. That’s why you see Streams initially in the java.io package and then the readers and writers. The later are aware of encoding while the former classes are not. And unfortunately, the System.in and system.out happen to be the stream classes. As a result, it’s not possible to do utf8 or other non-default encodings.

So, the way to fix this is

PrintWriter out = new PrintWriter(new OutputStreamWriter(System.out));
out.println(“some-utf8-string”);

Now, this should work just fine. And BTW, if you are trying to see the output in a linux console or using vi editor, you would still see the data as some garbage. One option is to open it in Firefox and make sure that the character encoding is in utf8.

6 Comments

Filed under Java

Why I (rarely) Hate Java?

I have spent a lot of time learning the many libraries of java to hate it actually. I have decent knowledge in perl that I am reluctant to learn other scripting languages like Python. Similarly, if I can write cgi scripts in perl, why learn PHP or Ruby (on rails or otherwise).

I had a strange Out Of Memory Heap exception and after reviewing the code didn’t find anything obvious in the java code that hinted at any memory leak. Then it turns out, my regular expression matching is what is causing the memory problem. That is in turn caused by the String.substring.

If you look at Java source code, in java/lang/String.java, you would notice some comments for the

public String(String string)

implementation. First, why would one want to create a String of another string? Well, here is the reason. Thing is, when you get a substring of a string in java, it doesn’t actually create a separate array to store that substring. Instead, the array of the original string is shared and an offset and length are used to track the substring. This type of implementation is possible in Java because strings are immutable.

So, in my use case, I have been fetching a bunch of large HTML pages and doing some pattern matching and extracting some tokens and keeping them in an array. So, even though they happen to be small tokens within my program and hence my initial code review assuming that I am only consuming very little memory, because those tokens happened to be substrings of the entire html page, the memory consumption turned out to be very high. Actually they are the return values of the javax.util.regex.Matcher.group(1). So, instead of directly adding the return value into the array, I created a string

String token = new String(matcher.group(1));

and then added it. This solved the memory problem.

Granted you don’t need to know about memory management when using Java as there is the garbage collector that takes care of things for you. But now and then, you get into this type of issues that require a little bit more digging (not the social type).

2 Comments

Filed under Java