Send Keys in Java or automate repetitive tasks programmatically.

If you have ever programmed in VB, you will definitely know what SendKeys are. You can use send keys function in vb to send in keystrokes to other applications, to be specific to that application which has focus. You can do similar work in Java.

You have send keys like functionality in java.awt.Robot class. You can instantiate a Robot class and then use methods like keyPress(KeyEvent.ENTER), keyRelease, mouseMove, mouseClick, etc.

This is very useful. Like I mentioned in the previous posts, I am running certain tests using Browser Widget. And I am not always infront of the system. And I do get a lot of dialog pop-ups which require a user to manually click OK. I mentioned about wmctrl in the previous posts but I did not get to send keys from wmctrl, so here is what I did.

I used wmctrl to get a list of open windows and specifying -x option to wmctrl would give you even the WM_CLASS and based on this, I use -a option to bring the particular window on top. Then once when the window is on top, I use Robot class to send keys.

There are other good tools like Expect scripts, Devilspie which can automate tasks but I did not have enough time to learn them from scratch. EXPECT is more prominent out there but Devilspie too looked promising.

Configuring SWT Browser Widget Preferences!

This technique I discovered works on Linux, ie. when SWT Browser uses Mozilla as its underlying browser. It uses JavaXPCOM to connect with Mozilla and does great job doing so. But then the Browser class do not have any options to configure the preferences of the browser. But you can still configure the preferences of the browser!!
Mozilla preferences are stored in a .mozilla directory for each user, in their home directory. Thanks to Dr. Li, that I could find this out. Now if you run a program that uses SWT Browser, you can see a a folder "Eclipse" inside this .mozilla directory.

[Note I am talking about running a Java Program which has SWT Browser, from ECLIPSE, thus the folder name is eclipse. If you run from console, you might have a different foldername(I guess as that of the program name)]
It is in the /home//.mozilla/eclipse directory that the SWT Browser used Mozilla preferences are stored. It is stored in the prefs.js javascript file. You can specify a user prefernces by using user_pref() function. For example, you want a network boosted SWT Browser widget, then may be you can add the following line to prefs.js
user_pref("networking.http.pipelining",true);
Close Eclipse and start again. Now when you launch the program, the browser would use this preference.
Another way is to use "about:config" within your program do the configuration!
For this you can just write a configure() function and use it when you want. The function can be something like
Class ConfigurationWindow
{
public void configure()
{
Display display = new Display();
Shell shell = new Shell(display);

Browser browser = new Browser(shell,SWT.CENTER);

shell.setLayout(new GridLayout());

browser.setUrl("about:config");

shell.open();

while(!shell.isDisposed())
if(!display.readAndDispatch()) display.sleep();

shell.dispose();
display.dispose();
browser.dispose();
}
}

Now you can create new instance of ConfigurationWindow and call configure() which shows the user preferences window, just like the one you get when you type about:config in Mozilla Firefox.
This way one can configure preferences for Swt Browser Widget!

Linux Window Management - When you are away from the PC

Like I mentioned in the previous posts, I have this requirement where I run tests using SWT Browser Widget on thousands of URLs. And I am start the test and leave the lab. But then for some URL, the SWT Browser just pops up a dialog either asking for password or warning about a bad certificate. This has been my biggest problem off late and has delayed my tests because unless you close the dialogs the program, the execution cannot move forward.(Like the one shown in the image)
I tried out various options like configuring the settings of Mozilla which is used by SWT Browser when the widget runs on Linux (It uses IE when is running on Windows). But this hasnt worked out.
Then I thought, in Linux if a window is created, there is some way that we can be notified about this. Like some kind of GNOME or a X window Events. So initially I thought I would write some program in C to capture these global events and then for those windows I want, I would close them as I get notified. But then unlike getting this idea, implementing it is not that simple. Even now I have no clue if I ever want to write such a program, then what should I start.
Anyways I browsed for a long time and found out about this wonderful new tool called "wmctrl" which does exactly what I want. With wmctrl I can get a list of open windows, their X Window ID, their title, their process IDs. And not only that we can manage a specific window using its ID or title and this includes opening, closing, resize, move, change title, etc. Isnt that great?
You can download it from "http://www.sweb.cz/tripie/utils/wmctrl/". Just install the tool on your machine using ./configure, make, make install and you are good to go.
Getting a list of open windows on Linux
In the command line, type in wmctrl -l to get a list of all open windows and their information. Example shown in the image.

The other options are also shown. You can close a window by passing its ID (-i option) and a -c option.
You can close a window by just passing the title along with -c option.


But then the next problem is how am I going to execute this from a Java Program!
Executing Linux Commands from Java Program
You can run a Unix command from a Java Program, as shown below
try {
Process p = Runtime.getRuntime().exec("wmctrl -l -p");
int i = p.waitFor(); // very important. You wait for the command to be executed.
// i would be the return value of the process you are running.
if(i==0) //if successfully terminated
{
BufferedReader buff = new BufferedReader(new InputStreamReader(p.getInputStream()));
String temp;
while((temp=buff.readLine())!=null)
{
System.out.println(temp);
}
}
else //wmctrl returns 1 if no options passed
System.out.println("error : "+i);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}

The above program executes wmctrl and displays the listing on the console. Isn't Java interesting to work and great?
I am just left with implementing this idea on my lab test machine and collect great results!!!!

BUG in SWT Browser Widget

I have just discovered a bug in SWT Browser Widget which is used by Eclipse for its internal browser. Just type in http://www.friendster.com/ anyway inside eclipse (some editor inside eclipse). Hold CTRL and click on that link to open Eclipse internal browser. As the page is about to finish loading, Eclipse closes!!
The version I am using is SWT 3.3, the latest, so called stable release.
No idea as of now why this is happening. I guess there is some component inside friendster.com homepage that SWT Browser widget does not like much, and infact it is a VM error!
By the way I am running Fedora Core 7.

Small Read from file, write to file utility

A Java Class that I wrote for my own use. Just to have ease of access.....
--------------------------------------------------------------------------------------------------
package buddi.thesis;

import java.io.BufferedReader;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.io.PrintWriter;

public class DataStore {

private PrintWriter writer=null;
private BufferedReader reader=null;
public DataStore(String input, String output)
{
try {
reader = new BufferedReader(new FileReader(input));
writer = new PrintWriter(new File(output));
} catch (FileNotFoundException e) {
e.printStackTrace();
}
}

public String getNextItem()
{
if(reader==null)
return null;
String temp;
try {
temp = reader.readLine();
if(temp!=null)
return temp;
} catch (IOException e) { }
return null;
}

public void writeToOutput(String text)
{
if(writer==null)
{
System.out.println("ERROR : COULD NOT WRITE TO OUTPUT!");
return;
}
writer.println(text);
writer.flush();
}

public void close()
{
try {
reader.close();
writer.close();
reader=null;
writer=null;
} catch (IOException e) {
}
}
}

Java Program to detect Server side redirections!

As a part of my research work, I wrote this small function in Java which gives you HTTP Response Code and with which we can actually detect server side redirects. The server side redirects have a response code as 302. Well here is the program,

public class NetworkTool {
public static int getResponseCode(String url) {

try {
URL urlItem = new URL(url);
HttpURLConnection con = (HttpURLConnection) urlItem.openConnection();
con.setReadTimeout(1000);
con.setConnectTimeout(5000);
//System.out.println(con.getConnectTimeout());
con.setInstanceFollowRedirects(false);
int res = con.getResponseCode();
return res;
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
//e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
//e.printStackTrace();
}
catch(Exception ex){//System.out.println("hehe");
}
return -1;
}

public static void main(String[] args)
{
System.out.println(getResponseCode("http://cs.uga.edu/index.htmx"));
System.out.println(getResponseCode("http://buddibuddibuddi.com"));
System.out.println(getResponseCode("http://128.192.1.21"));
}
}

The statements highlighted in bold are important. Otherwise, there are some url for which you get connected to the server but you keep waiting for infinite time for the data to come on the connection. So if we set time-out on read, we don't wait for long and this is especially helpful when you run the tests on huge data.
It took me quite a long time to get the actual code fixed, and then to find out the two statements. So I hope this post would be useful for others too.

Some interesting figures...

I was wondering how many elements of a type can you store in a
1. HashMap
2. Vector
3. ArrayList
4. Stack
and other Collections.
This is what i found. With the default Heap space intact,
ArrayList can store 33,07,499 Long or Integers and around 1222441 Strings(I am not sure how long)
Stack and Vector, including HashMap can store upto 1236653 Strings. While around 26,21,441 integral types could be stored.

I am not sure if this is same across all machines. Well this atleast gives me an idea to what extent i can go without facing OutOfMemoryError.

How do you construct absolute URL for links in the web page?

Suppose you crawl a page at http://cs.uga.edu/~krishna and one of the HREF links to contact.html, then how do you get the absolute URL for the contact.html ? We can say simply concatenate the original HREF to contact.html, but then it is not that simple. There might be cases where the href was "../../../../contact.html", we need to take into account those cases too. So the simplest way that I found was to use URL class in the java.net package. And the following code does the trick for us.

URL start = new URL("http://cs.uga.edu/~krishna");
URL newUrl = new URL(start,"contact.html"); //construct the absolute URL

Thats the way you construct absolute URL. Very simple, isn't it?

File URIs

URI - Uniform Resource Identifier
A URI for a web resource might be http://krishnabhargav.blogspot.com
Most of the api might require you to specify URI for local files instead of having a local path. Suppose you want to pass a local system file as a parameter and it should be in the form of an URI then you can do this:
Let us say the file is D:/Development/MyResource.txt
Then the fileUri would be something like
file:///D:/Development/MyResource.txt

One most common mistake is to use file:// instead of file:///. We go by the notion that http has "//" and so does ftp. So even this "file" would have "//" in its Uri. But it is "///" instead of "//".

One good and more detailed explanation is given here.
http://blogs.msdn.com/ie/archive/2006/12/06/file-uris-in-windows.aspx
I really appreciate his post.

Using JAva 6 New Scripting Capabilities

I was working on my thesis work where I had a requirement to execute certain code snippets of JavaScript and see if they make calls or modify a specific set of objects. For example, I wanted to see if the code I wish to run sets "location" property. So I wanted to use the Rhino Engine to execute the Javascript and the major problem with that is that Rhino is just an engine for JavaScript and does not provide any DOM API. So any DOM calls made in the JavaScript would terminate the execution. So we have to expose a set of objects by ourselves to the engine. Anyway in the mean time I came across an article which used Java 6 Scripting Engines! I knew Java 6 had some scripting facilities but never gave it a hard thought. It is a really simple API and works really well. In the past I have tried to use Rhino to run javascript but it is really a tedious process. anyway here is how it is done using javax.script package.

import java.io.FileNotFoundException;
import java.io.FileReader;
import javax.script.ScriptEngineManager;
import javax.script.ScriptException;
import com.sun.script.javascript.RhinoScriptEngine;

public class TestScripting {
public static void main(String[] args)
{
ScriptEngineManager mgr = new ScriptEngineManager();
RhinoScriptEngine js = (RhinoScriptEngine)mgr.getEngineByName("js");
try {
js.put("location",new TestScripting()); // tell the script engine to use the object passed when it encounters "location"
js.eval("function alert(x) { print(x);}"); // implement your own global alert function
try {
js.eval(new FileReader("/home/krishna/workspace/SuriLearning/bin/test.js"));
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
} catch (ScriptException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}

public void write(Object mesg)
{
System.out.println(mesg);
}

public void replace(String mesg)
{
System.out.println("Called location.replace to go to : "+mesg);
}

}

Then I have updated the test.js file with some statement like "location.replace("http://google.com")". This is a basic idea on how we can use Rhino to detect redirections, which is portion of my work.