Tuesday, October 24, 2006

Part 3: Jabbering with Google Talk over XMPP

This is part 3 of a series of blogs on getting IM working with a Google Talk client using XMPP. Parts 1 and 2 are here -
Jabbering with Google Talk over XMPP
Part2: Jabbering with Google Talk over XMPP

This one is going to be short one because I managed the last part of the Google Talk basic IM interaction which was presence and messaging without too much effort.

Presence is how you announce that you are available/unavailable/busy/whatever. It is through presence that people tell you that they have stepped away for a quick cuppa mocha - the part where you can have witty remarks show up against your name - basically your network availability.

The basic presence part is easy - requiring sending only a stanza. If you are already on the "roster" (XMPP's name for a buddy list) then your presence will be indicated by your id going "online" in Google Talk. You would have to be invited and on the users' roster for that.

Doing the presence part itself is optional - you can still send messages even if you have not announed your presence. The only difference is that messages directed to you are not sent by Google Talk because it is not aware of your avaiability and your friends won't know if you are online.

Presence has a bunch of interesting details which I won't go into mostly because I skipped most of them :-) but here's a quick list of what it can do -

  • announce availability/unavailability plus a bunch of sub-states
  • provide comments along with status
  • allow for subscription (notification of presence of others on the server)
  • change subscription
  • etc
Instant Messaging
The whole IM bit is where I was leading through and here it is. I did cover in Part 2 the messaging part which was basically

<message from=\"XXXXXXXX@gmail.com/D922F673\" to=\"YYYY@gmail.com\" type='chat' xml:lang=\"en\"><body>TESTING!!!</body></message>

and this is pretty much it actually. You can have the following in the message though - subject(s), body and a thread.

So once you have set up a Thread reading from your response stream you get message stanzas from all your friends and you can send them your messages on the request stream - thats it. Google Talk behaves just fine identifying the program as a non-Google Talk client.

Here's a silly picture of the two clients talking to each other.

Saturday, October 07, 2006

Answers to some seemingly common Java questions

Whenever I go through the Keyword Analysis page of Statcounter, which manages my blog web usage statistics for me, I see a bunch of Java questions which I can answer. However the page that people land on to never has the relevant answer to the query. With my recent Repetitive Stress Injury I am pretty much staying away from doing any more work than is required to help my hands heal faster. This is keeping me from working on my Google Talk programs. So, having nothing better to do I am going to try and answer some of the questions or searches that I saw coming to my blog.

Search Keywords: Java printStackTrace does not show line number
Happens if the classes that are part of the stack trace have not been compiled with the debugging option (-g) on. Here's the link for details on javac's options. The solution is to re-compile the classes with this option on and then recreate the exception. If it is not code you can build then there is not much you can do here unfortunately - you will have to analyze the code to figure out where it went wrong. Not having debug information while compiling usually also means that you won't get any local variable information while debugging in a JPDA debugger (to be complete precise that would mean that the -g:vars option has not been specified).

Search Keywords: how can i tell what caused a concurrentmodificationexception?
A ConcurrentModificationException happens if a java.util.Collection is modified while an Iterator is iterating over it. There are many ways you can end up doing this - I am going to try and list the situations that I believe are most common.

Disclaimer: Code snippets do not use generics which won't make any difference anyway.
1.] Listeners
Consider this listener interface -

1 public interface XYZListener {
2 void eventOccurred(Event evt);
3 }

and this implementation -

4 public class XYZListenerImpl.java {
5 public void eventOccurred(Event evt) {
6 evt.getEventSource().removeXYZListener*this);
7 // do something useful
8 }
9 }

where Event.getEventSource() returns the object against whict XYZListenerImpl's instance was registed as a listener.

Now, you will get a ConcurrentModificationException if the EventSource is implemented this way -

10 public class EventSource {
11 private List listeners = Collections.synchronizedList(new LinkedList());
12 public void addXYZListener(XYZListener l) {
13 if (!listeners.contains(l))
14 listeners.add(l)
15 }
17 public void removeXYZListener(XYZListener l) {
18 listeners.remove(l);
19 }
21 protected void fireEvent(Event evt) {
22 for (Iterator iter = listeners.iterator(); iter.hasNext();) {
23 XYZListener listener = (XYZListener)iter.next();
24 listener.eventOccurred(evt);
25 }
26 }
27 }

This is going to cause a ConcurrentModificationException at line 24 (assuming that there are more than one listeners registered and XYZListenerImpl is not the last one ;-)) because in line 6 the listeners List is modified while it is being iterated over in EventSource.fireEvent's Iterator.
The solution in this case is to use the right pattern for firing events -

protected void fireEvent(Event evt) {
List clonedList = new ArrayList(listeners);
for (Iterator iter = clonedList.iterator(); iter.hasNext();) { // [19 Dec] edited - thanks to Anon comment
XYZListener listener = (XYZListener)iter.next();
try {
} catch(Exception ex) {
// this prevents one bad listener from preventing the event from going to others
// log the exception

2.] Incorrect coding
This will cause a ConcurrentModificationException -

public void someMethod(List l) {
for (Iterator iter = l.iterator(); iter.hasNext();) {
Object o = iter.next();
if (someCondition()) {

Fix this by using Iterator.remove() instead of doing a list.remove()

3.] Concurrency
The trickiest one is when the Collection gets modified by another thread while it is being iterated upon. One solution is to clone the collection (e.g. new ArrayList(list)) and then iterate upon it. To find out where the List got modified -
  1. Create a wrapper List similar to Collections.SynchronizedList which delegates all methods to the enclosed List.
  2. In the add/remove and any other method that modifies the List dump the current Thread's stacks using Thread.dumpStack. Refer to this. I suggest printing the current timestamp and the Thread's id for easy collating and the List's hashCode() to identify operations against each List instance.
  3. When you get the ConcurrentModificationException print the List's hashCode and look for the last print from the List modification logs for a list of this hashCode and you should know which two threads are the "culprits".

Search Keywords: how do you add a print statement ever 5 minutes in java
This is quite easy. Create a java.util.Timer class and add a TimerTask and set it to fire every 5 minutes and write the print statements in the TimerTask.

Search Keywords: using ethereal to capture google talk
This is not really possible after TLS is set up as I discussed in the comments here. Google Talk mandates TLS and once the stream gets encrypted the whole point of that is to not be able to sniff out the contents using something pretty much like Ethereal.

There are a couple of other interesting queries that I have not taken up
what are some things that made java so popular?
java 5 why
(I would say - Generics)
why isn't java used for games
(I really don't know if it is or it is not used for games and what kind of games?)

Saturday, August 26, 2006

Part 2: Jabbering with Google Talk over XMPP

This is Part 2 of the posts that describe my attempts to do something interesting (eventually) with Google Talk using XMPP. Part 1 is here.

So finally I had my test Google Talk account successfully send a test message to my regular Google Talk account. The XML for that is

<message from=\"XXXXXXXX@gmail.com/D922F673\" to=\"YYYY@gmail.com\" type='chat' xml:lang=\"en\"><body>TESTING!!!</body></message>

Note that the from value is the JID returned by the Gmail server.

I am still quite far from where I want this to go. However, I am glad I am making progress. The last time (in Part 1) my quickly-put-together test code was reaching a point where it could no longer be used because I needed to save conversation state like the jid which it didn't allow.

Now, with a design and a framework in place I hope I can move on to newer things with Google Talk. Here's a brief class design diagram. I am not very formal with UML diagrams - I pick and choose what I like from UML so if it is not classic UML I apologize to those who may get irked by it. Let me know what you think of the design. I haven't seen what the Jive XMPP library design looks like yet.

XML Parsing Challenges
An interesting implementation challenge was XML Pull Parsing. I tried (briefly) the Stax parser in J2SE but I found it to be a little inconvenient with its event id based mechanism. I might be quite wrong though because I am sure I didn't spend as much time exploring its fit into my solution as I should have. I found myself thinking in terms of iterating over available pieces of XML as required.

The XMPP response stream is like an XML document where each response is another child of the document element. And you don't get the next child until you have sent a request. (I don't know how one receives messages yet so there might be some twist to this story later).

So, an XMLIterator which allowed me get the details of the current element and then waited until I asked it move to the next one was what I wanted and this open source project does pretty much that. Thank you very much Mark.

What I also wanted was an API that returned DOM nodes (preferably) as it saw them in the stream. Of course, the document element node will remain incomplete until the end is reached but thats just a technicality because all that is required to "complete" it is the end-tag which has no real information. I was looking for something like this -

XmlIterator iter = new XmlIterator(source);
// we are now on the document element node
XmlIterator children = iter.children();
while (children.advance()) {
Node node = children.next();

I have a simple XML Node structure built over XmlIterator which works for me - maybe that's a project for later.

Next post I aim to be able to maintain a conversation with a Google Talk client and hope to have enhanced the framework to be able to do more things than just chat.

Monday, August 07, 2006

Jabbering with Google Talk over XMPP

I am writing an XMPP client that can talk to Google and maybe implement some cool things on top of it. So this weekend I began exploring the specs and intend to maintain an account of how it is going. XMPP is a widely discussed/implemented topic and has many many clients - so it is no research topic. Why did I choose to do this? Just for fun to try something new (for me) out. I know I can get the XMPP library from Jive Software but thats no fun. Sometimes re-inventing the wheel has its own pleasures :-).

The XMPP specs are an open standard on which Jabber and Google Talk are based. There are a number of extensions which are under consideration and most of the cool things have already been thought of such as RPC over XMPP.

So far as a prototype, I have managed to get connected to talk.google.com, perform starttls and authenticate myself followed by resource binding and initiating a session. Now, my prototype code is hitting its limitations and I will have to spend some time fixing it to be able to do some serious talking with XMPP. I have also managed to get my account successfuly blocked for authenticating incorrectly as well - but got out of that mess. :-)

I was hoping to sniff out the conversation that the Google Talk client is having with the server using Ethereal as per this blog that describes X-GOOGLE-TOKEN, a Google mechanism for Single Sign-on. As per that blog, the Google Chat client does not do starttls but does an XMPP authentication over an un-secure socket using a Google generated token (the actual authentication with the Google token server is over https) so all the communication can be sniffed. However, my client seems to be doing a starttls and I can't sniff any details out after the proceed response. Too bad - appears that Google Talk has changed since the blog was written.

Here's the sequence of communication with the Google Talk server - (formatted for readability with text sent from me in this colour and the response in this colour and comments in this colour).

<?xml version="1.0" encoding="UTF-8"?>
<stream:stream from="gmail.com" id="X0B367FC8A9597BA4" version="1.0" xmlns:stream="http://etherx.jabber.org/streams" xmlns="jabber:client">
<starttls xmlns="urn:ietf:params:xml:ns:xmpp-tls"/>
<mechanisms xmlns="urn:ietf:params:xml:ns:xmpp-sasl">

<starttls xmlns="urn:ietf:params:xml:ns:xmpp-tls" /> <--- Start TLS - basically the rest of the communication is over SSL
<proceed xmlns="urn:ietf:params:xml:ns:xmpp-tls"/>

TLS Succeeded - we are good to go...

<?xml version="1.0" encoding="UTF-8"?>
<stream:stream from="gmail.com" id="X1A565C1E8E3FD7CA" version="1.0" xmlns:stream="http://etherx.jabber.org/streams" xmlns="jabber:client">
<mechanisms xmlns="urn:ietf:params:xml:ns:xmpp-sasl">

Now we get the PLAIN auth mechanism which is basically base64 encoded \u0000username\u0000password string which I have blacked out here.
<auth xmlns='urn:ietf:params:xml:ns:xmpp-sasl' mechanism='PLAIN'>XXXXXXXXXXXXXXXXXX</auth>
<success xmlns="urn:ietf:params:xml:ns:xmpp-sasl"/> <--- authenticated

<?xml version="1.0" encoding="UTF-8"?>
<stream:stream from="gmail.com" id="X77D6827CD0B365BA" version="1.0" xmlns:stream="http://etherx.jabber.org/streams" xmlns="jabber:client">
<bind xmlns="urn:ietf:params:xml:ns:xmpp-bind"/><session xmlns="urn:ietf:params:xml:ns:xmpp-session"/>

<iq type='set' id='bind_1'>
<bind xmlns='urn:ietf:params:xml:ns:xmpp-bind'/>
<iq id="bind_1" type="result">
<bind xmlns="urn:ietf:params:xml:ns:xmpp-bind">

<iq to='gmail.com' type='set' id='sess_1'><session xmlns='urn:ietf:params:xml:ns:xmpp-session'/></iq>
<iq from="gmail.com" type="result" id="sess_1"/>

Authenticated, Resource bound and Session created. Now, I need to send a message!


Next article on this, I hope to have successfully sent a message.

Friday, July 28, 2006

Usability in MS Outlook Tasks

I really love the way the MS Outlook's Task List Due Date field works. You can input dates in a variety of fashions and I am not talking about the MMDDYYYY, DDMMYY formats. Here are a few examples -

So if today is 28 July 2006 (Friday)
You can type

  1. W - (next week) - 4 Aug 2006 (Friday)
  2. 2W - (n weeks from now) - 11 Aug 2006 (Friday)
  3. Y, nY - (n years from now) - 28 July 2007 (Saturday)
  4. 3D - (n Days from today) -31 July 2007 (Monday)
  5. Aug - (next August, and other months work) - 28 Aug 2006 (Monday)
  6. Monday - (next Monday) - 31 July 2006 (Monday) (works for all the days of the week)
  7. Today - 28 July 2006 (Friday)
  8. 29 - 29 July 2006 (Saturday)
  9. 2007 - 28 July 2007 (Saturday)
  10. Aug 3 - 3 Aug 2006 (Thursday)
  11. 2 Wed - (n Wednesdays from the next one :-) ) - 16 Aug 2006 (Wednesday)
Some behave unexpectedly e.g. 3 Aug gives me a 3 Aug 2007 while 16 Aug gives me 16 Aug 2006. Not sure why.

M also behaves unexpectedly always returning today's date.

W1D, 2D3W work as well :-) I am sure there are many more that I missed.

I readily agree that some examples are extreme but what I found quite interesting was that someone put so much effort in making this component so you can type dates in an extraordinary number of ways and most times I find it does what you expect it to.

Monday, June 12, 2006

Finally some new things to do

After a long time dealing with software development pressures at work which consumed my energy to do little else, this weekend I dappled in new technologies (new for me). Feels so good - I have a high from it this Monday morning!

Got introduced to Google Web Toolkit (GWT) which looks very promising. The UI I am ending up with is very clunky and I'll have to work hard on writing (stealing?) some good style-sheets to make the UI look nice. This lead to getting re-introduced to Cascading Style Sheets (CSS) and a whole bunch of related resources. The whole research can be done rather nicely on Google Notebook.

More on GWT as I get more familiar with it.