Usability

Joel has an interesting article on social usability here.
A point he makes (besides the one that the article is actually about) is that

this just goes to show you that usability ain't all that important

But that depends.
Usability is not so important when there is a popular precedent. e.g. Mozilla Firefox doesn't need to reinvent any basic browser UI. They'll add to it and try to make it more usable but no user of theirs has to learn the basics (if they need to, there are enough "browser-gurus" around). Usability is also not that important at a micro-level like in Joel's example. You can go wrong and use buttons instead of tabs and most users won't even notice.

On the other hand, consider ConceptDraw MINDMAP. What they are doing might have a precedent but isn't that well-known. Usability for these guys is their holy grail. And Ramesh will tell you they are great. He just loves the product. He's their power user now. For them, the entire experience is important. Conceptual usability is paramount and then they must have the right micro-level usability worked out too. You see, they are a brainstorming tool. The last thing you want to do in the middle of a brainstorming session is to wonder how you insert another level in the damned diagram. For them, getting it right makes all the difference.

Usability changes over time too. When part of Pramati's Studio initiative, we were amongst the first IDEs that had J2EE development in it. People we showed it to were new to J2EE itself. Especially in terms of using archives like EJB JARs and WARs and EAR files. Over the next release, we had Express Development (tm) (which is present in one way or the other in all J2EE IDEs today) which does away with archiving (deploying source modules automatically - known popularly as exploded deployment) but by the time we had this out, many people had gotten used to the new paradigm. Thankfully we had provided the option of exporting modules to archives (which is what these people used extensively - instantly recognizing the "Export As Archive" option on a module). The point here is that over time, the definition of "good" usabililty had changed.

Excellent Applet

Check this out.

Why Java isn't cool : triggered by "Great Hackers"

Paul Graham's Great Hackers essay has really touched a lot of people's nerves. The wires are choked with people giving their point of view -

• Stephan — You Java Programmers are Losers!
  
• Tim — What a load of crap.
  
• Joe — Now Paul Graham's off base, IMO.
  
• Gavin — The Lie Of Simplicity.
  
• Dion — The Java Hacker Nonsense.
  
• Howard — This Great Hacker Does Use Java.
  
• Carlos — The Architecture of Participation vesus Hacking.
  
• Adrian — Why You Shouldn't Employ "Great Hackers".
  

[stolen shamelessly from Erik's LinkBlog]
And of course, no one puts it quite like him whether you agree with him or not .

Yet again, though, I have had to stop and think - what is it about Java that makes people brand it as the most un-cool language on earth? I have had friends look at me like I was a poor sod for "having to" develop in Java. So, let me list all the reasons I can think why people consider Java un-cool.

Java has considerably fewer surprises and prefers not to add complexity to the language for rarely used features thereby resulting in a language where you cannot really make your friends go ga-ga at amazingly brief programming constructs. You need to write something substantial [like Gosling's Huckster] for them be to impressed with your programming abilities and not your language knowledge. This is probably the biggest reason Java is un-cool. It's too easy (although programming or software development remains as tough as ever).
Java was always touted as the language that the "average" IT programmer can use. It's such a language-for-the-masses that yet again, it fails the "geek" test. And if you use Java, so do you.

Java has been considered slow for ages. The earlier allegations (1995) were true. However, with the recent advancements in the JVMs from Sun and IBM, Java runs pretty close to C/C++. Check this benchmark. Contrary to this, there are other benchmarks that prove that Java is slower. All considered, it would be fair to say that Java cannot be considered "slow" anymore, yet its stuck with the label. How cool is to be the jock with the second fastest race-car in the block?

Swing disasters continue to give Java a bad name. Swing is a brilliant, although hard to learn, API. But the vast majority of Swing applications are so bad that they give Swing and therefore Java a bad name.

Java is a strongly typed language therefore you have to tell the compiler exactly what you intend to use. And if you make a mistake in the way you use it, the compiler has the guts to tell you that you were wrong. Too much chaperoning?

Java has a vast library that is available to all Java developers without any ambiguity. Thus, if you wrote yet another Map you would not be considered a data structures guru by Java programmers but a guy who hasn't heard of java.util.*.

Java did not have a good IDE that compared with MS Visual Studio. I think this one was true. I am not so sure it is any more with IntelliJ. The absence of good tools probably pushed away a lot of good programmers.

Java is popular. Anything that is popular has lost its elite status and therefore is not cool.

Java is an application programming platform. You cannot do cool things like device drivers and games, etc (until recently - but Java gaming is coming in a big way).

On a different note take a look at these two projects - I like the direction they are heading in.

Teaching Object Oriented Programming

I have been training someone in Java over the last couple of weeks. This is my first serious experience teaching programming at a one-on-one level. Although, several times, I have been asked to give training or product demonstrations they were for an audience that had already bought into the concept and were simply looking for a human user manual. Not so here.

The person I am teaching knows C and can "think in C". He thinks of logic as a set of functions into which you pass data (mostly primitive types) which operate upon them and so on. I am trying to teach him Java and I am more interested in getting him thinking about objects, design, patterns, etc. I have, however, been having trouble showing him the benefits of OOP(Object Oriented Programming).

OOP has the following advantages:

• Abstraction
• Encapsulation
• Inheritance
• Polymorphism

Concepts like encapsulation can be grasped quite quickly by anyone who has burnt his hands with global data. But consider abstraction - this is not a concept that can be explained to or understood easily by someone who hasn't had to maintain and change the same software over a period of several years. Why should a user be represented by a User object and not a String userName? Why should there be a factory pattern used? The answers to most of these problems go like ... "If your specifications change in the future to include this feature also then designing it this way...blah blah blah". It is quite hard for someone whose programs have so far had a shelf life of a few laboratory assignment days to appreciate change management and the power of well designed Object Oriented Software.

Compile time polymorphism (function overloading)'s advantages are quite obvious. Perhaps, runtime-polymorphism can also be understood without too much trouble. It is however, Abstraction that is the core of well-designed software. Too much or too little breaks design completely. So, explaining abstraction continues to remain a challenge.

Proficient C programmers have their own way of making abstractions in C because they realize the value of abstractions to the human mind. I am not criticizing the C language. It is amongst the most expressive of computer languages ever devised. However, writing programs in the OOP paradigm using C is not something that comes naturally nor do most C books train in that manner. "Thinking" the OOP way is hard because one has to start thinking of <object>.operation(<params>) versus operation(<params>) that one is used to and this becomes a reasonably sized challenge for some.

I think there is a link somewhere between the proficiency in C and the ability to appreciate an OOP language. Once one has written programs in C that stretch one's mental programming capabilities where the complexities tend to cause that mist which makes one feel that things have gone horribly wrong, one starts to change one's way of programming (even in C). Resulting in an appreciation of what advantages "C with classes"(C++, Java) brings.

Experiences In Java Performance Engineering

I have been spending copious amounts of time trying to speed up our Web server. The results have been heartening and the experience typical of performance engineering. It's one of the most amazing activities of software development. Incredibly frustrating when you cannot speed it up but an adrenaline-rush when it does and the first person I bother when it works is poor Rajiv.

I am trying to log the various performance enhancements - some are just tips and some are things I discovered or used. Some you know, perhaps, some you don't.

Sockets


• Socket Writes : The lesser the better. Buffer your socket writes so you never inadvertently write multiple times to the socket. Copying to the buffer is much less expensive than a socket write especially for small data.
• Socket Reads : Similarly, read as much as you can from the socket in one shot. Smaller reads will result in lesser performance. These rules are true of File streams as well.
  
• Socket Connections : Again the lesser the better. Creating a connection is extremely expensive. You can do several ten times of requests/second more on a kept-alive connection than if you have to create connections.
• Socket properties : Setting socket properties is expensive. e.g. java.net.Socket.setSoTimeout. Avoid setting socket properties per request, see if you can change them to per connection? [multiple requests in a kept-alive connection]
  

Data Copies


• String Operations : If you are writing extremely performance sensitive code such as a highly benchmarked web server :-) then keep Strings to a minimum. I cannot stress this enough. If you have a large character stream that you need to tokenize, parse, etc, use references into character arrays to reduce String operations. Create a class that takes a pointer to this character array, its start and length and you can create lots of instances of this class to point to pieces of the character stream instead of String objects which make copies of the characters. String concatenations can be quite expensive too.
• Avoid data copies : This is obvious, isn't it? However, there are so many methods that try to be safe and make data copies it happens without even one knowing it. Examples of classes that do it are - String , ByteArrayOutputStream.toByteArray(). Don't get me wrong - there's nothing wrong with these classes, it's just that sometimes one doesn't realize that these methods are causing data copies which is affecting performance.
  

Data Structures

• Set instead of List : A common programming mistake is the wrong choice of data structure to do a contains() . List.contains() is an O(n) operation versus Set.contains() for HashSet has a best case behaviour of O(1). So, if order does not matter to you, use Set.
  
• Object Pooling : Some classes are expensive to create. eg. large arrays. Use pools of these objects so they can be reused, there by preventing GC lags for these, and primarily creation costs. Interestingly, PrintWriter, if pooled, shows considerable performance improvement. The creation of the object is expensive because of a call to get the line separator in its constructor.

Miscellaneous

• You can buffer at unexpected places. e.g. You may have a logger thread that asynchronously logs certain frequently running activities (e.g. access logs) and notifying the writing thread every time will be expensive. Might help considerably to collect a few and then notify the thread.
• Integer.toString() is much much faster than integer + "".
• try-finallies result in interesting performance degradation in Sun JVMs. Read about it here and Rajiv's excellent follow-up on it here.
• Lazy instantiation : Don't create something until you need it.
• The usual optimizations always apply - loop optimizations, moving loop-invariant code out of the loop, unused variables, repetitive processing, etc.
• Perceived performance : [I would love to collect my thoughts on that one sometime - we have had some very interesting experiences with perceived performance enhancements over the years ] The faster you respond so that the browser starts to refresh [if memory serves me right, a 50 milli- second response is perceived to be instantaneous by a human being] the more responsive the web-server looks although the total amount of time for the response might be the same as in the case if all data was returned in a single write.
  

Oh well, those are all I can think of right now. If I think of any I missed out, I'll post an addendum.

Neostream magic

Check out Neostream Interactive's latest Flash extravaganza!! These guys are creative geniuses! Extremely stylish artists. It's a largish download so you will have to wait a bit - but its worth it.

They have released a lighter wire-frame version of their . In case you missed out on the whack fun in the main flash page, go back and try it.

Virus Hunting

I recently spent several hours ridding a computer of a virus which had managed to avoid virus scanners. The experience was interesting and quite exhilarating. I am sharing my learnings with the hope that you might find it worth your while and might have some of your own to share. If you find it useful, let me know :-).

The Infected

It was a Windows computer (of course ;-)), the Windows 98 kind and it belonged to someone who had a lot of confidential data - financial and otherwise - on his computer. I suspect he accessed a site which ran some malicious code which downloaded the virus and installed it. AVG was the installed virus scanner, and believe me although its a good piece of software and manages to prevent most viruses, it failed in this case. His machine began freezing up and he could no longer access Yahoo! or Google or other sites.

The Hunt

Hosts

check the hosts file, you say? and you would be right.

Sidebar : The hosts file

In Windows 2000, the hosts file resides in %SystemDrive%\WinNT\sytem32\drivers\etc and the entries in there are the first to be used when a DNS name is being resolved to its IP Address. So, if someone/something goes and puts 127.0.0.1 www.hotmail.com, your browser is going to ask Windows for the IP Address of hotmail and is going to end up trying to connect to your own computer. Search for the hosts file in your Windows directory if you are using something other than Windows 2000.

It had several entries covering all the popular search engines, mail sites, and other popular web-sites. Interestingly, this virus made a backup of the original hosts file and called hosts.sam in the same directory [talk about being nice]. Well, that explained why he was complaining about not being able to connect to Yahoo!.
Trying to overwrite, modify or do anything to the hosts file resulted in it being rewritten almost instantly. Until then, I had no idea how frustrating it can be to not have access to any of the web's search engines. Does anyone know how to make Windows not use the hosts file?
Ok, so obviously there is a malicious program resident in memory that is monitoring changes to this file. So the next step is to prevent this program from executing.

Cleaning up the registry

Perhaps the most common mechanism to hijack a Windows machine is to install the virus in one of the Run registry keys. So the next step was to remove everything unrecognizable from the following keys -

1. HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Run
2. HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\RunServices
3. HKCU\Software\Microsoft\Windows\CurrentVersion\Run

Use your judgement when you do this and keep a copy of the original settings because most of the stuff there is legitimate. Since I didn't have access to Google I could not get SysInternal's extremely useful software Autoruns. It provides a list of all programs that run when your OS starts up. Extremely useful! If you run it, you'll see that there are a lot many more registry keys to be concerned about than the three I was aware of.

Sidebar : Registry Jargon

HKLM = HKEY_LOCAL_MACHINE

HKCU = HKEY_CURRENT_USER

To explore the registry you should execute a program called regedit.exe. Be careful though, you could potentially stop Windows from loading successfully.

Relapse

Back to the story :-). There was a suspicious entry for Explorer.exe and when that was removed, the hosts file could be restored. Internet access was available again. So, gullible me, restored the registry to its older setup minus the malicious Explorer entry. Everything seemed ok for a while and then again the legitimate explorer started to freeze and took the OS with it.

File Times

I have had considerable success with locating malicious programs by doing a search for all files that were modified/created recently in the WINNT directory. A careful analysis of this output seemed to indicate various programs created on the day of the attack and all of the same size. All the programs had very legitimate sounding names like dialer32.exe, system32.dll, wintime.exe, etc. There were about 5 such files that I could locate with the same size and creation date. Comparing with another computer that he had which ran the same OS, these files were not part of the Windows installation. Another way of finding out if a program is legitimate is to simply type its name in Google. You will have links to sites that list all well-known programs or suspected viruses and will be able to provide detailed information on your queried file.
After I deleted all these files, the computer was restored to its original healthy state. I hadn't expected multiple entries in the registry. However, I do suspect that there were perhaps two different viruses because this one didn't seem to touch the hosts file.

Psychological Stuff

I recently had to clean my own computer of a nasty little virus that kept trying to download itself and execute although AVG wouldn't let it. For me, since I am just a lowly programmer with nothing of any value other than code on my computer, it was nothing more than an irritant. However, for someone who uses his computer for his business and stores all his personal data, financial statements it was such a gross violation of his personal space - it was as if, his house had been broken into.

Googlemania

My interest in Google continues.

Zeitgeist

Google has a wonderful site called Zeitgeist that collects interesting patterns of data based on the queries they get from around the world. Since Google is no longer something only the "geeks" use, there's more credibility to the diversity of data. Depending on what you like to believe, it shows up some surprising and some not so surprising data!

Take, for example, that 87% of the operating systems hitting Google are Windows and that MSIE still rules inspite of my ardent wish that we all used Mozilla Firefox which really is a brilliant browser.

Zeitgeist can probably do better though. Perhaps, top 10 hits for queries on technology - anyone has any better ideas??!!

Labs@Google

Google has even more interesting stuff. To get a sneak preview of their technologies that might appear in the future try labs.google.com. For example, the key board shortcuts technology they have in gmail was first previewed at the labs. While you are at it give Google Sets a try. If you were wondering how they do all this then try this.

HTTP Headers

If you stop to think about it for a second, its an amazing location to do an automatic survey. And it all happens because of something called HTTP Headers.
As the initiated amongst you have already guessed that the rest of this blog is going to be about me bragging about how much I know about HTTP headers you can give this a clean miss!

When you instruct your browser to go to a particular web resource, it asks the server at that location for the resource. As a part of the communication [which is not visible to the user], every browser sends information about itself, the operating system it is running on and which site it came from. It also returns a small piece of information called a cookie which is stored on your computer. This is a substantial amount of information and with 200 million hits a day, a wealth of information about usage. This information is sent through HTTP headers.

Here's an example of the HTTP headers sent by the browser [Mozilla Firebird] told to access Google's home page. Similar headers will be sent by other browsers like Internet Explorer.

GET /search?q=geek&sourceid=mozilla-search&start=0&start=0&ie=utf-8&oe=utf-8 HTTP/1.1
Host: www.google.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6) Gecko/20040206 Firefox/0.8
Accept: text/xml,text/html;q=0.9,text/plain;q=0.8,video/x-mng,
image/png,image/jpeg,image/gif;q=0.2,*/*;q=0.1
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Cookie: en_IN; PREF=ID=52eef62d74670ae2:
FF=4:LD=en:NR=10:CR=1:TM=1048623851:
LM=1069183668:S=c2RqGKqGOQXoZV8g

Note the parts marked bold. They specify the client (browser) you used to access Google and the Operating System you are running. The second line that is in bold is a piece of information that Google has put on your computer to identify you the next time you access Google. [this example does not have a referer header]

By the way, in case you are interested there are some interesting sites that discuss privacy issues regarding Google. Whether you would like to believe them or not is upto you :-)!!

Gmail Experiences

I was invited onto Gmail by Rajiv recently. Google continues to surprise and amaze with its excellent technological grasp. Its amazing how someone can redesign something as traditional as web-email so fundamentally. Some very core and interesting concepts -

• Conversation Grouping Most desktop email clients provide this. However, there is something very nice about the way Gmail does it. Perhaps its simply because its so new and its so fast that it makes it fun to use.


• Labels This is a really nice feature. Instead of grouping into folders you can apply labels to it which are actually pretty similar to filters or saved searches. The advantage is that you can apply multiple labels to it. Nice metaphor.


• 1GB space Their claim to fame. Already available with a host of other web mailing sites, Gmai l wants you to not delete emails ever. They want their search engine to be used to find emails. I haven't had reason to use it yet though.


• Smart UI Smartly written JavaScript [which even has Joel impressed] that tries to provide rich-client features to a web UI. Does a very good job.
  
  
  
  • Keyboard shortcuts are amazing. If you are a power user you will enjoy gmail.
  
  
  • The Spell checker is brilliant.
  
  
  
  


• Sizzzzling Fast Google remains one of the fastest services on the Net and Gmail rocks. It is so blazing fast, sometimes it just outpaces my desktop Outlook Express Client.

And Gmail is still in beta!!

The reason why Gmail is creating a controversy is because it scans your email to provide more relevant ads next to the message. Although, this has gotten people pretty upset, I for one do not mind it too much because of the other features of the site. The ads are quite interesting too. Sometimes they get it so right its quite amazing. And they make sure your email comes first and then the ads so they hardly bother you. They are always text based and occupy very little space. Yet, Google's AdSense program must be quite successful because its all over the place.

They have some interesting rules about what content they will show ads for. e.g. ads are strictly family viewable. A bunch of interesting rules are available here.

From Google's support forum -

Only ads classified as Family-Safe are distributed through our content network and to your Gmail inbox. For example, Google would block certain ads from running next to an email about catastrophic news.

Gmail is available by invitation only so do not try to look for a sign-in link like I did!

Gmail is causing Hotmail, Yahoo to wake up to email competition.
A Microsoft Hotmail employee talking about his experiences dealing with the Gmail threat.

MSN announces 250 Mb disk space just like Yahoo did. Very Reactive!

Internet Explorer and its 512 limit

A 404 page sent to IE has to be 512 bytes at least otherwise IE will ignore the page. What a wierd quirk.

The link has a list of other pages for which IE has different size limits.

Web Services for Integration

Finally, we are talking Integration as the primary use of Web Services. All the inital hype and hoopla about it being the panacea for diverse applications so that you can discover and use any application that suits your query is giving way to more sensible uses like integrating diverse systems.

SOA is becoming a huge buzzword. Although, Web Services are a way to achieve SOA, it is today considered almost synonymous. BEA is talking about SOA along the lines of integration too.

[BEA's QuickSilver project one implementation of this vision. While no shipdate has been announced, the BEA vision is to provide J2EE developers a console-driven suite of integration software built around Web services protocols, the purpose is to enable them to transport data and business rules between Java and non-Java systems with as little coding as possible, using abstractions and an integration-enabled container (tied into the development/deployment console).]

BEA is also criticizing J2EE 1.4 lagging behind the actual Web Services capabilities which is true because Web Services users have been asking for QoS features that are provided by other frameworks in that space. Today, a variety of WS specifications are covering this very important ground and J2EE is lagging behind. BEA is asking developers to "look out of the container" which could be more out of necessity because the spec is not up to date with what we want rather than following "their" advice.

My Experiments With Delphi

After several years I have again started working on Borland Delphi. I used to be quite proficient at it. Its interesting how my perspective for things have changed. It still remains an incredibly productive environment for Windows programming especially for a hobbyist programmer like me.

The environment has changed a little since the version 2.0 I was using and the version 7.0 that I have now. However, the changes are quite miniscule compared to the jump in versions. I believe this speaks volumes about their excellent design in their first few versions. They have extended it to support Web Services, XML, Internet programming, newer debuggers, a host of new things but the original clean, easy-to-use framework remains.

Delphi 2.0 used to have an excellent coding editor. Visual C++ had an equally good one. They had identical debuggers of very high quality. The debugging abilities have remained quite the same, I think. Their ability to always run with debug on is quite convenient.

However, all is not great. I believe Delphi's innovation at basic coding productivity is very very low.

Comparing with any of the existing Java editors like IntelliJ Idea, Eclipse, Pramati Studio and you will find an a variety of tools that are integrated into the editor that pop up "Just-In-Time" to provide an excellent coding environment. Both languages, Java and Delphi, have coding tasks that interfere with the basic thought process. Delphi has a whole lot more than Java. Java needs import statements too be added which its IDEs do automatically. Delphi needs function prototypes, variable declarations, imports and some more house keeping which the Delphi editor does not do. VERY IRRITATING.

All the Java editors do online validation of code so that you always know what went wrong and they will manage code completion irrespective of syntax and semantic errors. The Delphi editor after half a decade of evolution cannot do that. It simply says that code completion is not available. That is unbelievable.

JBoss - TSS Feud

Ramesh's blog on the ongoing JBoss and TSS (The Server Side) Feud capture exactly what I felt on this issue. All parties involved are getting equally unethical.

Interestingly Ramesh's blog is beginning to get quite popular - so popular infact that they are having a TSS style argument in the comments section of his article! Kudos!

Read all about the feud here.

How much do you use your tools?

To what extent do you use the tools at your disposal? We were involved, for the longest time, in writing a set of productivity tools for developers. We had excellent competition but our experience with our customers and their users was always that people knew very little about their tools and its capabilities.

How much do you know your tools? I have used IntelliJ Idea. Its a brilliant IDE. I have seen other people use it. They suck at it. They could as well use any other text editor with code completion. They have no idea what Idea can do for them (no pun intended).

Gosling, James

I had a chance to meet James Gosling at a meeting with HYSEA which he graciously (albeit in a slightly bored manner) attended and gave us a few minutes of insight into his thoughts on a variety of subjects and primarily on Java.

Some excerpts : (I am writing this from memory so the words arent the same ones he used and I shall try to the best of my knowledge to convey what I understood :-) )

What do you think about releasing a version of Java which embraces the Windows platform, e.g. providing easy access to the Win32 API without having to resort to JNI?
Developers are not feeling a need for such a release. The way Java works is that if a bunch of people need a particular feature say Scanning, then the go out and design a high level API that captures the essence and much more of the requirement which will have the default implementation perhaps only on Windows but the ability to extend it remains. Other examples are the NIO subsystem.

What would you design differently in Java if you got to design it again?
I wanted to make the syntax as close to C or C++ at that time because they were the most commonly used languages. At the end however, syntax is syntax. I would have changes the syntax of the switch statement.

What can we expect in the JDK 1.6 release?
Its too early to say what can be expected in the JDK 1.6 release. However, if you feel strongly about something you should talk to the expert group at the JSR. JDK 1.5.1 will have a release providing features similar to AOP.

What do you think about Java being taught in colleges?
Pascal used to be taught because if you made any mistakes, it was easy to explain them. In C if you go past an array, you find out about it in mysterious ways. Java has an easy to use memory-model and has the same qualities as Pascal. Pascal did not have too many commercial implications. Java has considerably many. Therefore using Java in colleges to teach programming makes a lot of sense.

And some more...maybe later

Common Problems

Recently I concluded an engagement which involved reviewing and commenting on someone's Java code. While submitting the results, the discussion turned towards some common problems. Interestingly we are grappling with the same problem at my work place too. How do you do error handling, good exception handling with effective user feedback?

It all drills down to being able to identify the exact cause of the problem and to be able to suggest solutions to it. It so turns our that this seems to be quite difficult to do.

Where does the difficulty lie? I think the difficulty builds over time. e.g.
If I write some code that does a file copy I will end up doing the following

public void copy(InputStream src, OutputStream dest) throws IOException;

What is wrong with this code? A bunch of things and I do not know what to fix in it.
IOException has about 20 different direct sub-classes. Out of which this operation could potentially raise over half of them. Most implementations handle this by simply catching it and if they do not ignore it (god forbid) they just show an IO Error message to the user. If you have a Java aware user, well good for you (my experience with Java aware developers are that they do not understand these exceptions anyway) but for a Java unaware user what do you say?
The only way you can provide effective feedback is to catch every single type of exception and try and figure out what it means. Would you find in yourself the motivation to do that?. You are still left with a bunch of IOExceptions ranging from not enough diskspace to remote file system is not accessible, permission problems etc. You are still constrained by the effective user messages of your library.

How does one solve this problem? Our customer asked us if we knew of any third party exception handling package that he could download. Now thats a killer application.


Check out who all are arguing about it too...
gosling@artima
Hejlsberg@artima

Easy Software Development?

Joel has an Excellent new article that I agree with very strongly. About how software development is considered to be something anyone can do quite easily. Most people do not realize the complexities of software development. Along the same lines, recruitment processes in most companies are driven by aptitude expecting that a person with a good aptitude (IQ) will be able to code better because its all about logic.
I dont think so.
I think its much more than just logic. Programming well requires a discipline, requires you to enjoy what you are doing, it requires a way of thinking that comes from practise as well as love for programming. Its like saying that spending 4 years in college learning software development is a waste of time. It doesnt teach you everything but it makes you much better than someone who learnt something else.
Joel touches on several other aspects of how software development is complex and involves a lot of variables. An excellent read!

I have come across recent examples of working code which is so terrible that its disgusting. These are examples of high aptitude people who are coding for the amount of time needed by when they can be automatically promoted to a position where they can delegate it to someone. These "managers" will of course continue to fuel jokes in Dilbert.

Interesting Performance Problem : The Usual Suspects?

An interesting performance problem got solved this week. One of our clients noticed dramatically bad performance in our Java servlet/jsp web-server compared to a competitor.

The usual trick to narrow these down are to take thread dumps at the time when the server looks busy. It kept showing us dumps that seemed like most of the time was spent waiting for the database to respond. A quick timing calculation showed that the database was taking a short fraction of the time. After introducing more timing stats between some large pieces of execution we managed to find a bottleneck only to see that it moved the next run - thus making us suspect Garbage Collection. Switch on -verbose:gc and there we were minor Garbage Collection Runs a full 8 and a half seconds.

The Usual Suspects are runaway streams, memory leaks, excessive object creation none of which could be caught in our JVMPI profiler. Amazingly, even after 3 days of intense debugging we knew it was GC that was leading to our problems but we could not isolate what it was that caused it!

We tried IBM JDK and its VM and we compared identical to our competitor. IBM JVM is an amazing piece of software but thats a story for another time.

None of the usual suspects seemed to be involved. We went back to commenting out pieces of the offending JSP and isolated the problem to excessive try-finally blocks in our code generation. We got rid of them and lo behold! we were as fast too!

Why would try-finally result in such high GC times in Sun JVMs? I can only suspect if GC happens when there are a lot of try finally blocks the code is unoptimized? Would love the answer to that one...