Sunday, August 24, 2008

The Scientific Method and Test Driven Development

According to wikipedia the Scientific Method:

Scientific method refers to a body of techniques for investigating phenomena, acquiring new knowledge, or correcting and integrating previous knowledge. To be termed scientific, a method of inquiry must be based on gathering observable, empirical and measurable evidence subject to specific principles of reasoning.[1] A scientific method consists of the collection of data through observation and experimentation, and the formulation and testing of hypotheses.[2] Although procedures vary from one field of inquiry to another, identifiable features distinguish scientific inquiry from other methodologies of knowledge. Scientific researchers propose hypotheses as explanations of phenomena, and design experimental studies to test these hypotheses.

and Test Driven Development:

Test-Driven Development (TDD) is a software development technique consisting of short iterations where new test cases covering the desired improvement or new functionality are written first, then the production code necessary to pass the tests is implemented, and finally the software is refactored to accommodate changes. The availability of tests before actual development ensures rapid feedback after any change. Practitioners emphasize that test-driven development is a method of designing software, not merely a method of testing.

Can you see the parallels? Test are like experiments for the universe of predicates that describe your system requirements.

Thursday, August 21, 2008

JConsole Config

Here is an excellent guide (written by Mike Schouten of Componative) on using and configuring  JConsole:

I found Part 2 specially interesting, because it explained that JConsole uses  2 ports to connect to a remote JVM, and only one of them is configurable the other one is random, the way to detect it is to create a logging.properties file that looks something like this:


handlers = java.util.logging.ConsoleHandler
.level = INFO

java.util.logging.ConsoleHandler.level = FINEST
java.util.logging.ConsoleHandler.formatter = \
java.util.logging.SimpleFormatter

// Use FINER or FINEST for javax.management.remote.level - FINEST is
// very verbose...
javax.management.level = FINEST
javax.management.remote.level = FINER


Now we can start JConsole with the following command to enable detailed logging of the javax.management classes:



jconsole -J-Djava.util.logging.config.file=logging.properties



Connecting to the remote JVM will now result in the opening of a separate output window containing the detailed logging, in that output it is now possible to see the random port:



image

Saturday, August 16, 2008

J2EE Application Servers and Eclipse plug-ins: Like going back in time

All this Application Server development is funny... it is like going back to 16-bit Windows (remember, back then,  if a single application crashed, everything went down with it). It is the same with this modern application servers, if one application goes crazy (infinite loop, very long and slow process, memory leak, etc), it will take down the application server, and there is no way around that that I know of (Same thing with Eclipse, if one plug-in goes crazy, everything goes down) it is funny (and sad at the same time) because so much time and resources were wasted trying to give processes proper isolation at the OS level, and now we are using technologies that make that effort irrelevant.

Perhaps, one day there will be a Multi-tasking Virtual Machine (MVM) that solves this problem by providing an efficient and scalable implementation of and infrastructure  for multiple, isolated tasks, enabling the co-location of multiple server instances in a single MVM process. Such a technology would also enable the restructuring of a J2EE server implementation as a collection of isolated components, offering increased flexibility and reliability. But in the mean time, we are left with application servers where that are inherently unsafe. And that is really bad, specially for development and prototyping(lots of time waiting for the shared components in a J2EE application server to reload if the application in development crashed it), but also for production (it is really risky to deploy a new system in to a J2EE, if it crashes, it will take down all the other applications). And the same things happen to “integrated” solutions like, Eclipse, or more general technologies, like OSGi, if a particular plug-in or service crashes, it will crash the entire application.

But the future for the MVM doesn’t look so bright, the JSR 121: Application Isolation API Specification looks abandoned since 2005… Perhaps for the JDK 7? or 8? I hate when I see this kind of project die from indifference… Perhaps now, with the Open JDK, someone will find the way to implement it.

This is getting repetitive…

Software development, that is… it is getting repetitive:

  • Design the database (the database schema represents facts, but those facts will be incomplete)
  • Code the business logic (Queries, queries, queries, in uncomfortable semi-standard SQL, there will be problems with:
    • performance (inefficient queries, lack of indexes, degradation because cascade updates or deletes or because of integrity verification)
    • concurrence (overwritten data, death locks, inaccurate reads)
    • code maintenance( SQL is ugly, cumbersome, hard to refactor, hard to modularize)
    • ORM is either inefficient with rich domain model, or efficient and with an anemic domain model
  • Code de User Interface
    • If is a heavy client, it will be ugly (because people erroneously think that only web based applications can be pretty)
    • If it is built in HTML as they current hype dictates it will have browser compatibility problems
    • It will not map easily to the database model  / domain model (specially report like stuff).
    • If it is a web application, it will use convoluted and not type safe communication between the different screens (server side navigation) or, it will use lots of JavaScript and essentially will a a one page application, and therefore it will become :
    • In heavy clients all navigation will be deceptively simple and type safe, but that will lead to spaghetti like dependencies between the screens.
  • Test
    • Security concerns will be left to the end, and most queries will need recoding because what data can be seen is always dependant on the privileges of the users.
    • Queries will return the wrong data when they are cross compared (a summary report like screen will say that earnings were of the day were $1352, but if you sum the amount of each of the sales, it will be $1456, or $1265, or something else. The most common reason? a status field that you forgot to take in account and that indicates that, for example, one of the sales was canceled, or one of the sales was in a foreign country money, or you forgot to take in consideration a discount.
    • Dependant UI controls (like chained comboxes) when used in an unexpected order will fail (or, if used to save data, might fail when editing that data).
    • Some controls that are only comfortable to use if the list of rows displayed in them is short (less than around 25 items), will have list of more than 250  elements, sometimes even thousands of elements, and will need to be replaced by search screens.
    • Nobody will test the performance of the applications for the large amount of concurrent users that will use the system in productions.
  • Production
    • Some tables, with supposedly immutable data, will need CRUD functionality, since there is no time to create the CRUD screens, data will be inserted directly using POSQL (Plain Old SQL) , and sooner or later someone will have to edit or delete some data, make a mistake with the WHERE condition, and destroy a large amount of rows. The last backup will be a lot of weeks old, and data will need to be recaptured.
    • Since there were no performance tests, the processor will go 100% and RAM will not be enough, the server will be incredibly slow, or plain crash.

Wednesday, August 13, 2008

Navigation: No hard dependencies, no type dependencies, no code dependencies, Why?

In Java, first we had servlets, and each one of them answered to request made to a particular url (or an url matching a particular expression), but it was hard to create pages that way, and it was even harder to create a system composed of many “pages” (note the conceptual jump from servlets answering url requests to "pages”)

Then, JSPs were created, take a look at them and you will see that each JSP page knows nothing about any other, JSP was designed thinking that the best way to deal with many pages was to handle it them as “independent”: lets make each page unaware of what page is next (and what page was previously). Just as servlets were somehow unaware or each other.

After that, JSF was invented, this new technology sees "Pages" as components (composed of other simpler components) forming a component tree. All the components in the same page collaborate to give life to a JSF page (but it does it in a pretty limited way, unless you use Facelets, but they weren't invented until later, and they didn't help much with navigation). By now, we should be far far away from the servlets with which we started... but still each page is unaware of what page is next (and what page was previously). Just as servlets were somehow unaware or each other.

Why?

Then, we got Seam, with its powerful bijection, to help us deal with the job of exchanging information between pages. And its pages.xml files, to help us deal with the limited navigational capabilities available with plain JSF. But still each page is unaware of what page is next (and what page was previously). Just as servlets were somehow unaware or each other.

Why?

In a somewhat parallel universe, perhaps in the Twilight Zone, WebObjects was created, in WebObjects, we have no servlets, "Pages" are components (composed of other simpler components) forming a component tree. All the components in the same page collaborate to give life to a WO page. And if you want, you can treat a WO page as a WO Component, they are pretty much the same thing, at it is trivial to change from one to the other, and create libraries of WO Components (comparatively creating JSF components is so hard that is mind boggling). And each page is aware of what page is next (and what page was previously). Let me state that again: The code in each page is aware of what page is next (and what page was previously). Here is a pseudo code example:

public WOComponent gotoProfilePage(){

ProfilePage profilePage = new ProfilePage();

profilePage.setUser(this.getSelectedUser());
return profilePage;

}


And that's it, that is how you send the currently selected user from the current page into the next page, plain and simple, you just create an object of the type of the page where you want to go, and use its getters and setters go share state with it... what could be simpler than that? why I have been unable to find a framework that deals with page navigation like this? Well, you might say, it is not that flexible, what if you want configurable navigation? Well, then you can do it like this:

public WOComponent gotoProfilePage(){

WOComponent profilePage = this.getPageWithName("ProfilePage");

profilePage.takeValueForKey(this.getSelectedUser(),"user");
return profilePage;

}


Any page with the name "ProfilePage" that has a "user" attribute will used for this navigation action. So... the question is still in the air... why? why I have been unable to find a web framework that deals with navigation in this way... why? We have frameworks that create dependencies between otherwise only dynamically related stuff, like Hibernate, creating properties in java objects that mimic the dynamic relations in a database, but we refuse to offer the same services for page navigation... I don't get why... this seems as such a natural way to deal with this kind of problem... is there a disadvantage here that I am not seeing? Or was WebObjects really created in the Twilight Zone?





Update: Just discovered another disadvantage of xml based navigation in JSF/Seam: It is not possible to use the java debugger or logging frameworks "out of the box" to debug the conditionals in the xml file, one has to wait for (or build) a custom debugger/logger for JSF/Seam navigation.

Requirements Analysis: Negative Space

A while ago, I was part of a team working on a crucial project. We were confident, relying heavily on our detailed plans and clear-cut requi...