I’ve enjoyed my relationship with my brother-in-law for a long time. We’ve often talked about business and career, sharing where we are at, and the directions we are taking.

Over the past few years Paul has been doing a great job as a project manager in Canberra, in particular his involvement with the Department of Defence in the successful and ambitious e-voting project (Remote Electronic Voting Federal Election 2007). It has been great hearing about his work, and some of the stories around it. He has also been sharing his growing understanding of project management, and some of the courses and information he has been collecting, and applying. I have been listening with interest, and a degree of skepticism, thinking that much of what he has been learning would not apply directly to me, as it seems to be focused on heavyweight/rigorous methods, and I very much prefer working in Agile environments, where good smart people are encouraged to do great work.

So it was with great interest that I was recently listening to him discussing and presenting some of his recent thinking and work around scrum. Paul has been looking at how to apply scrum to government projects, and has some great ideas about this. I was particularly interested to hear what he saw as the strengths that scrum would bring, and the advantages he saw in outcome based results (delivery of working software that meets outcomes) versus delivery based results (traditional waterfall delivery of documentation is part of the results).

I look forward to seeing Paul’s work going forward, and to seeing how he continues to apply agile techniques to his work. If you are in Canberra, and are looking for a great contract or consultant project management, it is well worth talking to Paul and the team at Barton Institute of Management.

I recently have been learning a number of different APIs. After reflecting on my process for learning I have ended up with the following strategy for working with APIs.

1. Find someone who has used the API in the past and get a brief introduction.
2. Read documentation about the product (starting at the central documentation site for the product, include any tutorials and getting started guides). The goal from this first step is to get a general background understanding of what the product is about, the core concepts and jargon and principles of the product.
3. Use the product without the APIs. If there is a gui to access to product that is available, ensure that you know this, and the jargon around the product. Try and ensure that the information you have from step 1 is grounded in reality.
4. Read more documentation around the APIs. Make the top goal for this round to get enough of an understanding of what you can do.
5. Explore the APIs to see what they expose of the feature. The set of features exposed by APIs often do not match up with what is exposed through the GUI. The most important spot to aim for is the subset of features that are supported by both the API and the GUI.
6. Google (as usual it’s your friend). Sometimes there are some useful results in google that are missed elsewhere. Start with searching for the product name, try adding in API/howto/tutorial/getting started keywords to filter down content if you get too many results.
7. Write some exploratory code to see how the APIs work, and what they do. This should enable you to know enough to actually write the code that you need.

All steps are required, as there are often features which are not easily discovered by just taking one of the above paths. One of the most important steps is ensure that you do understand the subset of what is supported by the API and the front-end. This helps to ensure that you are actually hitting the right part of the API.

From time to time there are options which are not well documented at all, and are only alluded to in technical documents. It can be worth contacting authors of documents if the feature is not clear. The coding and exploration of the system will need to continue after this. It should be expected to continue learning the APIs. At every stage there is value in talking to whatever experts/more experienced people you have available. Being able to quickly pickup a technology and move from no knowledge to confidence in using the API is a valuable tool for developers. Learning new technologies has been an important part of my job as a software engineer, and I expect that it will continue to be for many years to come.

In Case you were wondering…. <span class="replaceMe"></span> does not behave the same as <span class="replaceMe"/>  in ie, even though they are semantically equivalent. The second option will cause sites to crash, and not render if using ajax calls to update them (in particular with Prototype, with the use of Element.replace, or event innerHTML).

So, even though <span class="replaceMe"></span> should be equivalent to <span class="replaceMe"/>. Don't use the later if you want dom manipulation to work (which is pretty much the main reason to use an empty span tag in the first place.  The real moral to the story is Don't use empty span tags if you want to support the most popular browser on the planet.

Having an automated deployment procedure is an important goal for any project. This post talks about the the steps taken in making E2 autodeploy to WebSphere Portal in Development.  The below process assumes that the application has been deployed once through other means.

After a friendly pointer from AJ to an article about automating websphere deployment, I decided that the time had come to revisit trying to automate the deployment of our EAR file for E2. I had done bits and pieces of thi in the past using a combination of the WebSphere scripting client, and the WebSphere ant tool (like ant but built with WebSphere bits and pieces). The killer constraint for me is that my client machine didn’t have access to any of the WebSphere binaries (perhaps unsuprisingly Apple and IBM have don’t perfectly interoperate).

In order to work around thi sproblem I ended up leveraging one of the classic protocols (telnet), the websphere scripting client, samba (to mount the application server drives), and a bit of ant to glue it all together. The process outlined below is focused around the deployment of the application E2Application to WebSphere portal.

Step 1: Enable telnet

In Windows Server 2003 this is relatively straightforward. Simply go to your control panel -> services -> Telnet go to the properties, make it automatic and start it.

Step 2: Write scripts to start and stop the app server

I did this using tcl and ended up with the following scripts (the pythonistas may like to use jython to do the same):

stopApplication.jacl  

set cell [$AdminControl getCell]
set node [$AdminControl getNode]
set appManager
   [$AdminControl queryNames
   cell=$cell,node=$node,type=ApplicationManager,process=WebSphere_Portal,*]
$AdminControl invoke $appManager stopApplication E2Application


startApplication.jacl  

set cell [$AdminControl getCell]
set node [$AdminControl getNode]
set appManager
    [$AdminControl queryNames
   cell=$cell,node=$node,type=ApplicationManager,process=WebSphere_Portal,*]
$AdminControl invoke $appManager startApplication E2Application


Step 3: Write the ant code to compile and copy the application

After the series of standard building tasks, we used a simple copy task to copy the files to the exploded WAR directory on the server. (something like: ${portal.home}/WebSphere/profiles/wp_profile/installedApps/{cell}/{application}.ear/{app_war}.war). Since I develop on a laptop I added in a combination of available and fail task to stop me from accidently creating directories if I hadn’t mounted m

Step 4: use ant to telnet to the server

Make use of the telnet ant task1 to telnet into the server, and run the stop and start server jacl scripts. My Ant targets (passwords scrubbed) are below (excuse the duplication):

<target name="startApplication">
   <telnet userid="Administrator" password="${portal_admin_password}" server="${portal_server}">
   <read>Administrator</read>
        <!– change to the IBM AppServer directory –>
        <write>cd \ibm\AppServer\bin</write>
        <read>AppServer\bin</read>
        <!– Execute the jacl –>
        <write>wsadmin -conntype SOAP -port 10033
                -username wasadmin -password **** -f startApplication.jacl
        </write>
        <!– wait for the connected to process message, which will occur on the successful completion of the script –>
        <read>Connected to process</read>
        <read>AppServer\bin</read>
    </telnet>
</target>


<target name="stopApplication">
    <telnet userid="Administrator" password="${portal_admin_password}" server="${portal_server}">
        <read>Administrator</read>
        <!– change to the IBM AppServer directory –>
        <write>cd \ibm\AppServer\bin</write>
        <read>AppServer\bin</read>
        <!– Execute the jacl –>
        <write>wsadmin -conntype SOAP -port 10033
                -username wasadmin -password **** -f stopApplication.jacl
        </write>
        <!– wait for the connected to process message, which will occur on the successful completion of the script –>
        <read>Connected to process</read>
        <read>AppServer\bin</read>
    </telnet>
</target>

Step 5

mix ingredients and enjoying being able to have a one click deploy to WebSphere Portal.

Appendix

For security paranoid people with time on their hands, it is possible to do this using ssh and scp. The key parts will be step 3 and step 4. In step 3 we are using folder share and the ant copy task currently. It could be done with scp. In step 4 the use of telnet could be replaced with ssh.

1 - The telnet task depends on the availability of the commons-net jar on the classpath. (click here to download commons-net).↩

When reading through the Selenium FAQ, I noticed a piece of coolness that I didn't know existed in HTTP Authentication (Selenium FAQ - authentication).  RFC 1738 specifies the following scheme for specifying authentication information in a url //<user>:<password>@<host>:<port>/<url-path>.  I have been familiar with this for commanline based protocols (ftp/telnet and their secure equivalents) but didn't realise that it would work in a browser. In a browser you can specify http://user:password@server:80/path to do basic http authentication. While this ia a bad idea for usernames and passwords in real applications, it is a great tool for development purposes.  My firefox speeddial plugin now has development pages specified in this way.

Of course this approach is not a silver bullet, and other tools are needed. I have used this technique to access websphere portal and webapps hosted on a WebSphere Portal server, as well as applications using HTTP Authentication on Tomcat Servers. Unforutnately it doesn't work with the WebSphere administration console, but thankfully you can simulate the form based login using the following type of url: https://serverName:10039/ibm/console/secure/securelogon.do? action=secure&j_username=username&j_password=password 

The combination of //<user>:<password>@<host>:<port>/<url-path> tye urls, and faking login forms, makes it possible to access most web urls very quickly in a browser environment. 1

1 -Internet explorer doesn't support this for recent browsers (see: http://support.microsoft.com/kb/834489) ↩

There is always a bit of a culture in IT (is that an oxymoron?) around macho programmers.  Real programmers use VI/EMACs etc.   It is always fun to read some of the interesting stories, and urban legends around real programmers, basically all taking variations of the Monty Python "We were poor" sketch

The Story of Mel is one of my favourites, only upstaged by the tooltip on this XKCD Comic on Real programmers.

What's your favourite story (real or made up) of real programmers?

For the regular readers of my blog, you may be wondering why I am stepping over to the dark side, and using Eclipse. Not to fear – it is just for an Eclipse plug-in. Using the right tool for the right job is important to always do.

When recently using the SAP Memory Analyzer tool to look at a large (1GByte) hpprof output file, I was running out of Heap space, so was lookin for where to set the -Xmx and -Xms settings on Eclipes. It was somewhat harder to find than desired, and the internet didn’t tell my as quickly as I wanted, so here’s the answer (while the video at http://www.brooksandrus.com/bog_assets/screencasts/eclipsejvmheap/ was cool, a simple text blog entry would have made me happy).

The eclipse.ini file in Eclipse.app/Contents/MacOS is the file that needs to be edited. (not the one that for me was close in location /Applications/eclipse/configuration/ eclipse.ini was bad, /Applications/eclipse/Eclipse.app/Contents/MacOS/ eclipse.ini was good).

After finding the right eclipse.ini, setting the memory is straightfoward. Simpley update the -Xmx and -Xms settings (see: The Sun Tuning java guide for what these do if you don’t know).

BeanShell promises lightweight scripting for Java. Support for BeanShell is included with the Apache ant build tool, and it is targetted to be included as a part of the standard edition of Java with JSR-274. I’ve recently included BeanShell in a project, and will share what I have achieved with this, and some of the motivations and considerations when doing this.

My primary reason for adding BeanShell was that we use some heavy enterprise APIs. We are able to mock these out for development, but have experienced consistent issues when working with the APIs, primarily caused by missing or misleading documentation. This means that there is often a need to write code to understand the APIs. To deploy to the application server takes about 3 minutes, and the wrestling required to make the code work is often much more painful that expected. It is pain which motivates the creation of some nice tools, and this pain motivated the creation of a simple tool using BeanShell.

In it’s simplest form, the tool allows the execution of code on the server that has been input into a text area. This is a powerful tool for developers and admins, but is somewhat (ok very) risky from a security standpoint, and needs to be treated as such.  Getting BeanShell up and running is really straightforward.  I've only got a couple of recommendations and comments to make about doing so.

1. Integrating BeanShell itself is really straightforward. Simply have a servlet (action/jsp/whatever) which reads in a request parameter, uses an instance of the beanshell interpreter class to eval the string. The quickstart guide makes it clear and easy to get started (makes it quick to start ).
2. In order to make the interpreter useful, you will want to have add imports of the important packages for your application (such as to your Data Access Layer), and also the set some variables on the context. Prepend the imports to the scripts before they are eval'ed, and set the variables using the interpreter set(String, Object) method.
3. In addition to the result of the script, and the application variables that I put on the interpreter, I also add in a PrintStream, making it easy to output debug information.
4. Finally, it is really useful to keep track of what statements have been executed, and the results of the statements.  Save these to the session, and list these on the page. A simple javascript function makes it possible to replace the contents of the text area with previous scripts, and also the output, and result messages.  This rounds out a simple little tool nicely.1

By doing this, it is possible to explore APIs, and work at a very quick pace. Whenever something behaves unexpectedly on the server, one can quickly explore what has happened, and produce the updates to production code, which are required to make this work. BeanShell is a powerful tool for serverside development, and well worth considering adding to your toolkit.

1 - for extra credit, it is really easy to keep a list of previous statements on the page, and use the javascript above to keep track of the history.↩

… or how to setup your Oracle XE database so that JDBC database urls will work like they traditionally have.

For many years Oracle has been telling everyone Oracle SID != SERVICE_NAME.

Java Developers have always smiled and nodded with this statement as the oracle thin client jdbc url would always be something like:

jdbc:oracle:thin:@server:1521:SERVICE_NAME, or
jdbc:oracle:thin:@server:1521:SID 

which would work because databases always either had the SID and SERVICE_NAME as the same, or there was some magic that the Oracle listeners did to make this work.

Somewhere in the past couple of releases of Oracle this transparent mapping has ceased to exist. In particular Oracle 10G XE out of the box doesn’t support this sort of option.

It seems the new preferred way of doing Oracle JDBC URLs is:
jdbc:oracle:thin:@server:1521/SERVICE_NAME.

I discovered this information while working with making a WebSphere Portal instance use Oracle as it’s data repository. The documentation for doing this suggested using the old school SID style of url.  I tried this first as it felt familiar, but things didn't work.  Changing the url to use the PORT/SERVICE_NAME approach looked to mostly work, but didn’t quite spread the changes to everywhere required. I needed to run WPSConfig connect-database to update all the hidden bits of configuration around the portal installation. Unfortunately, this failed as the wpsconfig script calls out to programs that expect the url to follow the old school syntax (the stack trace show an index-out-of-bounds exception while parsing connect srings). I then had to go in and update the configuration on the database to enable connection via the old SID style approach.

This was done by working on the listener.ora file
(C:\oraclexe\app\oracle\product\10.2.0\server\NETWORK\ADMIN) to add in a SID_LIST_LISTENER entry for the database.

This was simply to add:
(SID_DESC =
    (GLOBAL_NAME = XE)
    (SID_NAME = XE)
    (ORACLE_HOME = C:\oraclexe\app\oracle\product\10.2.0\server)
)

to the SID_LIST of the SID_LIST_LISTENER

I ended up with a SID_LIST_LISTENER that looks like this:

SID_LIST_LISTENER =
    (SID_LIST =
        (SID_DESC =
            (SID_NAME = PLSExtProc)
            (ORACLE_HOME = C:\oraclexe\app\oracle\product\10.2.0\server)
            (PROGRAM = extproc)
        )
        (SID_DESC =
            (SID_NAME = CLRExtProc)
            (ORACLE_HOME = C:\oraclexe\app\oracle\product\10.2.0\server)
            (PROGRAM = extproc)
        )
        (SID_DESC =
            (GLOBAL_NAME = XE)
            (SID_NAME = XE)
            (ORACLE_HOME = C:\oraclexe\app\oracle\product\10.2.0\server)
        )
    )

This configuration works with an out of the box Oracle XE installation, and lets me use the jdbc:oracle:thin:@server:1521:SID.

As I recently posted, I’ve been using wordpress everywhere currently. It is simple and easy to use. I am, however, always prepared to learn new things, and use the appropriate tools for a job. So, after hearing drupal come up as an open source CMS at the CMPros summit in Boston, I figured that I’d try it out.

The basic install process was pretty straight forward. Download and untar the install available from drupal.org. I created the user for the database on my mysql instance, and followed the simple install guide to get it up and running, which finished with me having created an admin account and logging in as admin.

After completing the basic install, it was relatively easy to go and create content. The front page includes a create content link available, making it easy to create stories and static pages. I could go in and do this quickly, and get some nice rapid gratification by creating content. Unfortunately the urls were kinda ugly and drupalish. I wasn’t happy, figured that someone had to have come up with a way to do nice urls. I started with a quick look at the admins screen, and almost ran away after seeing the huge number of options.

Google came to the rescue, and I downloaded the pathauto tool, and started reading documentation. Getting the friendly urls definately isn’t as easy as in wordpress. I had to go in and edit the .htaccess and set the path option:  RewriteBase /~rob/drupal-5.3

Installing a module is pretty straight forward, and simply consists of copying it into the modules directory. It is then enabled through the gui, using: Administer>Modules. When I did this for pathauto I noticed dependancies on the path, and token modules, which needed to be resolved before pathauto could be run.

As token wasn’t installed by default I had to pause my installation of pathauto. So I was left with exploring path, and the rest of drupal. Path actually works ok. It makes it easy to specify friendly urls for your pages and stories, via a simple text field.

Pathauto works pretty well to. After getting it installed (just unpackaging the tars for pathauto and node into the modules directory) I was easily able to update the configuration to ensure friednly paths are created. Functionally this ends up making the urls work as well as wordpress. The admin interface also provides heaps of additional options (which I didn’t really need).

I’ve started putting together a proof-of-concept module for EditLive! as well. After some input from the legendary support engineer (Suneth), we got this working.

Overall drupal seems to be much more powerful that wordpress. Like wordpress has a rich module system, allowing people to customise and enrich their installation. It is a good enough system, and while I don’t plan on replacing my wordpress installations with it, I can see that it may be useful for bigger sites.