Category Archives: Agile

Choosing a data storage format

In case you haven’t noticed, XML is not a silver bullet. (google xml+silver+bullet). It is not, and should not be an automatic choice when thinking of a data storage format. The ubiquitous libraries for working with XML are often hard to use, and are often overkill for a simple storage format. In today’s world, I’d suggest that the following options should be considered (at least briefly).

  1. Native Object Serialisation
  2. Custom format
  3. XML – Extensible Markup Language
  4. YAMLYAML Ain’t a Markup Language (obviously created by geeks with the recursive name)
  5. JSON – JavaScript Object Notation

Join me in having a look at these formats, and I’ll let you know some of the issues to consider. The main problem I’m solving is for data that belongs to your own application. I’m not considering databases or interoperability.

Native Object Serialisation

Consider this briefly before running away. I’m particularly familiar with the idea of Java Object serialisation. I’ve used Prevayler in the past storing java objects, and xml (So while I’m having a dig at Java Object serialisation in general, I’m not specifically having a go at prevayler).

While the use of native object serialisation is often easy, it has costs, making the content unreadable by humans, coupling the data storage to your implementation language, and can create object migration issues. These costs will typically outweigh the benefits. Having human readable data to aid debugging would provide reason for not using native object serialisation if there was nothing else.

Custom Format

The use of a custom simple text format should not be discarded out of hand. The lack of any third party dependancies is a useful feature, and should be considered. That said, if you have a library that does the parsing for you, that should not be sneezed at.

XML

As wikipedia says, “XML is a general-purpose specification for creating custom mark-up languages” (Wikipedia on XML). Parsers and tools exist for many platforms and environments, which makes it a useful tool when you want to share information between different environments. While a good tool, the syntax is verbose, and can be hard for humans to read.

XML has influenced the birth of two of two more recent notations which are useful for data storage: YAML, and JSON

YAML

YAML purports to be “a human friendly data serialization standard for all programming languages” (Yaml.org). It has a well defined specification (YAML Spec), and makes for an easy to understand data storage format. Implementations of YAML exist for a wide range of languages, including Java, C++, Ruby and Javascript. It’s been around for a while, and has a decent amount of uptake. If it wasn’t for JSON, it would probably be a good default choice.

JSON

At first glance JSON seems much less suitable than YAML for languages other than JavaScript. The kicker against it is that it has “JavaScript” in the name, which has always made people feel icky. That said, it does make for a good cross platform format, it is human readable, and is implemented on a wide range of platforms (Json.org).

JSON has also has the advantages of having mindshare, and is slightly more familiar to developers than YAML. Every developer who has had anything to do with the web has done stuff with JavaScript, so the basic format will be familiar to them. Also in JSON’s favour is the fact that JSON and YAML are syntactically very close (see Redhanded). JSON appears to be very close to a subset of YAML(Ajaxian). In addition, the general applicability of JSON is higher, particularly for people who are going to be doing Javascript development. Also, if you have any possibility of playing with JavaScript, JSON is a very good option because of the native support in JavaScript.

These factors combine to make JSON an excellent choice.

Summary

Tim Bray makes a good case for this being an automatic choice based on your circumstances(http://www.tbray.org/ongoing/When/200x/2006/12/21/JSON ). You’ll still need to think about the pros and cons of the different technologies for your situation (see http://webignition.net/articles/xml-vs-yaml-vs-json-a-study-to-find-answers/), but you’ll often find that JSON is a good format to use for data storage.

Can the Government be Agile? – discussion with a PMP PM

I’ve enjoyed my relationship with my brother-in-law for a long time. We’ve often talked about business and career, sharing where we are at, and the directions we are taking.

Over the past few years Paul has been doing a great job as a project manager in Canberra, in particular his involvement with the Department of Defence in the successful and ambitious e-voting project (Remote Electronic Voting Federal Election 2007). It has been great hearing about his work, and some of the stories around it. He has also been sharing his growing understanding of project management, and some of the courses and information he has been collecting, and applying. I have been listening with interest, and a degree of skepticism, thinking that much of what he has been learning would not apply directly to me, as it seems to be focused on heavyweight/rigorous methods, and I very much prefer working in Agile environments, where good smart people are encouraged to do great work.

So it was with great interest that I was recently listening to him discussing and presenting some of his recent thinking and work around scrum. Paul has been looking at how to apply scrum to government projects, and has some great ideas about this. I was particularly interested to hear what he saw as the strengths that scrum would bring, and the advantages he saw in outcome based results (delivery of working software that meets outcomes) versus delivery based results (traditional waterfall delivery of documentation is part of the results).

I look forward to seeing Paul’s work going forward, and to seeing how he continues to apply agile techniques to his work. If you are in Canberra, and are looking for a great contract or consultant project management, it is well worth talking to Paul and the team at Barton Institute of Management.