Logging parameters for Hibernate's SQL statements

I spent a little time solving a problem this morning that I had previously solved a month or two ago, so I thought I should write down the solution!

It is often handy to see the exact SQL queries that Hibernate produces when it is interacting with the database. In order for the queries to be logged, the log4j.properties file is modified such that the log4j.logger.org.hibernate.SQL property is set to 'debug'. The problem is that while the queries themselves are logged correctly, the queries' parameters are not displayed at all.

There is a log4j.logger.org.hibernate.type property in the same file, however setting its value to 'debug' does not work. The solution is to set this property to 'trace', and then the parameters will be logged after the query itself has been logged.

Bash Completion + Java

I think the title says it all, I was just checking something at work and needed to run one of our Java apps using the command line, I did the standard:
java -cp jarfile1.jar:jarfile2.jar au.gov.


and hit tab key twice out of habit and to my surprise bash had checked the jar files inside the command I was writing and presented me with all the class options!!

Bash auto completion just continues to impress me! Thank you bash, you have just made my life SO much easier yet again.

For those of you who have no idea what bash is, its a terminal shell used by default in Linux.. so for those of you out there not yet using Linux, here is yet another reason to do so.

OSX also uses bash as the default shell for the terminal, so I'll now have to test it on my macbook.. after a quick test the answer is no, OSX is either using an older version of bash which doesn't support this feature, or hasn't turned on/set up all the autocompletion features by default.

Website Normalisation

Well I've completed the Xena Website Normaliser. Unfortunately that was the easy part - the hard part is the Xena Website Viewer.

After playing around with the Lobo Browser I don't think it's quite ready to be used as our primary viewer for web sites. I used it to view a number of different Australian Government web sites (the National Archives, the Department of Defence and the Department of Immigration and Citizenship) and none of them were rendered at anywhere near an acceptable level.

This means that it looks like I'll have to go down the path I was trying to avoid: initialising a web server whenever a web site normalised by Xena is viewed, and using the web server to map the links contained within the HTML to the normalised files that Xena has produced. So not the most elegant solution, but I think actually being able to navigate properly around our normalised web sites is reasonably important.

The Lobo Project

The Lobo Project is an open source, pure Java web browser. It looks like it could be useful for viewing complete websites normalised by Xena, as well as individual HTML files and the HTML preview produced from OpenOffice.org documents. It has support for JavaScript and CSS so should be much better than the default Java HTML viewer which can only handle the simplest of HTML.

I'll be experimenting with it tomorrow as I begin work on the Xena website normaliser.

Outlook 2003 PST files

Xena uses an external program called 'readpst' (from the libpst project) to normalise PST files from Outlook. PST files are exported from Outlook to archive users' mail, etc, but it is a binary Microsoft format.

Unfortunately, readpst does not support PST files from Outlook 2003 or later. Or so we thought. Just for the fun of it, I tested the latest version 0.6.38 on an Outlook 2003 PST file and it worked!

We are now going to perform some more complete tests, but initial results look promising :-)

Chris

Postgres on OS X 10.4

Yesterday we received our shiny new MacBook Pro laptops. So I've been spending a lot of time installing everything needed to make it the awesome development machine it should be! After wasting a lot of time getting Firefox running properly (after reinstalling it about 10 times the problem turned out to be with Google Toolbar - the problem only manifested itself after the second restart of Firefox which made it hard to pinpoint the problem) the biggest hurdle has been Postgres.

I used developer.apple.com's PostgreSQL on Mac OS X instructions as a base, but because of our NTLM proxy, and the fact the the MacBooks are brand new, there were a few extra steps that needed to be made. The full sequence is listed below for posterity:

Install ntlmaps
The NAA uses an NTLM-based authenticating proxy, which cannot be used by software package managers such as fink and apt. To get around this, download the source for a python program called ntlmaps from its sourceforge downloads page. Unpack the contents and copy the whole directory to the mac's Applications folder. In the main ntlmaps directory, edit the server.cfg file to suit your proxy (I needed to edit the parent proxy address, parent proxy port, domain name, username and password fields. Yes, unfortunately the password is stored in this file in plaintext - if you don't mind starting up ntlmaps manually each time you can leave the password field blank and ntlmaps will request the password when you start it). Finally, run the main.py script to start the ntlmaps proxy.

Install Fink
Fink is a package management system for OS X, based on apt. It's the easiest way to install postgres as it means postgres doesn't have to be built from source! Follow the instructions on the fink download page to install fink.

Set the http_proxy environment variable
Apt needs to know about the ntlmaps proxy. In your system-wide configuration file, add the following lines:

http_proxy="http://localhost:5865"
export http_proxy

To find the system-wide configuration file that the mac was using, I referenced Configuring Your OS X Unix Environment and found that I was using the bash shell, and so I added the above lines to the /etc/profile file.

Install Postgres using apt
Firstly, update the apt cache of packages by executing the following command:

sudo apt-get update

Next we need to find the correct package to install. Use the following command to search for postgres packages:

sudo apt-cache search postgres

Install the most up-to-date package in the returned list of packages. For me, it's postgresql80. The package is installed with the following command:

sudo apt-get install postgresql80

Follow the prompts, and postgres is installed!

Setup root user
The postgres user is created during the postgres installation, however the user is not informed as to its password... which is needed to do just about anything useful with postgres. To set the postgres user's password, we need to be root. However the powerbook does not initially have a root password set so we need to change this first. Use the command:

su passwd root

and follow the prompts to change the root password.

Change password for postgres
Use the following commands to change the password for the postgres user:

su
passwd postgres

Create pgsql data dir
Create a directory in which the database tables will live. I chose /usr/local/pgsql/data. Run the following commands to create the directory and give ownership of it to the postgres user:

sudo mkdir /usr/local/pgsql/data
sudo chown postgres /usr/local/pgsql/data

Fix the postgres user
The postgres user created by the postgres install process is not set up correctly - its home and shell properties are set to null. To fix this, run NetInfo Manager (I used spotlight to find it), click on "users" and then "postgres", and change the "home" property to /Users/postgres, and change the "shell" property to /bin/bash.

Increase shared memory
The first time I tried to run the initdb command I got an error saying that there was not enough shared memory to run that command. It's probably because I had eclipse open at the time, but I decided it was probably a good idea to increase it anyway. To increase shared memory, edit (or create if it doesn't exist) the /etc/sysctl.conf file so that it contains:

kern.sysv.shmmax=16777216
kern.sysv.shmmin=1
kern.sysv.shmmni=128
kern.sysv.shmseg=32
kern.sysv.shmall=4096

This will increase the shared memory from the default 0f 4MB to 16MB.

Run initdb
Run the following command to initialise the database:

initdb -D /usr/local/pgsql/data

Start postgres
Now we're ready to start postgres! Run the following command:

pg_ctl -D /usr/local/pgsql/data -l logfile start

And we're done! It's that easy!

LCA 2007 CFP open

The 2007 Linux Conference Australia is to be held at the University of NSW in Sydney. The Call For Papers has opened. I wonder if I can:

1. Get permission from NAA to speak there
2. Come up with a technical enough presentation to impress the LCA papers committee
3. Get selected to be a speaker
4. Actually write the presentation
5. Get the presentation approved by NAA executive and editors

The fallback position of course is that I take a week of recreation leave and attend the conference anyway.

Xena Audio Plugin

I've finished my half-assed Xena Audio Plugin. It can normalise a number of audio formats, including WAV and MP3, to fLaC format, and when these xena files are "viewed", the audio files can be played back with a simple player, consisting of a play button and a stop button. Unfortunately it's not written purely in Java - similar to the PST normaliser I've put a Windows executable of the fLaC encoder in the winx86 directory, and used this from the Java program to perform fLaC encoding. I had actually started to write the Java version of the encoder, but as I delved deeper and deeper into layers of C code and header files, I realised that there was no way I was going to be able to complete it before work commenced on the primary Stage 10 products.

So we can now normalise audio files... but it's not exactly ready for prime-time yet! It's been added to sourceforge as the audio-prototype module. Hopefully someone else will produce an open-source Java fLaC encoder before we're forced to do it ourselves...