Puakma: Under the hood

I'm Brendon Upson, jack-of-all-trades, master of one or two. I'm talking about life running a small ISV tackling business issues and leaping technology hurdles in a single bound.

webWise Network Consultants is based in Sydney, Australia and develops the groundbreaking Tornado Server technology.

Custom software development outsourcing

Filed under: by Brendon Upson on 2005-06-30

I was thinking today about the consulting projects we have completed over the last year or two, ranging from successful to painful. I started to think about the factors that differentiate these two broad project types and it occurred to me the human factor was a critical element.

Those projects that involved pain were typically those which we performed at arms length, like, "You send us the requirements and we'll build it". This type of project typically causes the rubber ball effect where new requirements and bug reports get thrown over the fence to us, while we do the work and throw back what we think the customer wants. The fence in the middle stops the human elements being effective, as all means of "normal" communications are stopped, or reduced to email, a completely emotionless communications medium.

The more I thought about this, the more it seemed that there is so much you learn by being physically close to your customer. The primary advantage is that you learn their business completely by observing conversations and events that are otherwise missed. Face to face meetings allow you to observe how people's body language betray what they are telling you. You know in an instant if what they are saying is what they neccesarily believe or whether there is more to the story. In email and on the phone these subtle gestures are lost, allowing the solution provider to unknowingly continue down the wrong alley ever toward impending doom.

Obvoiusly this customer loving comes at a cost. The more love you give the customer, the more time it takes and typically that time is not "productive time" (such as actually building the solution). Someone needs to pay for this time and that someone is the customer, meaning a higher overall cost for the project. Fear not, the higher cost also brings with it a much better fit and generally far smoother running project.

In future, we will endeavour to always work closely with our customers, even if it means losing some jobs due to the increased cost. I'm sure it is better for our reputation to miss out on a few jobs than to have disgruntled customers spreading the word.

Standards and market leaders

Filed under: by Brendon Upson on 2005-06-29

From time to time we come across issues with internet RFCs. The RFC will say something about how a client or server should behave, but during testing we find that the market leader (usually Microsoft Internet Explorer) does something a little bit different. This creates a dilemma, do we:

a) Stick to the spec because the standard is correct (and exclude 90+% of users)

b) Make it work with IE, breaking the standard and further reinforcing their market dominance

Tough questions. The ultimate issue is that we need to eat and need to make products that are useful to our customers. This usually means heading through door b. It's annoying as a developer to either purposely break something or introduce a ream of inelegant code which amounts to a kludge to make things work. 

The experiment

Filed under: by Brendon Upson on 2005-06-23

A week ago I decided to run an experiment. We had a new developer start and when we gave him his laptop we decided to see what would happen if we made him use Linux (Fedora FC4, the latest and greatest) as his desktop machine. It was interesting.

My theory was that if we were forced to use open systems (no vendor lock-in), then the solutions we provided to our customers would be more open also. Win-win. I personally use an Apple OS X Powerbook, so I figured "I'm using *nix now so how hard could it be to run Linux on x86 as your daily machine?"

...the initial install was simple, but then things went south.

This post is not a "Linux sux" post (so lay off the flames ok?), I am merely stating how our personal experience trying to use it as a web developer's personal desktop was not a resounding success. Did it work? Yes, functionality-wise the system operated and did so moderately well. The problem was the setup time. It involved literally hours of tweaking. We have all seen the Microsoft Windows TCO charts etc saying how much cheaper Windows is. I am now a believer that in our case, Windows is a cheaper solution. Sure it costs us dollars up front to buy XP Pro, but we now have a machine installed and configured with all the software we need in 4 hours. The Linux test had taken 3 days already and was still not working reliably. We pay this guy by the hour - admittedly he didn't waste the full 3 days setting the thing up, but this still represents a substantial chunk of lost productivity for us.

The biggest issues we had were VPN setup and JVM setup.

The standard JVM that comes with Gnome is gij. What the hell that is I don't know. It's java but the commandline parameters are just that bit different to Sun's that all our scripts needed a tweak to make them work. So we decided to install the Sun JVM (we need 1.4.x). After install we can't make network connections due to some obscure error (SocketException). This looks like a known problem with Fedora FC4. Sun JVM 1.5.x works fine. Except if we compile using 1.5, it will break our customers 1.4.x installations. More tweaking.

The VPN had to be made from source (...of course it does) and runs in a command shell. Fine. Except on connection the DNS server addresses did not get pulled down correctly. The VPN worked, but only if we specifed everything by IP address. More tweaking... Meanwhile the Gnome UI still looks cheap and clunky.

And so the tweaks mounted and the time ambled on. At midday today I pulled the plug and we installed a spanking new copy of Win XP Pro SP2. It took 4 hours total to install the OS and all the apps. No tweaks, no glitches. We now have a productive employee.

We love Linux servers. If I were installing an app server tomorrow I would definitely recommend Linux over Windows. On the desktop? Windows still rules. It's gonna take a while to knock MS off it's perch as top dog. Apple is the only company at this point who looks like even coming close to doing it.

49 hours a day

Filed under: by Brendon Upson on 2005-06-20

The last couple of weeks have been manic. So much going on! It's a great sign, but certainly bad for my mental health. One thing is clear - you have to take time off periodically to recharge.

Running a business is a funny thing. You're a lawyer one minute, then accountant, then HR manager, then marketing manager, then programmer. I love the challenge of learning new things and trying my hand at whatever the day throws at me. Trouble is, as the workload for programmer goes up (good sign, because ultimately this is what customers pay for) the relative time available to spend on the other tasks drops. And you find yourself doing them outside of the "normal" 40 hour week, weekends, public holidays etc.

Then there's the fire fighting. Out of nowhere a "completed" product or project bursts into flames and requires some drop everything immediate attention. This is never good but happens. It's not always because of any bad software or design, but often the client starts using things in a way they were never designed.

Sometimes it is bad software. Like last friday. Kudos to Andrew Tetlaw for uncovering a rather ugly bug in our NTLM Single Sign On product. It's only through feedback from our customers and a focus on quality that we can deliver great products. Needless to say, friday night was a late one spent programming and testing (not at the pub where most sane folk are). The bug was ugly and we had to fix it. Period. So we did.

Snowcam has now been checked and I'm thinking about a snowboarding trip in the next few weeks. Call it a mental health break ;-)

Websphere tokens

Filed under: by Brendon Upson on 2005-06-14

I'd like to report otherwise, but this Websphere token decoding is proving a mite more challenging than I had first expected. The kicker is in the trailer.

The way the Domino Ltpa token works is by hashing the first part of the token (one-way MD5 fingerprint) into a chunk of bytes. These bytes are the appended to the token a little like a tamper proof seal. If you change any of the first part (name, expiry date, creation date) then the finger print hash will not match and the token is therefore invalid. Quite clever really.

I believe the trailer on the websphere token works in the same way, but it's not a one-way MD5 hash. I think it's either public key encrypted or signed. Trouble is we're having a real struggle with the RSA data. There is a field on the Domino configuration doc called WS_RSAData, but RSA is a public/private key PAIR. Here we have only one field. To make matters worse, it's 259 bytes long. 2 * 128 != 259.

Next step was to try to decode the websphere export file directly to reveal the public and private keys (since they are in the file seperately). Again unfortunately, it would appear that you need some salt to reverse the password based encryption. Salt is 8 bytes of "random" data to make guessing the password difficult. To decrypt you need to know: the password, the salt and how many iterations were used to encrypt. This means a brute force attack take so long that it's not worth it.

Back to the drawing board. Unless some kind soul who knows a ton about encryption has some clues.... 

When is an Ltpa token, not an Ltpa token?

Filed under: by Brendon Upson on 2005-06-08

I confess. We don't have a copy of Websphere (or Websfear as it is sometimes known). Dilemma: Customers are testing the Web Booster Single Sign-On (SSO) module with Domino and Websphere - and find it doesn't work.


It took a deal of patience and time to decipher the Domino Ltpa token and foolishly we assumed that IBM would have a standard format. Bzzzzzt. Wrong. The Domino token is 100% different to a Websphere token (why??? these departments should talk - really!).

To date we are able to pull a Websphere token apart and see what's inside it. Next step (with the help of a few customers) is to work out how each part of the token is generated and be able to verify a token, then create one from scratch. I really enjoy these sorts of puzzles - the challenge is up ;-)

Now it occurs to me some of you out there may not know exactly why decoding these tokens is so important to us. We have some web server acceleration software Puakma Web Booster that has an additional plugin module that allows users to seamlessly access a Domino server without actually entering a username and password. We achieve this by challenging the browser (NTLM authentication in the background) for the workstations Windows NT credentials and using those to find the user in the Domino LDAP directory. Once we have located the user, we make a Ltpa token and tell the browser to use it with every transaction with the Domino server. The Ltpa token contains the canonical username, creation/expiry date of the token and a signed area which confirms the token has not been tampered with. The token is passed as a cookie from the browser to the server. By using Booster for SSO another of those annoying username/password prompts are avoided, which as you know equals happy users. Happy users = happy tech staff.

Unique Selling Proposition and Value

Filed under: by Brendon Upson on 2005-06-06

Sometimes you feel like you're in a rut and although you are paddling madly to go forward, it feels like you only have one oar in the water. For all the drive and enthusiasm you are merely circling.

So I did something about it.

I'd been thinking for a long while about finding a business coach or mentor to give WNC (and myself) a better sense of direction. I had been put off by all that schmaltzy "person empowerment - ra ra ra!" stuff so approached the whole exercise with much caution.

The key to coaching is goal setting, refining those goals and building a blueprint of small steps to achieve them. Sounds easy and obvious but it's not. Hey, if it were then everyone would be a huge success.

So far I have discovered two important things. 1) Our Unique Selling Proposition (USP) is a bit very ordinary. We need to define a better as a business exactly what we do and really project that message. 2) Personally, I constantly undervalue everything. I need to make some mental changes to objectively acknowledge what we as an organisation are truly worth.

Every day I learn and every day I get better. The other oar is beginning to realize where the water is.