Puakma: Under the hood

For those of you that know me, you know I like to indulge in a little DIY. You also know I like tools (Mmmmm powertools). On the weekend I started building a wine rack for our newish kitchen (I started the renovation in January and it's nearly done). The big challenge I find when hitting the hardware store is: Will a 3 dollar tool perform an adequate enough job when compared to a 20 dollar tool?

In short, a 3 dollar set of hole saws is worth somewhere less than 3 dollars. After drilling the 9th hole the metal sheared and my drilling day was done. Last night I hit the store again and (this time) bought a 20 dollar hole saw. Much thicker metal :-)

Is this the same with software?

This week has been huge. Busy. Really busy. Not much time for sleeping and I'm starting to feel it, luckily the weekend is here ;-)

One of the big tasks for the week was to improve the http logging in web booster and tornado server. From the outset we had supported one log format (IIS 5.0, ok shoot me) but a few customers have asked about supporting different log formats. Investigation proved the best way forward was to go with the Apache mod_log system. This approach affords an almost limitless number of logging combinations with each variable to log specified as a %x, for example: "%h %l %u %t "%r" %>s %b".

Our internet server was upgraded on Tuesday where everything that could possibly go worng did. Floppy containing the bios gane bad, kernel panic when booting from the Fedora install CD, fried boot sector which killed grub. The job took 6 hours instead of 2, but I guess that's the norm for computer upgrades.

Fingers crossed next week will be a little quieter. Only a little or it wouldn't be fun ;-) 

For about nine months now we've been working on an application for a customer to manage their construction projects. IT IS AN ABSOLUTE MONSTER! I kid you not. We are about 80% done and the current table count is 32. 32 tables! As the architect of the solution I found it very hard to get my head around how this would all logically fit together. The application basically tracks a project's progress against the quote. Remember we're talking construction projects here so some of the numbers get pretty big (like hundreds of millions). Also the number of items to manage in a quote are in the thousands. Then each month we do a forecast on each item to determine the project's position. Each month thousands of rows are created for each projects as a snapshot of the project's state.

This is easily the most difficult project I've been involved in. I love a good challenge but there were times when I doubted what the client wanted was even possible.

We upgraded a customer's Tornado servers today to the new 3.22 release. These machines power a mine management system we wrote and get a reasonable pounding each day, 7 days a week (mines never sleep!). I was most pleased when I saw the shutdown message at each site:

PUAKMA Shutdown (Uptime: 44d 7h 18m 22s)

and

PUAKMA Shutdown (Uptime: 27d 9h 19m 47s)

Total outage for each site was around 15 seconds and performed from 1,500km away. The whole upgrade was performed using only a web browser. 

I'm still amazed at the number of consultants that tell their customers they should really be reboot servers weekly. One customer with a Sunfire E15K (read HUGE machine with 106 CPUs) was recommended by an IBM consultant to restart it each sunday! Rebooting machines is a sign of poorly written software.

Yesterday I bit the bullet and ordered a data projector. I'd got by without one until recently but with the falling prices and increased need I decided it was time to procure.

Ten years ago I would have considered myself a technical guy. I knew all the mobile phone models by heart (amongst other trivial technical matters). Today I'd be flat out naming even 5 mobile phone manufacturers. In a very short space of time I have become my mother. [Insert cold shiver here] [luddites]

Yesterday, trying to decide which was the best projector for me was tough. Small and light enough to lug out to customers, bright enough because I don't know what their walls are like, and has to fit the budget.

DLP or LCD? I had picked out an inexpensive NEC that I had all but made up my mind on - until I went to see it in action. It was like looking through dirty venetian blinds. Next to it was a tiny HP DLP projector worth a cool 4K. Brilliant picture but well outside the budget. Hmmm. Spring for the few hunderd extra bucks methinks and definitely look for a DLP.

After about four hours of checking over ANSI Lumens, contrast ratios, price, weight, size, bulb life and replacement bulb costs, I settled on the Toshiba TDP-S25. How any "Average Joe" can wade through all this techno-babble and make an informed decision is beyond me. I suspect a sale is usually preceded with a "Oooooooh that one looks pretty!".

After ringing a few different people (thanks guys!) I realised the margin in these things is waaaaaaay slim. Glad I'm not a hardware guy.

So, the purchase is made and I'm waiting by the front door for the delivery guy :-)

Five months in the making. There is always a great sigh of relief when a release is shipped. Things are getting trickier now since we have to be fully backward compatible with earlier releases, there's alwasy this nagging doubt in the back of my mind that we've broken something important to someone. With all the possible permutations and combinations of test scenarios testing every possibility is near on impossible. This is mostly why we run early release copies on puakma.net. there are a number of applications up there that do all sorts of wierd and wonderful things, and if something is broken we'll find it pretty quickly.

The 3.22 release was an important one. We shipped computed page functionality and parent page functionality which required some heavy rewriting of the page rendering internals. With every rewrite comes the risk of something not working exactly like it did before.

3.22 is also the base release for Vortex IDE which is now in beta. Everyone testing the IDE should have 3.22 or greater. I am using the IDE currently for a customer project and having great results (backing up regularly just in case!). I think it compares favourably to the likes of Lotus Domino Designer (except runs on almost everything and you can develop through firewalls). Our plans for future releases of Vortex have some REALLY cool features to further cut development time. 

Today we solved an annoying problem in a pretty simle way. A customer's app has a page which does some hideously complicated calculations to determine if a planned coal mining operation would actually make any money. Today they want the ability to group multiples of these plans together and provide (for example) a set of plans for a week. Trouble is now we have to repeat those hideous calculations inside another part of the application so we can sum the plans for a week and determine if we'll make any money that week.I absolutely hate duplicating code, and I also hate rewriting perfectly good code to do something slightly different.

So we used the cruel and cunning plan of calling the original action programmatically for each plan in the group, summing the results as we go and outputting the totals on to the page. This enabled us to reuse 100% of the exiting code and write a small wrapper to get each plan's totals. Here's a snippet:

while(looping_through_plans)
{
int iLongPlanID = RS.getInt("LongPlanID"); //unique id of the plan
HTMLDocument docTemp = new HTMLDocument(pSystem, pSession);
docTemp.rPath = new RequestPath(getDBURL() + "/LRPlan?OpenPage&ID=" + iLongPlanID);
docTemp.setParameters("&ID=" + iLongPlanID);
//now run the action for the other page on our temporary document
pSession.runActionOnDocument(docTemp, "OpenEditLRPlan");
if(docTemp.hasItem("grpPlantCost"))
{
//get the values off the temp document and sum them
m_dPlantCost += docTemp.getItemNumericValue("grpPlantCost");
m_dLabourCost += docTemp.getItemNumericValue("grpLabourCost");
}
}//while