Puakma: Under the hood

There's a show I've been watching, but due to pre-natal classes I've only ever seen the first half hour and miss the rest. Call me old, but I'm lovin' Grand Designs Abroad. But I never get to see how the houses turn out!

Amazon to the rescue. I log on to Amazon UK (bit torrent is naughty, don't do that) and locate series 3 and 4 DVDs and order them. Amazon's super service has them to me in super quick time. Perfect. Next I poke the DVD into my MacBook and......

Bah! The disk is coded for region 2 and I am in region 4. This sucks and blows all at the same time. Here I have done the right thing and purchased a legit copy and now can't play it - well I can, it's just that I can only change the region on my DVD drive 4 times before it's locked to the last region.

Fortunately I have a PC with a LiteOn drive wich seemed to not care so much about this region business and read the disk ok with a pesky region change. All files are now copied to disk and I'm able to watch it with VLC player. Of course you could also use this: http://www.mrbass.org/dvdrip/ ;-)

Question is what would a mere mortal do that does not have access to extensive computer hardware and software, let alone even understand this confusing region coding?

Region coding is only good for the movie studios so they can charge $35 for a DVD in Australia and 3 rupees in India. DRM is bad for the customer. 

For last last 6 or so years I've been running a company that is split between products (software) and services (contract software development). As the services side demands more of my time, I am increasingly questioning the "value" of a services business. In this context I am referring to a services business as a business which requires a human to perform some service at an hourly rate for another company.

Growing a services business is difficult. Since a human is providing the service, to provide more service you need either a) more humans, b) more revenue per service or c) more time. Scaling these variables is challenging part.

1. We know that time is fixed and that 100% capacity is 168 hours per week, in reality much, much less, around 40 hours.
2. We also know that companies have a natural cieling for what they are willing to pay for a service, let's say $300/hr.
3. We know that the more complex the service provided, the more difficult it is to get additional qualified staff to perform said service. 

So, according to the variables above, the only real way to expand a services business is to add more staff. In reality, as the number of customers grow other activities take up an increasing amount of time in communications, account management, staffing issues, etc. Thus the original 40 hours begins to fall as more and more hours are taken up with non-billable time.

As a small business it is almost impossible to compete for the high end consulting work as customers deem you as high risk. A perceived "safer" choice is to engage large consulting companies with supposed "experience". The reality is the large consulting company will subcontract out the work to smaller suppliers such as myself while skimming a (often large) margin. 

Another factor particularly important in IT services is the fact that software development projects are not discrete. That is, it is unlikely for a software development project to actually end. We regularly have projects that come back like the ghost of Christmas past as the customer wants to add a few more tweaks and updates. Conversely, if you are an an excavator operator you dig a hole and leave, the customer will never get you back to continue work on the same hole to perhaps turn it into an airplane ;-) These continual mental context switches, jumping back to projects that were developed years ago, is a real mind bender and absolutely kills concentration since a small change involves relearning what was done in the past and what the implications of adding the new change are.

The provision of complex services require the use of competent and highly skilled staff. The more complex the service, the higher the staff knowledge needs to be. A couple of examples: 1) You run a window washing business. 3 of your staff members leave to go backpacking. It is very easy to replace them since after around 30 minutes of training you are a competent window cleaner. 2) You run an obstetrics business (you deliver babies). 3 of your staff leave to become GPs. Replacing them is a difficult task since they need a medical degree plus more years specialising. Software development is somewhere in between.

So what's the bottom line? If you plan to run a small services business you need to really think about the economics and compare that to working in some other services business. Effectively you are buying a job and a lot of headaches (tax, accounting, marketing, planning, staffing, ...). 

Product based companies scale much better. If you make widgets, selling one or ten requires little extra effort. You can also engage less skilled staff in the sales and billing parts of the company to streamline the sales and billing process. In addition the negagement of resellers further alleviates the sales burden. Rather than dealing with 100 customers directly you deal with say 10 resellers. Sure, you take a small hit on the margin but the resellers really absorb a large portion of the sales effort.

Now I'm sure the skeptics out there may not believe the above hypothesis, so here's some research for you: Take a look at the following top 50 companies for 2007 and work out the proportion of products vs services companies. The results should be obvious. http://www.diversityinc.com/public/1595.cfm

in the last few weeks I've been taking a rather critical look at what I do. One thing became abundantly clear I have somehow managed to push aside most of the stuff that I love to do to make way for the stuff I have to do. End result = unhappy camper. Last Monday I started making some changes.

Some of you may know I play the guitar. When I was younger I played it LOADS. It's something I really enjoy but somehow in the last 10+ years I never was able to make time to do it. On Monday I visited the local guitar store and bought a shiny new guitar stand and some new strings (Mmmmm DR 9 to 42's rock!). Now I have my guitar set up right next to the desk. When I need to think something over I'll pick it up and play it for a couple of minutes while I mull over the problem. Two birds, one stone. Perfect.

I had completely forgotten how great that guitar is. Sadly in the same music store they are selling the Ibanez RG550 "20th Anniversary" model. Bummer I have the original and I now feel oooooold! 

So Tornado is getting pretty mature, but all the work to date has pretty much revolved around a single company server where everyone is in the same timezone and speaks the same language. Until now :-(

As we peg away at ProjectReactor, things crop up. Important things. Let's say we have a project with 2 project managers, one in Sydney Australia and one in New York USA. The Sydney PM enters a milestone to be due mid-day friday, when the New York PM looks at the same record he also sees due mid-day friday. Disaster! Different time zones.

Then there's the date input formats. Sydney PM uses "dd/MM/yy" and New York PM uses "MM/dd/yy". Ouch. How do we make one application design easily cater for the different formats (and also "yy/mm/dd" and many other variations)?

When you have the sourcecode, anything is possible :-) 

Step one: Extend the session object. Add two new attributes to the session; time zone and locale. Upon login we get the timezone from the browser and the locale (language) from their account record. The server code then uses these new properties (if they have been set) when dealing with dates. If they are never set by the application then we default to the server's locale.

Step two: Custom date input formats. This was the tough one. The Tornado app uses <P@Date @P> fields with a format attribute, eg <P@Date name="x" format="dd/MM/yy" @P> The format attribute is stored in a page design element and is certainly not easy to change on a per user basis! Custom session objects to the rescue. Tornado allows any kind of object to be stored on a user's session. So now we just store a session object called "dd/MM/yy" on the session and give it value we want for that user eg "yy.MM.dd". Now when the server is parsing and rendering dates it does a check on the session to see if there is a session object of the same name as the format attribute's value, if there is use this value instead of the format attribute value. If there is no session object by that name, then just use the value this allows it to work either way.

My head hurts. I'm off to bed. 

It seems no matter what I've got going on my mind is always keeping one eye on Tornado server performance. A while ago I implemented a method of 'minifying' Javascript code on the server side. That is, you develop the js code with comments, linebreaks and all the formatting to make it easy to ready but when a web app asks for the resource it is automatically squished to reduce the file size. As our web apps begin to include more and more client side functionality, the size of javascript libraries is steadily growing.

Here's a real world example: prototype.js

Prototype is a very common library now used extensively in a lot of web applications. Version 1.6.0.2 is 126,127 bytes in it's raw format and looks like:

When we turn on the automatic minification the file is reduced to 93,689 a saving of 32,438 bytes in network I/O. This is what it looks like after minification:

Of course Tornado also has gzip compression built in too, so the overall file size is further reduced to 23,527 bytes, a reduction of 102,600 bytes or 81.3%.

So what does this mean in terms of network I/O?

Based on a MTU size of 1492, the raw prototype.js file will be broken into 85 packets. The minified and gzipped version will result in just 16 packets. Also remember with TCP/IP we have guaranteed delivery so each of those packets has a matching ACK (acknowledge) packet sent back to the sender, so packetwise it's more like 170 vs 32. Note these figures are approximate, based on 1492 MTU and an ethernet network, disregarding socket creation and breakdown overhead and any packet loss. 

More information:

http://www.inconspicuous.org/projects/jsmin/jsmin.java
http://support.microsoft.com/kb/140375

It's been a loooong time coming but we're gearing up for a release of ProjectReactor.net. We have been using this web application to manage our consulting business for the last 3+ years. It enables us to interact with our clients and manage the time spent etc on things our customers ask for. For a long time it's been rather "bare bones", that is, it had just enough in there to enable us to get our primary job done. Over the last couple of months we've been re-architecting it a little to be more flexible and dynamic.

In essence it will be a basecamp competitor. Basecamp is a great tool but I believe misses the mark a little in managing the lifecycle of a project. A bascamp project is essentially a flat to do list. We have approached the whole project management proposition from a different angle. Rather than focusing inward on the team and tasks, we focus on the customer and what they want (features). By managing and delivering a customer's "wants" the project is delivered. The ability to break a project into phases allows pre-planning of the next version or point release, so over time you get a fantastic picture of the project; what features were delivered in what phase, what time was billed to various features, why things took longer than expected, etc. 

Initially access will be by invitation only. If you are looking for a better way to manage your projects, drop me a line :-) 

In the last couple of weeks I have been setting up a voice over IP system for the office. Something I have been meaning to do for a while but now the neccessity has arisen to make it so.

After a little research it seems Asterisk is the best software available. After failing miserably at a bare Asterisk install (due to the plethora of extra sh17 to install on top of it). I tried AsteriskNOW and prebuilt ISO with the linux OS, asterisk and apparently all the gear to make it fly. I burnt a pile of hours trying to make that flat with limited success so abandoned that too. Last and successful attempt was using trixbox. AWESOME! Really great package, it is the whole box and dice (as per AsteriskNOW).

I have installed it in a virtual machine inside the office, opened the relevant ports in the firewall. The asterisk virtual pbx connects to my VoIP service provider Exetel.

Call quality is average, a little worse than the regular telephone system but quite acceptable. Where it wins is in flexibility. A single incoming number can be routed to ring groups, IVR, a specific user etc. Video calls are possible, voicemail built in, web interface to configure.

Next job is to hook up the regular phone to divert through the PBX.  

I'll admit that part of the reason for not writing much last year was a general sense of being somewhat jaded with the IT industry. The industry I find is a bit wierd and not really "welcoming". Some examples:

a) a noob (no, not me) on a linux site asks a simple question.  He/she gets 1000 flame responses treating them like the village idiot. Get with the program guys: LINUX IS NOT INTUITIVE! A little help and patience may make everyone happier and produce a better software product for all.

b) another noob posts a question on a programming site. "I need to do X send me the code". Er, no. We're all happy to help with a specific problem but if you want to be a programmer you have to pay your dues - that means a CS course (or whatever training/apprenticeship). A bad copy/paste makes all programmers look unprofessional.

c) Spam. There is soooo much spam and sh17 floating around now to make email almost unusable. On top of that we waste sooooo much time securing server and application code to stop people doing destructive things to applications. Just stop it - one day it will become too much and we'll just turn off.

d) For chrisake CREATE SOMETHING. Most of the blogs etc out there just comment on stuff and throw rocks. eg A post on why language X is better than language Y. Who really gives a f#^&@?? Are you really contributing to the greater good or serving to create a greater divide between X and Y users? Instead, how about you build something useful in language X and put it out there? In this day and age it really doesn't matter what language you write in, they all get the job done. 

...there are a few others perhaps too sensitive to post here :-)

The end result is I often think about the simplicity of digging trenches for a living. If I disappear from the young person's interweb one day you'll know what happened.

Now I feel better for having vented :-)