I actually a while back wrote a discussion on software estimation that someday I'll post.  However, this evening I ran across another item that I felt it worth pointing out.

In general I consider software estimation to still be more art then engineering.  I don't mean this as a complement, I mean that its predictability, which in engineering we strive to keep high, is in reality low (experience people operate closer to medium predictability...) Part of the problem of course is the ability to learn over time and effectively apply the lessons.  The fact is an experience software lead has learned over time how to estimate and may come close but the learning is of a more intuitive nature.  That's why someone will ask 'why did you estimate that X would take 3 days?' and the engineer's answer is the equivalent of 'because.'

So why am I bringing this up, well I'm going to reference one of those people who's blogs I follow in terms of the business of software.  As I've said in the past - if you are working in this industry not only should you be keeping up with technology, but you need to take some time to learn about the business side - and estimation definitely falls into that category.  Let's face it, from an engineering standpoint I wouldn't care if I finished in a week or a month, but from a business standpoint - well that is a huge difference.

With the release of their latest version of FogBugz, Fog Creek has introduced a new feature for software estimation.  In short the idea is that over time as you define and estimate tasks the software tracks the accuracy of an individuals estimates over the course of a project.  It then determines an accuracy factor.  Over the course of several projects it refines this factor.  It then applies this factor to an individuals estimates - for more information check at Joel on Software's blog at: http://www.joelonsoftware.com/items/2007/10/26.html 

On the surface this is a great feature and I like it but a couple notes, not to bash but as a warning since even my first impulse is to say "I'll take 2".  First the estimator (person creating the estimate) can't know what the tool's adjustment factor for their estimates is because they'll probably 'game' it.  That is to say the estimator will consider their estimate and if they know that the tool will double it, well they'll reduce it by some percentage, because of their inate desire (and it may be totally unconscious) to hit some number (generally as low as possible).  After all the estimator says (ex: "I know I was over by 50% last time over the project so I was going to double my initial gut estimates but the tool is going to do that so I'll need to reduce my estimates so that when the tool doubles my estimates they won't be too high.")  The net is this can result in a reduction of the original estimate to account for the tool's automated increase.

The second thing to keep in mind is that the tool doesn't account for the fact that each project is truly different.  If the developer were consistently estimating the same type of task then the 1 to 1 corrolation that the tool applies makes sense.  But if project 1 is say a desktop application and the developer's UI estimates are 50% under and the next project or the third or fourth consectutive project is suddenly an web UI or a pure business interface and as such the developer's estimate is 90% correct - well suddenly the game has changed and in reality the previous estimates shouldn't be weighted as heavily - but how heavily is the question, that 50% and 90% could also be reversed.  The tool isn't magic and it can't account for all of the variables. 

Finally by its nature this tool will always penalize someone who's estimation skill is improving.  The first set of estimates might be all over the map, the next set might be consistently 25% of actual and then if the estimator sees this and ups their estimates by 150% the correction factor of the tool will suddenly make the estimate way over the top.  Something to keep in mind whether your tool is a fancy algorithm or a set of multipliers in an Excel Spreadsheet - the tool is only a tool - you will still need to carry out a reality and business cost review of the estimate.