Friday, 27 November 2009

The 'IDS' "Mythical Man Month"

The idea of the man month has been analysed in coding theory for at least a generation. Frederick Brooks addressed this in the seminal work, "the Mythical Man-Month" (1975). Using these forms of analysis, we can demonstrate that incident response, intrusion analysis and other complex security tasks are "tasks with complex interrelationships".

Many in the industry would be stating that this is nothing new to them, what is new here is that I am attempting to quantify this relationship. In this post I have included a small amount of data from a paper I will be publishing next year. I have taken data from a 5 year period on incidents for 165 companies and other organisations with an Internet presence and some level of security reposnse capability.

This has been an analysis of 423,000 incidents. Costs are based on reported financial figures; this is the actual financial and accounting records for these firms. The data that is being analysed was collected from the 2003 to 2008 Australian Financial year.
The boxplot above lists the individual incident response data for the times (in minutes) against the number of personal involved in the process (including management etc). We see that the data has a positive or right-skew. This is clearly displayed in the histogram of incident response times from a 6 person teams (displayed below).
For the purpose of this post, I have simplified the results. The same data is displayed in the following graph. The difference is that is has been summarised with only the mean (average) values being reported.
In this, we see Brooks' (1975, Pp 18-19) supposition that in tasks with complex interrelations, "the added effort of communicating may fully counteract the division of the original task". This is shown in the inflection point. When around 6-7 people start to become involved in the incident response process, the amount of time required per incident actually increases sharply with additional team members.

This holds in both response and detection time. Additional people help an incident response team to a point. Adding the system administrator, a coordinator and other such parties does reduce the time per incident, but only to an inflection point, where the effort to coordinate team members starts to negatively impact the gains.

We see from the co-plot above that additional team members (over the ideal) have a greater negative effect on the incident response time than the detection time. These differences can be used to create the teams that are customised towards the needs of an individual organisation.

The Economics
The real issue here is the economic impact. This is more difficult to quantify overall. For this I shall be conducting a multivariate analysis of the data with separate classifications for industry, size, etc.

The economic results are what really matters. These results are tied to the organisation.

In the plot above, I have compared the mean costs of an incident (in consulting fees, displaced revenue, etc) for a number of types of organisations. Here we clearly see the Online Casino operation has the greatest impact of under-staffing its incident team.
Conversely, a construction firm with little existing online presence may find little benefit in doing anything. In this particular instance, with little tortuous impact, no PCI or other restrictions and an open form style design for blueprints - there was little to make them want to care about security. In this instance, the economics placed doing anything as a cost. This is the exception and only 2 of the 165 organisations displayed this pattern.

What we can take from this is that it is best to determine the costs and impacts of incidents for your organisation and construct a team suited to the needs and requirements that you face, before the impact hits.

[1] Brooks, Frederick P. (1975) "The Mythical Man-Month" Addison-Wesley, USA

No comments: