Friday, 6 June 2008

"But what if they (economics) were ignored?"

"But what if they (economics) were ignored?"

Economics can not be ignored. We live in a world with limits. To speculate on a world without economic constraints is on one where there is no shortage of anything. All people have anything they want any time.

This is fantasy. You may as well ask "what if dragons where real?" Who cares? I have too many real world things to consider to bother with fantasy.

Finance is simply micro economics. We are bound by limits. The Universe has limits on the speed we can travel. We have only so much energy per person on the earth. We have only so many materials. When we get into space and start mining Jupitor, this will increase, but supply and demand will bring costs back in line. Everything has a limit.

Time is also a limiting factor, we all have set limits to life. Mind you, the genetic possibilities may be larger, but this is sci-fi as yet. Time is money is stated for a reason.

A factor of finance is time. This is where the concept of the time value of money comes into play. When assessing risk and possible costs, at least an NPV and IRR calculation needs to be factored.

Pen testing is limited economically. Companies can either go for more low cost testing that rarely finds anything at one extreme to an infrequent test by highly skilled individuals at the other. This can range thus from $100 per hour people, to $370-600 for the top people. These find more, but less frequently and the number of people who can do this are limited.

Limits pose constraints.

Risk should be a quantitative function - in many organisations it is required to be (such as BASELII) quantitatively defined even for IT. This means calculus. Either risk is optimised at a inflection point that is a maxima/minima or the function is compounded by saddle points. Quantitative does not equal assigning numbers - this is a perception exercise. Risk needs to be scientifically calculated within defined confidence levels.

If we take a pen test team with 10 members all working on a large site with 500 hosts, we give them 5 days per host (large budget here). The test time is the entire working year. Either a sample is taken or the test takes a year.

This means systems are tested at best yearly.

A full test generally takes more than 5 days for a system so I am being conservative. I do not want to look at "but I broke x in 10 minutes" etc.

Security and risk like finance have a time factor. How long it takes for a consultant to come in and test and how frequently they do this is important.

Testing is a detective control and a validation. Validation is only effective in 2 ways:
if the control has a failure - to find and rectify
to ensure that a control is in place.

Testing is not generally done scientifically. We seek to prove a negative. In Pen testing this can be simple as the standards are frequently low enough already. Testing a broken model (a system with poor controls) will lead to discoveries. The problem is that it can not state that a system is secure.

A pen test can at best find and exploit a flaw. A worst it can make speculations. In any event, it does not test the full compliment of control failures.

The alternatives need to be in place. An infrequent detective control with no preventative controls is less than useless. A pen test is only effective in pointing out holes and control flaws. Even with zero days, there is defence in depth to be considered.

A good control framework will detect and stop even most zero days far more than it stops many other threats. Good logging and monitoring at the host and network level with competent people is a more effective control than pen testing.

I will discover a breach with a combination of integrity checks that run live and database triggers faster than I will using pen testing.

Monitoring and baselining network traffic is more cost effective if done correctly than testing.

Yes testing and audit have their place. Their place is to ensure that other controls exist.

No comments: