Saturday, 28 June 2008

More Security Metrics

At the end of the day it is not the metrics, or the measures themselves that make the difference, it is their ability to affect positive change.”
Agreed. To do this, there needs to be some basis in fact.

if it doesn’t meet at least some of the above criteria than it is simply an exercise in academia
Agreed as well. Most of the issue I see are more a lack of trust in what is there. What we can get to with the models I am proposing in later stages to deploy is a quantitative economic risk model. This is one that will calculate expected and forecast risk in dollar terms. This is using financial language where needed, IT speak where needed etc.

I have the unusual ability to understand most professions as I collect professions and degrees as a type of hobby. Quantitative heteroscadestic risk models are going to feature more and more. Strangely enough, they already do for a large part of what we are ignoring.

BASELII requires a quantitative measurement and the calculation of IP value is generally based on these methods. Treasury functions run by a firms CFO are commonly based in these terms. Even marketing people are starting to deploy quantitative analysis.

IT people do not need to understand all the ins and outs of the math, just that it is verifiable.

The model should eventually get to a dollar amount where that is a valid measure and an accepted ordinal measure where necessary.

Hence the call for something simple and the same to start such as the CIS tool output. Coupled with a number of nominal and categorical fields, this data could be modelled into a decent hazard/survival model which could then be converted into a quantitative financial calculation for expected loss. The other is an expected time value on systems. This would be a survival metric of the expected distribution of compromise and recovery.

I do this for modelling PCI and POS risk now (in the real world) with good results (95% confidence or greater). So I fail to see why it can not be extended to more than PCI systems.

The output I generally report to boards is a cost distribution. That is:
· Max Cost and likelihood
· Expected cost
· 95% CI predictive cost range

This can then be compared with alternatives. Add a firewall and the risk =x.

A) identify your assets
B) identify the asset value for the enterprise
C) identify the risks to the asset
D) identify the impact and probability of these risk
E) calculate the risk per asset

B is economic value. This is a finance and not accounting calculation.
D is based on opportunity cost and the NPV and IRR of the asset/associated project.

There is a time basis to money. NPV (net present value) is a good means of determining value without being too complex. IRR is the Internal rate of return. (I also did finance)

Impact in D is the loss associated with the asset being removed or tarnished.

In many large firms, the finance and treasury groups will generally determine project metrics. Most firms buy risk beta's rather than calculating their own.

Security Metrics

What I would see as a simple starting point for metrics.

What is needed is a measure that is not going to be subject to the views of the individual making it. That is, an expert or a trained monkey should be able to get the same results.

What I would propose.
I would leverage the existing work of the CIS for a start. Each of the metrics that are produced using the CIS test tools are a start.

For instance, you can run router tests, Windows system tests on servers and workstations, Unix/Linux servers etc. At least this way we have a starting point.

In the event that there are many systems, we could add a simple random sampling routine. We have 500 servers, 15 are selected (by software) randomly to be tested. The CIS tool is run on these and the results added for the week, month… whatever.
Next we can add other statistics. Network traffic, router logs etc. Again the idea being to keep them as objective as possible in measure. This is, remove subjective bias. There are many ways of doing this in IT metrics. This is but the start, there are 100’s of metrics such as this we can add and start to use to get some really effective results.

This is far from ideal, but it is a starting point where we use existing CIS tools and create metrics that we can improve over time.

As metrics are improved and we start getting data, we can make a level 2 tool that has forecasting and other enhanced capabilities. This is:
1. Stage 1 – Simple Frequentist approach with results from CIS tools
2. Stage 2 – Bayesian predictive model
3. Stage 3 - …

For stage 1, we can do this now.

I will even write the tool to do something such as this.

From the simple boxplot (and there are many ways to visualise this, this is just a quick and nasty) we can see an improvement overall, but also a trend in measures 4-6 where the security metric of some systems has decreased. This not only gives an indication of improvement, but aides in discovering where problems may lie.

The good stuff will not be there In phase 1, but at least it is a measure that is going to be the same each time any person runs it.

Friday, 27 June 2008

Who's to Blame - You or Your ISP? [IEEE Question]

The position of the British Government[1] with its recent moves to call Intermediaries to action in the formation of a voluntary body to stop Intellectual Property violations is a start to the reforms that are needed. The problem is well defined in this call for reform, however, the call for voluntary changes are unlikely to bring about the required changes. Intermediaries have the capability to stop many of the transgressions on the Internet now, but the previous lack of a clear direction and potential liability associated with action rather than inaction[2] remains insufficient to modify their behaviour. Even in the face of tortuous liability, the economic impact of inaction is unlikely to lead to change without a clear framework and the parallel legislation that will provide a defence for intermediaries who act to protect their clients and society.

1 The French President announced a plan on the 23rd Nov 2007 to curb Intellectual property theft and other Internet related crimes. He stated that: "Today an accord is signed and I see a decisive moment for the civilised internet. Everywhere, in the US, UK and others, industry and government have tried... to find a permanent resolution to the problem of piracy. We are the first, in France to try to build a national grand alliance around clear and viable proposals." Geoff Taylor of the British record institute stated that: "We will continue to pursue voluntary arrangements, but unless these are achieved very soon we believe that the UK Government must act, as the French government has, to ensure that the urgent problem of internet piracy is tackled effectively."
2 In the US, the most obvious example of this action can be found in the history of the Communications Decency Act. Congress directly responded to the ISP liability found in Stratton Oakmont, Inc. v. Prodigy Services, 23 Media L. Rep. (BNA) 1794 (N.Y. Sup. Ct. 1995), 1995 WL 323710, by including immunity for ISPs in the CDA, 47 U.S.C. § 230(c)(1) (2004) (exempting ISPs for liability as the “publisher or speaker of any information provided by another information content provider”), which was pending at the time of the case. Similarly, Title II of the Digital Millennium Copyright Act, codified at 17 U.S.C. § 512, settled tension over ISP liability for copyright infringement committed by their subscribers that had been created by the opposite approaches to the issue by courts. Compare Playboy Enters., Inc. v. Frena, 839 F. Supp. 1552, 1556 (M.D. Fla. 1993) (finding liability), with Religious Tech. Ctr. v. Netcom, Inc., 907 F. Supp. 1361, 1372 (N.D. Cal. 1995) (refusing to find liability).; The fear of being seen as a publisher rather than mere conduit has resulted in many ISPs and ICPs to a state of inaction.

Competition Law in the UK

While it is true that undertakings with monopoly power can act anti-competitively and harm markets and consumer interests, and that NCAs should actively recognize and prosecute such behaviour, an undertaking’s size alone does not reveal damage to competition or an infringement of the competition laws. The appropriate aim of competition law enforcement needs to be focused against anti-competitive behaviour and consequence, not just size or market share.

Council Regulation (EC) No 1/2003 was adopted in the Council of Ministers on the 16th of December 2002 and came into effect from the 1st May 2004. Additionally, the Commission also published various guidelines and notices designed to aid in the interpretation and application of Regulation 1/2003. These guidelines also formulate the formal rules the dealing with procedural matters. Replacing Regulation 17/62[1], the Modernisation Package (as Regulation 1/2003 its associated guidelines are also known), established the procedures to implement the European Community [EC] competition law contained in Articles 81 and 82 of the EC treaty.

The growing delays associated with the formerly centralised enforcement of EC competition law and the ensuing growth of the community with the addition of 10 additional member states paved the way for the introduction of the Modernisation Package. The goals of this package included empowering the competition authorities[2] to enforce Articles 81 and 82 together with the European Commission. Further, Article 81(3) is no longer founded on a notification to the European Commission.

Modernisation has been described by the Commission “an ambitious and fundamental overhaul of the antitrust rules implementing Articles 81 and 82 of the Treaty”[3]. Four years on, have the goals been met? Regulation 1/2003 has sought to increase the efficiency of enforcing EC competition law through simplified administration and by creating a greater level of certainty through a uniform application of EC competition law within the European Community. It was hoped that this would lead to lower administrative costs and a greater level of competition.

In order to determine whether Regulation 1/2003 (EC) has achieved the lofty objectives associated with implementing a completely new competition law enforcement framework, it is necessary to look both to developments which led to these changes and to evaluate their effectiveness.

What came before and where are we going?
In order to effectively evaluate the objectives achieved by the implementation of Regulation 1/ 2003/EC, it is necessary to first look to both the origins of the act and the associated preceding legislation. Regulation 1 was enacted in parallel with the enlargement of the community. Further, moves were already taking place away from the formalistic and prescriptive to a process based in economic reality.

The introduction in 1999 of updated block exemptions and new guidelines for vertical agreements was in itself a radical shift from the previous approach to competition law enforcement.

Competition Act 1998
The enactment of the 1998 competition act introduced both a “carrot and stick”[4] into the enforcement of competition law within the UK. This act set to prohibit business practices, behaviour, and any associated agreements, which could result in diminishing open competition within the market.
This act jointly prohibits anti-competitive agreements, which have “an appreciable effect on competition” (in Chapter I of the Act), and further prohibits the abuse of a dominant market position through either discriminatory or predatory pricing structures. This in effect mirrors the provisions of articles 81 and 82 of the European Council competition legislation.

The “stick” consists of fines designed to make anti-competitive behaviour uneconomic. This has been implemented by means of the burden of fines, which can amount to 10% of the organisations UK turnover for a period of three years.

Alternatively, the carrot is defined by the leniency policy. Provisions have been made for organisations and whistleblowers who cooperate openly with the competition authorities. In some instances, fines may be reduced by as much as 100%[5].

Enterprise Act 2002
Shortly later, it was decided that a “bigger stick” was needed.
In 2003, the enterprise act introduced further measures to grapple with the perceived anti-competitive behaviour in the market. This act introduced an offence with a sentence of up to five years imprisonment and/or an unlimited fine directed at individuals within an organisation criminalising cartel activities. Additionally, the act introduced a process to disqualify company directors who have been found in breach of either UK or EU competition law.
These measures were further coupled with an increased range of investigative powers[6] allowing the Competition Commission to compel people to provide evidence[7] and to enter the private premises of company directors or officers in order to search for supporting evidence. These additional powers are in addition to the ones granted previously under the Competition Act.

One goal of this act was to limit behaviour, which, although not illegal, could influence the market by preventing, restricting or distorting competition. The act also fashioned the apparatus for appeals to a specialist competition court for organisations impacted by merger or market investigations.

The bigger Picture
European Commission articles 81 and 82 which correspondingly prohibit the application of anti-competitive agreements and any abuse other firms dominance within a market continue to shape the form of UK competition law. The enlargement of the member states from 15 to 25 and the reform of the EC merger regulations[8] resulted in a need to (again) change competition law and policy within the UK.

The Modernisation Regulation[9] decentralised the application of European Competition Law. The EC has produced an expectation where competition authorities and national courts are to become more involved in the enforcement of articles 81 and 82. The removal of the ability to notify agreements to the European Commission in order to seek exemption has also been removed. As such, individual organisations have to assume more responsibility have in not breaching EU competition law.

The goal of these changes is to allow the European Commission to concentrate on the most serious breaches. Serious abuses, which could lead to the subversion of market power such as pan-European cartels, are their primary target.

This restructuring of the EC Merger Regulation includes a change to the substantive test. This is now a test of whether a merger “significantly impedes effective competition”. However, as will be explored later, this test is often not clear.

Why a need for change?
Changes to competition law brought about by the modernisation legislation have brought about much disorder and uncertainty. It has been stated that these reforms were necessary due to both the imminent enlargement of the EU and existing administrative overload. This overload was said to be a result of the system of notifications. The European Commission’s Press Release of November 2002 declared that the system of notifications could not remain workable following the expansion of the EU.

However, it should be noted that the European Commission had initially referred to the obstacles to effective protection of competition presented by the Commission's monopoly over article 85(3)[10] and the Commission’s lack of ability to put into effect competition rules throughout the enlarged community in their original justification to the Council.

It was argued that the notification system effectively consumed resources that should be assigned to anti-cartel enforcement. Further, the notification system was stated to increase compliance costs and create uncertainty within the economy. This was stated to have a direct adverse economic impact on the EC.
It would seem that the years of the national competition authority would hold that modernisation was necessary to fill the need to enforce EC competition law. The European commission rarely referred to national competition law and when it did so these references were mostly restricted to noting the absence of harmonisation.
It was noted by Lord Fletcher[11] “NCAs were seen as partners, or future partners, in the shared task of applying the E.C. rules rather than as enforcers of national competition law.” The drive for change was to foster a “common competition culture throughout the Community”. This, it was said, would provide significant benefits in the form of greater resources. It was additionally the case that competition laws were either not prominent or nonexistent in many EC countries that where to be in the expanded Union. Empowerment through the EC rules was a powerful instrument. The intent was to use this to align these countries with those (such as the UK) with a strong tradition in competition law.

The effect conversely on countries such as the UK and Germany with a strong history of Competition Law enforcement was more challenging.

Articles 81 and 82
Articles 81 and 82 of the EC Treaty fundamentally define the competition rules for the European Community. These rules cover both coordinated and cooperative behaviour[12] and unilateral behaviour by undertakings[13]. Both Articles 81 and 82 may be imposed either by public or private enforcement and both are a piece of an enforcement system that seeks to discourage anti-competitive behaviours prohibited by competition law and to protect undertakings and consumers from adverse practices and any costs caused by them.

Article 81(1) prohibits agreements[14], which, either through purpose or consequence, confine, prevent, or distort competition or trade between the member states. The exemptions provided in Article 81(3) allows for agreements that are deemed to provide objective economic benefits or otherwise benefit consumers. The four requisite conditions that need to be met by an agreement such as it is excluded from the provisions of article 81(1) include provisions for:

  • efficiency gains (in that the agreement contributes to promoting technical or economic progress, an improvement in production or more efficient distribution of goods);
  • fair share for consumers (in that the consumer receives a “fair[15]” share of the consequential benefits derived from the agreement);
  • indispensability of the restrictions (that the anti-competitive effects of the agreement are a necessary part of providing competitive benefits to the market);
  • no elimination of competition (that competition is not substantially impacted across the products or services in question).
Block exemption regulations were introduced to increase certainty associated with the application of these exemptions. These regulations automatically exempted agreements from the prohibition in article 81(1) if they satisfy the conditions set out in the regulations. An agreement, which did not satisfy the conditions the block exemption, would need to be separately assessed in order to resolve whether they were exempt pursuant to article 81(3). If an agreement is prohibited under article 81(1) and did not meet the requirements to be covered by a block exemption, it is deemed both as void[16] and to be prohibited. Unlike Article 81, Article 82 has no specific exemption. It may however be possible to justify the allegedly abusive behaviour in some objective manner[17].

However, whereas the Community law “demands an effective system for damages claims for infringements of antitrust rules, this area of the law in the 25 Member States presents a picture of “total underdevelopment”[18]”.

The introduction of the Modernisation Package
Regulation 1/2003 has resulted in a more decentralised system. Articles 81 and 82 apply involuntarily without allowing for the individual notifications intended to obtain an exemption. This more decentralised system will naturally mean that the Competition Commission [CC] and UK courts will have a larger role to play in the enforcement of competition law.

As any agreement or conduct which violates either Articles 81(1)[19] or 82 is now automatically restricted, the administrative load would be expected to decrease allowing for NCAs and the commission to concentrate on more severe forms of anti-competitive behaviour and illegal cartels. Likewise, any agreement, which fulfils the requirements to be exempted under article 81(3), is deemed valid without any interaction of the NCAs or commission.

The requirement for the CC to apply articles 81 and 82 to all behaviour and agreements that “may affect trade between member states”[20], coupled with the requirement to give the commission at least 30 Days prior notice before opting to end an infringement, allow commitments, or eliminate the benefit of a block exemption; would be expected to draw the member states closer together and increase uniformity. Further, the commission is able to decide on matters relating to Articles 81 and 82 with the result of producing decisions that are binding on both the CC and the national courts.

Close cooperation has developed between the various NCAs and the Commission[21]. This cooperation has extended to the national courts as is required in Regulation 1/2003, art. 11(1). As a result, the CC must inform the commission of all active investigations involving EC competition law. Regulation 1/2003 allows that two or more NCAs may establish their own enforcement priorities[22].

Regulation 1/2003 further expands the scope of information exchanges between NCAs over that provided for in regulation 17/62. For instance, the modernisation package provides for exchanges of information that the national courts. Regulation 17/62 had no such provision.

The commission has maintained its investigation privilege in regards to Articles 81 and 82 following the implementation of the modernisation package. Regulation 1/2003 sets out the investigative powers of the Commission, but has left the prescription of the powers of the NCA and national courts to national law.
The process of assessing market dominance is problematic. To be in a position of dominance and undertaking must have the ability to act independently of its competitors and consumers. This complex assessment of data across various market definitions of market share, the condition of competition, entry and exit barriers, buyer power, and relative business strength is beyond the capabilities of many undertakings. 40% of and undertakings market share could be held to be a rough presumption of dominance.

There is little given in the way of guidance to aid in the complex economic calculations needed to produce these figures. These additional costs have to be borne by the undertakings themselves.

The Commission will grant immunity[23] to an undertaking where one of the following two conditions are met[24]:
(a) ‘the undertaking is the first to submit evidence which in the Commission’s view may enable it to adopt a decision to carry out an [inspection in the sense of Article 20(4) of Regulation 1/2003]’ and ‘the Commission did not have, at the time of submission, sufficient evidence to adopt [such] a decision …’,
(b) ‘the undertaking is the first to submit evidence which in the Commission’s view may enable it to find an infringement of Article 81 EC’, while ‘the Commission did not have, at the time of submission, sufficient evidence to find [such] an infringement …’ and ‘no undertaking had been granted … immunity’ under (a).
It is further noted that even where an undertaking does not fulfil the stipulations for the granting of full immunity, it could be permitted a lessening in fines[25].
What is overlooked is that the undertaking may still be subject to civil proceedings[26]. In these proceedings, the evidence may be available through the process of the former investigations.

The European Court of Justice (ECJ) has held that EC law allows the Commission to compel interim measures[27] against an undertaking. This power was confirmed under article 8(1) of Regulation 1/2003[28].
Regulation 1/2003 assures the dicta of the ECJ in that it explicitly allows the Commission to issue interim orders. The circumstances that need to be in place before the Commission can impose such orders are consistent with those decided in Camera Care Ltd. v. Commission[29]. It is stated in Article 8(1) of Regulation 1/2003 that: ‘[i]n cases of urgency due to the risk of serious and irreparable damage to competition, the Commission, acting on its own initiative may by decision, on the basis of a prima facie finding of infringement, order interim measures.’ The requirement here is that there is a risk of “damage to competition”.

The new rules have provided the competition authorities with significant powers to investigate suspected behaviours that are adverse to an efficient competitive market. Powers to enter and search business premises with a warrant and even to search the premises of directors have significantly increased the reach of the competition authority.

There has been a growing recognition[30] that a more explicitly economics based approach to competition law enforcement is needed. Current thought amongst the majority of economic theorists is that form-based rules are not generally appropriate[31] as a deterrent to anti-competitive behaviour. In fact, it is arguable that there are both pro and anti competitive motivations, the most of the relevant forms of behaviour noted within the Modernisation package.

The overlaps that exist between pro-and anti-competitive behaviour make form based “lines in the sand” difficult to enforce and to justify on economic basis. It can lead to a danger of inconsistent treatment of different parties of similar facts. Often, conduct with equivalent effects on the market may be treated in opposing manners.
Markets are dynamic, as a result, the impact of an overzealous competition authority may result in mistaken interventions and false precedents that are difficult to assuage. The economic cost of these decisions may easily outweigh the effects of non-intervention.

It is often forgotten by the competition authorities that they are open freedom to intervene creates uncertainty[32]. It has been stated that the foundation of commerce in the UK was built on the pillars of contracting and common law. It has been further argued, that these pillars provided the necessary environment and freedoms to promote growth within business and create certainty. This idea of certainty is fragile.

Concerns around price increases, output reductions, and quality deterioration are critical to decisions about monopoly. It is true that conduct which allows nothing other than price raising and output reduction decidedly wounds competition, the free market and thus consumers. Economics theory supplies the deadweight loss triangle[33] as a representation of this premise.

However, it needs to be remembered that the economic value of enforcement needs to be weighed against the full opportunity costs and value in non-intervention commitments. It should be remembered that even if some detrimental behaviour escapes detection, it is the overall strength of competition within the market that is of primary concern.

Economist John Hicks noted that “[I]t seems not at all unlikely that people in monopolistic positions ... are likely to exploit their advantage much more by not bothering to get very near the position of maximum profit, than by straining themselves to get very close to it. The best of all monopoly profits is a quiet life.”[34]
In the USA, Judge Easterbrook noted, "[t]he gale of creative destruction produces victims before it produces economic theories and proof of what is beneficial.[35]" It is imperative to consider both current business practice and the related principles that may be applied to innovative progress in the commercial realm.

Risks of not complying
An undertaking faces serious consequences where it is found either to be party to an anti-competitive agreement or to have abused a dominant position. The potential consequences include serious fines (of up to 10% of the undertakings worldwide turnover), threats of criminal prosecution against individuals, disqualification for directors of up to 15 years and the voiding of key provisions in agreements.

Arguably, it could be said that the disruption to a business end the ensuing damage to a corporation’s reputation ensuing from competition investigations are potentially worse for the undertaking then be aforementioned possible consequences. Additionally, the undertaking could face subsequent litigation from its consumers or competitors.
Lengthy investigations by the competition authorities, whether conducted by the ECJ or an NCA, are both time-consuming and costly. It is in an undertakings interest to avoid such costs and delays due to the associated economic losses.

The introduction of self-assessment through Modernisation has placed the risk of non-compliance squarely on the shoulders of the undertaking. By removing the formal process were an undertaking could notify an agreement to the European commission or the office of fair trading, the Modernisation Regulations have significantly reduced legal certainty for businesses.

It is often forgotten by regulators that: “The mere possession of monopoly power, and the concomitant charging of monopoly prices, is not only not unlawful; it is an important element of the free-market system. The opportunity to charge monopoly prices - at least for a short period-is what attracts "business acumen" in the first place; it induces risk taking that produces innovation and economic growth.”[36]

The effects of Modernisation on Competition
“The system of EU merger control which was in force before May 2004 failed to explicitly endorse efficiency considerations”[37].

Harvard economist Joseph Schumpeter's examinations[38] of monopoly found that that superior profit provides "baits that lure capital on to untried trails," thereby fabricating a "perennial gale of creative destruction" with the result that the needs and desires of the market and consumers are better served. NCAs often miscomprehend how the existence of undertakings with large market shares does not reveal competitive harm. In fact, undertakings characteristically attain large market shares through tendering products and services that the market favours over the offerings of other less successful undertakings.

In the US, it has been recognised that the "costs" of "antitrust intervention" must be evaluated against the "benefits."[39] The difficulties in demonstrating conduct alleged as anti-competitive and the "cost of false positives" factor in the decisions of the court. Wide-ranging and effective tests, which provide an accurate assessments of any alleged anti-competitive conduct need to be introduced if the Modernisation Package is not to hurt competition.

The business world is characterised by a continually increasing and intense globalisation. Undertakings must by nature respond with mergers and acquisitions not only to shield and strengthen their competitive position in the market, but also to survive in it. Economies of scale, workforce rationalisation or managerial efficiencies[40] all add too help an undertaking remain competitive.

On occasion, the commission has relied on an undertaking’s efficiencies in order to prove the anti-competitive nature of the transaction[41]. This welfare approach to competition law has changed the nature of competition in the EU. Because of these changes within the UK and EU in general, is a “great gap separating the EU approach from the US system”[42].

There is a wide agreement that the aim of competition law is to promote efficiency in markets.
While it is true that undertakings with monopoly power can act anti-competitively and harm markets and consumer interests, and that NCAs should actively recognize and prosecute such behaviour, an undertaking’s size alone does not reveal damage to competition or an infringement of the competition laws. The appropriate aim of competition law enforcement needs to be focused against anti-competitive behaviour and consequence, not just size or market share[43].

Both in the UK and abroad, NCAs need to balance cost of false positive and false negatives. Value may be found in non-intervention even if some harmful conduct escapes. Market growth and finding the right balance is what is important.

The competition authorities have to remember that the goal of the Modernisation Regulations is to protect competition, not competitors. Moreover, undertakings and the market as a whole are advantaged by providing certainty in apparent, administrable, and purposeful legislation that permits undertakings concurrently calculate the legitimacy of their conduct prior to acting and simultaneously enable the competition authorities and courts to arbitrate disputed behaviour as expected and precisely.

Professor Carlton, has noted, “[e]fficiencies are hard to measure, and the benefit of the doubt should go to defendants, not to plaintiffs; otherwise, the continued generation of the large efficiency benefits responsible for raising our standard of living will be jeopardized”[44]. Competition is wounded by the competition authorities on every occasion that an undertaking is prevented from pursuing an aggressive strategy due to doubts founded in uncertainty caused by a gratuitously unrestrained interpretation of the Modernisation Regime.

Finally, enforcement needs to be effectual and administrable by the courts and competition authorities without constraining competition. Any remedy that troubles competition is inferior to no resolution at all.
The Modernisation Package has reduced processing times, but uncertainty remains.

1. Allen, T. (2005), “Interest in Class Action Grows Outside the U.S.” Securities Litigation Watch.
2. Andreangeli, Arianna (2004) “COURAGE LTD v CREHAN AND THE ENFORCEMENT OF ARTICLE 81 EC BEFORE THE NATIONAL COURTS” Case Comment, E.C.L.R. 2004, 25(12), 758-764 (European Competition Law Review)
3. Barnett, Thomas O. (2006), “The Gales of Creative Destruction: The Need for Clear and Objective Standards for Enforcing Section 2 of the Sherman Act”, Opening Remarks for the Antitrust Division and Federal Trade Commission Hearings Regarding Section 2 of the Sherman Act Washington, D.C.; June 20, 2006
4. Bittlingmayer, G. and T. Hazlett (2000), “DOS Kapital: Has Antitrust Action Against Microsoft Created Value in the Computer Industry?” Journal of Financial Economics 55 (March), pp. 329-35.
5. Buchan, Nancy (2004) “Fairness is in the eyes of the beholder”, University of Wisconsin - Madison
6. Carlton, Dennis W. & Perloff, Jeffrey M. (2005) “Modern Industrial Organization” 4th ed. Addison-Wesley USA
7. Clarke, Christopher (2004), “Competition Policy in the UK and its Implications for Japanese Competition Policy” Japanese Fair Trade Commission seminar, The UK Competition Regime, Recent Changes and Future Challenges. Speech by Deputy Chairman, UK Competition Commission in Tokyo on 6 December 2004
8. Commission Notice on Immunity from Fines: the "whistle blowers charter" 19/02/02 OJ C45/9
9. Commission (1999), “White Paper on Modernisation of the Rules Implementing Articles 85 and 86 of the EC Treaty”, Commission Programme No. 99/027, Brussels. April 28, 1999
10. Colley, L (2004), “From “Defence” to “Attack”? Quantifying Efficiency Arguments in Mergers” 25 ECLR 342, 343.
11. Easterbrook, Frank H. (1984) “The Limits of Antitrust”, Tex. L. Rev. 1, 5.
12. European Commission (2005) “Green Paper – Damages actions for breach of EC antitrust rules”, 19 December. 2005, COM (2005) 672 final, available at
13. Freeman, Peter (2005) “Lord Fletcher Lecture: UK Competition Law after Modernization”, Lord Fletcher Lecture; Tuesday 15 Mar 2005
14. Grout, Paul A, & Zalewska (2006) “Profitability Measures and Competition Law”, W. D. Collins (ed.), Issues in Competition Law and Policy, American Bar Association, July 2006
15. Gaughan, P.A. (2002), “Mergers, Acquisitions, and Corporate Restructurings” John Wiley & Sons, Inc.
16. Hemphill, C. Scott (2006) “Paying for Delay: Phamacetical Patent settlement as a regulatory design problem”, NEW YORK UNIVERSITY LAW REVIEW Vol. 81: pp1553-1616
17. Hicks, J.R., (1935) ”Annual Survey of Economic Theory: The Theory of Monopoly”, 3 Econometrica 1, 8.
18. Kirkwood, John B. (2005) “Buyer Power and Exclusionary Conduct”, Antitrust L.J. 625
19. Kocmut, Mitja (2005) “THE ROLE OF EFFICIENCY CONSIDERATIONS UNDER THE EU MERGER CONTROL” The University of Oxford Centre for Competition Law and Policy Working Paper (L) 09/05
20. M de la Mano (2002) “For the Customer’s Sake: The Competitive Effects of Efficiencies in European Merger Control”, Enterprise Papers No 11 European Commission 2002.
21. Mannix, Elizabeth A., Margaret A. Neale and Gregory B. Northcraft (1995), “Equity, Equality, or Need? The Effects of Organizational Culture on the Allocation of Benefit and Burdens”, Organizational Behavior and Human Decision Processes, 63 (September), pp 276-286.
22. Monti, M. (2004a), “The EU Gets New Competition Powers for the 21st Century”, Competition Policy Newsletter Special Edition.
23. Monti, M. (2004b), “Private Litigation as a Key Complement to Public Enforcement of Competition Rules and the First Conclusions on the Implementation of the New Merger Regulation, Speech 04/403”, paper for IBA 8th Annual Competition Conference, Fiesole, 14 September, available at
24. Motta, M. (2004), “Competition Policy: Theory and Practice”, Cambridge: Cambridge University Press
25. OECD “Competition on the Merits”, DAF/COMP(2005)27 Organisation for Economic Co-operation and Development, 30-Mar-2006
26. Rodger, Barry (2001) “Competition law compliance: role for profession” The Journal, January 2001, p 23
27. Röller, L.H. (2005), “Using Economic Analysis to Strengthen Competition Policy Enforcement in Europe” Brussels, March 21: Chief Competition Economists at the European Commission.
28. Schaub, A. (2002), “Antitrust Law Enforcement - A Shared Trans-Atlantic Vision”, paper for Bi-Annual Conference of the Council for the United States and Italy, New York, 25 January 2002.
29. Schumpeter, Joseph A. (1942) “Capitalism, Socialism and Democracy”, Harper Perennial, 1976.
30. Whish, Richard (2003) “Competition Law” Butterworths UK
31. Zwick, Rami and Xiao-Ping Chen (1999), “What Price Fairness? A Bargaining Study,” Management Science, 45 (January), 804-823.

Case 311/84 Centre belge d’études de marché – Télémarketing (BEM) v. SA Compagnie luxembourgeoise de télédiffusion (CLT) Information publicité Benelux (IPB) [1985] E.C.R. 3261 at 26.
Case 56/65, Société La Technique Minière v. Maschinenbau Ulm GmbH, [1966] E.C.R. 235, [1966] C.M.L.R. 357.
Case 792/79R Camera Care Ltd. v. Commission [1980] E.C.R. 119
Case 8/72 Vereeniging van Cementhandelaren v. Commission [1972] E.C.R. 977, at pp 30-31
Coöperatieve Vereniging ‘Suiker Unie’ UA, [1975] E.C.R. 1663, at 173.
Höfner & Elser [1991] E.C.R. I-1979, at 21
Imperial Chemical Industries Ltd. v. Commission, [1972] E.C.R. 619 at 64
Verizon Commc'ns Inc. v. Law Offices of Curtis V. Trinko, LLP, 540 U.S. 398 (2004).

Statues and Regulations
Clayton Act § 7, 15 U.S.C. 18
Commission Notice on Immunity from Fines: the "whistle blowers charter" 19/02/02 OJ C45/9.
Community Merger Regulation (Council Regulation (EC) No. 139/2004, OJ 2004. L24/1) (“ECMR”),
Article 81
Article 82
Council Regulation (EC) No 1/2003 (known as the Modernisation Regulation)
Council Regulation No. 17/62, 13 J.O. 204 (1962)
Council Regulation (EC) No 1216/1999 (OJ L148, 15.6.1999)
Sherman Act, 15 U.S.C. 1 et seq.:
§ 1, 15 U.S.C. 1
§ 2, 15 U.S.C. 2
The Competition Act 1998 (Commencement No. 3) Order. 1999
The Enterprise Act 2002 (Commencement No. 6) Order 2004. 2004 No.1866 (C. 78 )

[1] OJ 1962, 13 204/62, p87, as amended by Council Regulation (EC) No 1216/1999, 1999 OJ (L 148) 5.
[2] Regulation 1/2003, Art. 35 requires member states to designate responsibility for the enforcement of articles 81 and 82 in their respective territory to one or more national competition authorities. It is deemed acceptable to appoint national courts for this purpose.
[3] EUROPEAN COMMISSION, DIRECTORATE-GENERAL FOR COMPETITION, EUROPEAN UNION COMPETITION POLICY – XXXIInd Report on competition policy (2002) (32nd Report), at p.19, available at
[4] Rodger, 2001 notes that the Competition Act effectively sought to introduce a system of punishment and reward to induce compliance with the Act.
[5] The legislation provides for fines of up to 10 per cent of a company’s U.K. turnover for three years. In contrast, the “carrot” (see Rodger, 2001) is a leniency policy for whistleblowers who cooperate with the competition authorities. This provides for their fines to be reduced by as much as 100 per cent in some instances. It needs to be noted that the protection does not extend to civil actions against the party seeking leniency leaving them liable to possible civil claims for damage.
[6] The powers of the Competition Commission are available at their web-site -
[7] The standard protections against self-incrimination in criminal matters remain in force.
[8] As noted by Clarke, 2004, the member states was about to enter a period of significant expansion.
[9] EC Regulation 1/2003, referred to as the Modernisation Regulation
[10] As it was numbered at the time of presentation
[11] Freeman (2005) in the Lord Fletcher Lecture: UK Competition Law after Modernization
[12] Imperial Chemical Industries Ltd. v. Commission, [1972] E.C.R. 619 at 64; and Coöperatieve Vereniging ‘Suiker Unie’ UA, [1975] E.C.R. 1663, at 173.
[13] The term “undertaking” is used in this paper to broadly define “any entity engaged in an economic activity regardless, of its legal status and the way in which it is financed” as in Höfner & Elser [1991] E.C.R. I-1979, at 21. The term “undertaking” may thus consist of natural persons, partnerships, companies and public bodies engaging in an economic activity.
[14] Article 81(1) relates in particular to conduct in the nature of “agreements between undertakings, decisions by associations of undertakings and concerted practices”. All of these categories of conduct are referenced together as “agreements” for the purposes of this paper.
[15] “Fairness is in the eyes of the beholder” [Buchan, 2004]. As a result, it is difficult if not impossible to come up with a logically defensible means of assessing “fairness” for all parties. Mannix et al. (1995) effectively supports this assertion and demonstrates that there is no point of “fairness” in that this is an arbitary point of view with no justification in economic thought. As Buchan states, “the link between what players believed to be fair and what they actually did was somewhat tenuous. Specifically, when what was perceived to be fair coincided with what was in player’s self-interest”.
[16] The European Court of Justice (ECJ) has held that Article 81(2) does not require that agreements be automatically declared void ab initio. It may where suitable separate the infringing provisions where doing does not result in an injustice to the parties’ objectives. The scope and possible availability of any severance is left as a matter of national law at the discretion of a national court to apply. See Société La Technique Minière v. Maschinenbau Ulm GmbH, Case 56/65, [1966] E.C.R. 235, [1966] C.M.L.R. p. 250.
[17] Case 311/84 Centre belge d’études de marché – Télémarketing (BEM) v. SA Compagnie luxembourgeoise de télédiffusion (CLT) Information publicité Benelux (IPB) [1985] E.C.R. 3261 at 26.
[18] As was noted in the European Commission, “Green Paper – Damages actions for breach of EC antitrust rules”, 19 December. 2005, COM (2005) 672 final.
[19] Assuming that the agreement does not satisfy the conditions in Articles 81(3))
[20] Art. 3(1). The test for “affect on trade” has been interpreted expansively. It may consist of agreements amongst parties located in the same member state and which is intended to manage their behaviour in that same member state. See Also: Case 8/72 Vereeniging van Cementhandelaren v. Commission [1972] E.C.R. 977, at pp 30-31.
[21] Freeman, 2005 states that the Commission and the CC have developed close ties and operating efficiencies.
[22] Cooperation Notice, at p 5. Article 13 of Regulation 1/2003 corroborates the power allowing NCAs and the Commission to suspend proceedings or reject a complaint altogether where identical behaviour is being investigated by another NCA. Apart from where the Commission has commenced proceedings the NCAs are not required to suspend proceedings or disallow a complaint where the same conduct is being investigated by a different NCA.
[23] Commission Notice on Immunity from Fines: the "whistle blowers charter" 19/02/02 OJ C45/9
[24] However, the undertaking is required to “cooperate fully, on a continuous basis and expeditiously throughout the Commission’s administrative procedure and must provide the Commission with all evidence that comes into its possession or is available to it relating to the suspected infringement; end its involvement in the suspected infringement no later than the time at which it submits evidence”; and not have taken steps to coerce other undertakings to participate in the infringement”.
[25] See: Leniency Notice, at pp 20-23. To qualify for leniency, an undertaking must supply the Commission with evidence of the suspected infringement that characterizes significant additional value with respect to the evidence previously in the Commission’s control and must conclude its participation in the suspected infringement no later than the time at which it surrenders the evidence.
[26] See: Allen, 2005. It is noted that the numbers of class actions against undertakings outside the US is increasing making the acceptance of the Leniency provisions more risky from the perspective of possible class actions against the undertaking.
[27] 792/79R Camera Care Ltd. v. Commission [1980] E.C.R. 119
[28] “[i]n cases of urgency due to the risk of serious and irreparable damage to competition, the Commission, acting on its own initiative may by decision, on the basis of a prima facie finding of infringement, order interim measures”.
[29] Case 792/79R Camera Care Ltd. v. Commission [1980] E.C.R. 119
[30] See: Easterbrook, 1984 and Gaughan, 2002.
[31] See: Easterbrook, 1984 and OECD “Competition on the Merits”, DAF/COMP(2005)27
[32] Hemphill, 2006
[33] Carlton, 2005
[34] Hicks, 1935 show that the monopolist is more seeking an “easy” position rather than to actively pursue a strategy of crushing their opposition.
[35] Easterbrook, 1984
[36] Verizon Commc'ns Inc. v. Law Offices of Curtis V. Trinko, LLP, 540 U.S. 398 (2004). The reference to “business acumen” comes from United States v. Grinnell Corp., 384 U.S. 563, 571 (1966).
[37] Kocmut, 2005
[38] Schumpeter, 1942
[39] Trinko 540 U.S. at 414.
[40] See, M de la Mano, 2002
[41] AT&T/NCR (Case IV/M050) [1991] OJ C016/1, [30], the Commission responded: “It is not excluded that potential advantages flowing from synergies may create or strengthen a dominant position”.
[42] L Colley ‘From “Defence” to “Attack”? Quantifying Efficiency Arguments in Mergers (2004) 25 ECLR 342, 343.
[43] See: Kirkwood, 2005.
[44] As noted by Professor Carlton in Barnett (2006): “The Gales of Creative Destruction: The Need for Clear and Objective Standards for Enforcing Section 2 of the Sherman Act”

Thursday, 26 June 2008

Random Forests

A random forest algorithm is an ensemble of unpruned decision trees. They are commonly deployed where there are extremely large training datasets and an exceedingly large quantity of input variables. In Security and risk, the dimensionality can run into the thousands of input variables. A Random Forest model generally comprises of up to hundreds of individual decision trees.
The primary benefit to risk modelling is that Random Forests tend to be very stable in model building. Their relative insensitivity to the noise that breaks down single decision tree induction models makes them compare favourably to boosting approaches while they are generally more robust against the effects of noise in the training dataset. This makes them a favourable alternative to nonlinear classifiers like artificial neural nets and support vector machines.

As the performance is frequently reliant on the individual dataset, it is a good practice to compare several approaches.

Each decision tree in the forest is constructed using a random subset of the training dataset using the techniques of bagging (replacement). A number of entities will thus be included more than once in the sample, and others will be left out. This generally lies in the two thirds to one third ratios for inclusion/exclusion.

In the construction of each decision tree model, an individual random subset of the training dataset uses a random subset of the presented variables in order to decide as to where to partition the dataset at each node. No pruning performed as all decision trees are assembled to their maximum magnitude. The process of building each decision tree to its maximal depth results in a less biased model.

The entirety of the decision tree models taken together form the forest. In this, the forest characterizes the final ensemble model. Each decision tree in this model effectively casts a vote with the majority outcome being classified as the outcome. In the case of regression models, the average value over the ensemble of regression trees is averages to produce the assessment.

A random forest model is effective for building Security Risk models due to a number of reasons:

  1. The amount of pre-processing that needs to be preformed on the data is minimal at most,

  2. The data does not need to be normalised and the approach is resilient to outliers,

  3. Variable selection is generally not necessary the event that numerous input variables are present prior to model building,

  4. All of the individual decision trees are in effect independent models. When taken with the multiple levels of randomness that exists within Random Forests, these models tend not to overfit to the training dataset.

The Challenge

Conventional dissimilarity measures that work for simple Risk data may not be optimal in modelling security risk. The use of dissimilarity measures that are based on the intuition of multivariate normal distributions (clusters have elliptical shapes) are generally found not to be optimised in modelling risk. This makes it desirable to have a dissimilarity that is invariant under monotonic transformations of the expressions derived from the risk metrics.

The RF dissimilarity focuses on the most dependent markers whereas the Euclidean distance focuses on the most varying marker.

Casting an unsupervised problem into a supervised problem

The more important a system is according to RF, the more important it is for survival prediction.

This allows the security risk practitioner to select systems based on quantitative measures rather than perception.

Ho, Tin Kam (1995). "Random Decision Forest". Proc. of the 3rd Int'l Conf. on Document Analysis and Recognition, Montreal, Canada, August 14-18, 1995, 278-282 (Preceding Work)

Maindonald, John (Australian National University) Notes from his upcoming publication.

Wednesday, 25 June 2008

Risks of Outsourcing

Outsourcing is a process of acquiring services from an external party. In the strictest of legal senses, all companies outsource. The statement makes the most sense when you understand that a company can only act through its employees and directors. The company is not an independent entity with a will of its own. Employees are sourced from a location external to the organisation and can for the most part, leave with little effort.

In addition, services that do not have as much of a strategic focus are generally acquired from contractors. In fact, for any service that is not critical to the organisation in the sense that it is not something that they will excel at, it is generally better to obtain the services from a specialist. In most cases the specialist is a contract. Companies have relatively high degree of control over both employees and contractors. When they do this, they shouldn't most if not all the risk of what these parties do. There are some exceptions. In the case of vicarious liability, criminal acts that require mens rea cannot be directly attributable to the company but are rather associated with the individuals responsible and the directors.

In the majority of cases, the company will usually owed or otherwise control the technologies and assets that the employees (and sometimes the contractors) utilise. In this case, the whole risk of the system remains with the company. For key systems this can make sense. As an example, it would not make sense for a manufacturing organisation to outsource the quality control systems within their own facility.

On the other hand, call centre, payroll, data facilities, telecommunications and even warehouse inventory control systems may not be within the general scope of what an organisation specialises in. In these cases, the outsourcing of these operations may be the most effective way to manage the risk associated with these functions.

There are a number of possible solutions to outsourcing. Some of these include:

  • Acquiring a turnkey solution that is commissioned in its entirety. In many cases the company will take over risk and acceptance or the latest at the end of the warranty period.
  • Using a variety of suppliers and vendors the source individual component pieces and then hiring systems integrators to install and run them. The company takes more upfront risk in this instance but can handle some of the ongoing risk to the outsourcing party.
  • The other option is to contract the entire operation to a specialised service provider. This has become common within IT and especially with the ASP models. In this instance, the outsourcer is responsible for acquiring all the equipment and expertise. In this instance the outsourcer has complete control of risk in the system. In this latter example the risk of non-performance remains. The company will have no control of and no risk in the technology itself however.

The important consideration in all of these instances is that not all risk can be transferred. In the case of financial reporting obligations, a failure on nonperformance by the outsourcer will leave the company vulnerable. Likewise, many other risks will not be transferred but will rather be shared between both the company and the outsourcer. In this instance insurance is important.

It is necessary to protect against the risk the supplier will not deliver upon its promises. To do this, a company needs to ensure that escrow arrangements have been made. Escrow arrangements rely on a third-party those entrusted with the source code, drawings, plans, designs and other documentation necessary for the operation and maintenance of the system.

Escrow arrangements are enacted only in the event that the supplier fails.

There are many types of insurer will risks and just as many types of insurance to go with them. Some of many categories of insurance include:
  • professional services,
  • product insurance,
  • asset protection,
  • contract liability,
  • workers compensation,
  • intellectual property protection,
  • insurance against damage to property or injury to people,
  • litigation insurance, and
  • business interruption protection.
The key point to remember is that insurance does not remove risk but rather transfers selected instances of risk. It is important to always examine the insurance contract. Named parties and covered risks should be taken into account and always understand the exclusions that exist within the policy.

Tuesday, 24 June 2008

Can't see the tress because there's no forest

Continued from Part 2.

(The start... Part 1)

What the varve data analysis demonstrates is that climate changes. This is not a result of human intervention through CO2 emissions, rather the correlations to CO2 could be be demonstrated to be related to temperature. That is, if the other way around.

Deforestation is also related to CO2. Mature trees do not make a great amount of O2 as people are taught in school. This occurs as the grow and store carbon. When mature the balance forms an equilibrium. When they are removed, the balance is disturbed.

In addition to deforestation around the globe, the most significant sources of forest around the world aredisappearing in every country where they are located.


The problems with deforestation

  • Soil erosion With the loss of a protective cover of vegetation more soil is lost.
  • Silting of water courses, lakes and dams This occurs as a result of soil erosion.
  • Extinction of species which depend on the forest for survival. Forests contain more than half of all species on our planet - as the habitat of these species is destroyed, "Biodiversity" declines.
  • Desertification The causes of desertification are complex, but deforestation is one of the contributing factors to desertification. Less trees equals a dryer climate.
  • Flooding
Cutting down trees is not the issue as is commonly stated either. The issue is not replacing what we have removed. This is a problem with poor governance and stewardship. Deforestation is an issue that has an impact now. It is something we can do now. Debating CO2 is a way of avoiding the real issues.

Further Reading
The following refernces provide more information.

Can't see the tress... Because there's no forest

Climate has varied on every time scale to which we have any observational access. Ice ages come and go on time scales of tens of thousands of years, for example. . . . Climate changes. It changes on all time scales. What's different between our time and our grandparents' time is that now humankind, which has been a passive spectator at this great natural pageant, has become an actor and is up on the stage. And what we—all 6 billion of us—do can affect the climate”. (Quoted from PBS, What's Up with the Weather?)

Speculation into CO2 changing the whether is the least of the world’s problems. We are doing our best to both deforest the planet and to reduce the algal plankton. Both factors do impact climate. More importantly, they have a positive effect to existing situations that can be modelled now without speculation.

Paleoclimatic Glacial Varves

Melting glaciers deposit yearly layers of sand and silt during the spring melting seasons, which can be reconstructed yearly a period ranging from the time deglaciation began in New England (about 12,600 years ago) to the time it ended (about 6,000 years ago). Such sedimentary deposits, called varves, can be used as proxies for paleoclimatic parameters, such as temperature, because, in a warm year, more sand and silt are deposited from the receding glacier. Dataset varve.sav contains glacial varve thicknesses from one location in Massachusetts for 634 years, beginning 11,834 years ago. The timeplot of the data is given in the figure 1 below. Because the variation in thicknesses increases in proportion to the amount deposited, a logarithmic transformation could improve the nonstationarity observable in the variance as a function of time.

Figure 1 - A time plot of the varve thickness

The variance of log(varve) is more stable. Outliers are less obvious. Some minor spikes (see observation numbers 568 and 572 remain unusually large and are unusually high). The plot suggests non-stationarity.

The slowly decaying autocorrelations of log(varve) indicate nonstationarity which will be examined to see if it can be removed by differencing.

A time plot of first differences of log(varve) is given below.
The first difference looks like a stationary series. The autocorrelogram is given below. In the differenced series, autocorrelations decay rapidly. In fact, only the lag 1 autocorrelation is significant, suggesting the possibility of an ARIMA(0,1,1) model. First, however, We examine the partial autocorrelations of the first difference of log(varve).

The partial autocorrelations decline slowly, supporting a moving average, rather than an autoregressive process.

From this we see that a good starting model would be an ARIMA(0,1,1) model of logarithmically-transformed data. Other models could be ARIMA(p,1,1), for p=1,2,3,4,5. Calculating the AIC and SBC, we obtain the most minimum AIC and SBC with the ARIMA(1,1,1) model without constant.

To be continued... Post Part 2

Can't see the tress... Because there's no forest (Part 2)

Continuing from Part 1.
The ARIMA(1,1,1) model has a smaller AIC and a smaller residual variance. In the ARIMA(0,1,2) model, the MA(2) parameter is not significant, and all estimates are highly correlated. This suggests that the ARIMA(1,1,1) is preferable.

To check the need for additional parameters, we fitted both models with extra parameters, with and without a constant. Results are summarised in the following table:

In models with (p+q) = 3, estimates were very highly correlated. Consequently a number of such models had identical AIC and residual variance. This suggests that that many parameters are not necessary. Therefore I prefer models with (p+q) not exceeding 2. Among these, the ARIMA(1,1,1) without constant has the smallest AIC and residual variance.

Details of the ARIMA(1,1,1) model with no constant are given below.
Note: Both parameters are statistically significant, and they are only moderately correlated.
A time plot of residuals does not show any pattern. and a plot of residuals against fitted values looks like a random scatter (below).

The autocorrelogram of residuals is shown below.

The results are consistent with a white noise process. The Box-Ljung statistics are not significant at any lag. We can the see a histogram of residuals seems to approximate a normal curve (as shown below).

The histogram shows a good approximation to a normal distribution. The normal Q-Q plot and normality tests also are consistent with the residuals being normally distributed. The normality tests also confirm the normality.

Next we do a diagnostic on the overfitting through the fitting of an ARIMA(2,1,1) or ARIMA(1,1,2) model to verify if the choice of ARIMA (1,1,1) is a good. As is seen below, with ARIMA(2,1,1), the second coefficient of AR is insignificant with the p-value is approximately 28.5%.

This summarises the good of fit of ARIMA(1,1,1) without constant.

As shown in the above plot, on average the fitted values underestimates the original values, so ARIMA(1,1,1) without constant is a good model, but is not the best one. According to the text, Shumway and Stoffer (2000), page 170, one should consider Long Memory ARMA models for this dataset (ie differencing d = 0.384), but this model beyonds the level of this course.

To be Continued in Part 3.

Monday, 23 June 2008

Legal issues with Reversing

In Australia, as with many other jurisdictions, the reverse engineering of software will be considered to be legal as long as it falls within one of the defined defences against copyright infringement. These are detailed below.

Additionally, it is possible to prove the legality of a reversing procedure if it is done using a cleanroom process. A cleanroom procedure involves:

  • obtaining a legally licensed version of the software,
  • having one team of analysts to disassemble and the compile the software. This team would then create flow documents and process requirements from their understanding of the software and structure. Can it be taken when explaining the structure to ensure that a breach has not occurred,
  • an alternate team of programmers could then use the structure of document to rewrite new software with new code.

In order for the copyright owner to prove the reverse engineering of the software is an infringement, it is necessary that the following conditions apply:

  • a valid right to copyright subsists on the software (it is currently unlikely that any copyright term would have expired),
  • the source or object code (or a substantial section thereof) has been created through a significant reproduction or adaptation of the original code, and
  • one of the valid defences (listed below) does not apply.
When a user has legitimately obtained a copy of software, there are a number of relevant defences to copyright infringement. It should be further noted that the copyright holder or owner is not permitted under law to require the user to contract out of these rights. These defences include:
  • creating a backup copy of the software,
  • making a copy of the software in the normal course of running the program,
  • making a copy of the software or its components in order to patch or otherwise correct errors in the software. This defence is limited to the situation where the said patch is not available within a reasonable time and also at an ordinary commercial price,
  • making a copy of software to obtain information in order to ensure system interoperability. This defence is limited to cases where that information is not readily available from another source already, and
  • making a copy of software for the purposes of testing the security of the product.
Reversing is still dangerous. Even while trying to avoid copying software an unconscious copy could be made. This is why cleanroom procedures are so critical. A strange as it seems, it is possible to copy something without making reference to the original source in a manner that breaches the copyright laws. This is generally known as unconscious copying.

Even when a cleanroom procedure is correctly followed, some illegal copying could have occurred. There needs to be taken in the creation of the explanatory document to ensure that this is not a consequence of unconscious or unintentional copy.

Autodesk v. Dyason (1992) 173 CLR 3330; 220 IPR 163 is a prime example of this risk.

Although there is a valid defense of security testing and for patching, it is essential to ensure that a commercial fix is not readily available prior to testing. Most importantly, document each step and ensure that records of all the testing exist.

Sunday, 22 June 2008

Another week...

Some more images from the property.
It has been raining, but only lightly over the last week. As such the stream is flowing gently and clear.
Our quarter house - Flame.
And Lady. The other are off somewhere.
Winter is mild this year. The frost was not as bad as last year's one and the grass has not browned. The trees are still bare.
A hawk. All is quiet at the moment with the hawk overhead.