Australia's First Online Journal Covering Air Power Issues [ISSN 1832-2433]
F-22A Raptor, FB-22, F-22E, F-22N and Variants Index Page [Click for more ...] People's Liberation Army Air Power Index Page [Click for more ...]
Military Ethics, Culture, Education and Training Index Page [Click for more ...]
Russian / Soviet Weapon Systems Index Page [Click for more ...]


Home - Air Power Australia Website


Last Updated: Mon Jan 27 11:18:09 UTC 2014







An Analysis of Defence Materiel Organisation Major Projects Management and What Needs to be Fixed

Air Power Australia Analysis 2011-02
  23rd February 2011

A Monograph by
Air Commodore Ted Bushell AM (Retd)
Text © 2011 E. J. Bushell


A Royal Australian Navy Kaman Seasprite helicopter in 2007. The Seasprite program failure was characteristic of DMO practices which persist. The program resulted in the much publicised cancellation and withdrawal from service. None of the root causes of the Seasprite failure have been addressed (©2007 Carlo Kopp).

Executive Summary

Over the past decade or more, a steady stream of Defence capability acquisition problems has been highlighted in the Press and by defence analysts. The underlying cause may be traced to the abandonment in 1998 to 2001 of the successful project and engineering management systems and procedures that had been built up within the three Service Departments over decades of experience.

Domain experts, working to tested and proven, capability and evidence-based, project management procedures, were replaced, under Defence executive direction, by non-experts possessing no knowledge or experience of the technologies involved, or of the project management systems essential to success. Project Management Boards were replaced by Governance Boards that lacked critical competencies and experience, and quasi-commercial/business-orientated processes were adopted as the primary management focus within Defence/DMO.

After some seven years of major acquisition problems, Parliament, in 2003, called for the Auditor-General to produce an annual report on major Defence projects. An initial (trial) report for 2007-08 (1) was issued in November 2008, covering nine major projects. A second report, covering fifteen major projects (2) was issued in November 2009. Analysis of the latter report indicates that, after ten years of Defence/DMO (3) management, the problems introduced by the change in management methodology that was imposed between 1998 and 2001 remain.

The 2008-09 Defence Materiel Office Major Projects Report (DMO MPR), builds upon the 2007-08 DMO MPR, but analysis shows that nothing of substance has changed, apart from some more data, many more levels of process, and some optimistic words. In short:

  • There is little, if any, evidence that critical deficiencies in the project, engineering, risk management and Independent Verification and Validation (IV&V) areas have been recognised and corrected. As a result, the Auditor-General has highly qualified his Department’s report.
  • The 2008-09 report contains more contract and financial detail, but still lacks clear, consistent, coherent and objective project status information. Until the root cause(s) behind DMO's capability, schedule and cost management problems are acknowledged, accepted, and corrected, the defence and security of Australia will continue to depend upon an increasingly threadbare patchwork of late, capability-deficient, and over-costly project outcomes, dependent for their through-life support upon risk-laden contracts with foreign companies.
  • The proper management of risk still eludes DMO, with 'identified risks' now incorporating redundant and inappropriate terminology that dilutes the risk management process, for example, 'There is a possibility that..' or 'There is a chance that..'. Risks are still too often ill-defined simplistically as 'Schedule' or 'Attainment of contracted technical performance', giving no indication as to the underlying technical, financial and/or schedule management factors that gave rise to the ‘identified risk’, its consequence(s), and the likelihood of its materialising.
  • The DMO still remains focussed upon supposedly commercial approaches and controlling financial liability to its contracts, both of which have proven ineffective, rather than managing the technical and project risks that could arise- an inevitable result of the switch to financial/business targets at the expense of functional values.
  • The 'lessons learned' as presented in the second MPR are still generic, recording only project management factors that should have been thoroughly recognised and catered for more than a decade ago, indeed before DAO/DMO embarked upon its first project. That these generic 'lessons' are still being put forward indicates that Defence/DMO is not a learning organisation.
  • While much is made of the many changes undergone by the DMO since the 2007-08 MPR, these have been in process rather than in improved management. DMO has increased the numbers and levels of executive oversight, rather than acquired much needed project management competencies and technical skills. Other changes, such as the Enterprise Risk Management Framework and Project Longitudinal Analysis, simply miss their mark. The practice of DMO trialling new contracting methodologies, such as the Alliance approach, which DMO plans to use over several projects so as to be able to assess its effectiveness, is a bizarre and extremely risky way of managing Australia's defence capabilities.
  • Within DMO, measurement of the critical performance elements of capability and schedule rest solely upon Project Maturity Scores, which, on analysis, are shown to have no firm basis, being, in the main, subjective in nature. Similarly, DMO's benchmarking is faulty, and only encourages poor performance.
  • As a result, ANAO/JPCAA are, understandably, meeting continuing problems and challenges in obtaining traceable, repeatable, and objective data upon which clear and consistent information on the status of projects may be derived.
  • Concern is also raised over the absence of any coherent development of Australia's Defence Industry in Defence/DMO plans, programs and reporting, which is shielding a continuing erosion of our national self-reliance. Defence Industry development policies and objectives need to be included in all Defence/DMO planning, as well as governance oversight activities. 
  • A case is made for the New Air Combat Capability (NACC/Joint Strike Fighter) Project to be subjected to an ANAO management audit, as this is probably the most risk-laden endeavour upon which Australia has ever embarked, and carries serious implications, both directly and indirectly, for the defence and security of Australia over the next three decades or more.
  • The analysis also draws attention to the findings of the recently-released 2010 US Quadrennial Defense Review (QDR), and particularly to the Report of the QDR Independent Review Panel which shows that the problems seen with the management of defence capabilities and acquisition in the US parallel those seen in Australia and for the same reasons.
Finally, while the root problem seen with DMO may be traced to the reorganisation, downsizing, and failure to manage the consequential de-skilling of the Services, there are also organisational fault lines that dictate against Defence/DMO becoming competent organisations. These fault lines come to a point at the professional military and bureaucratic Defence/DMO interfaces, where responsibilities, accountabilities, resource allocation and professional skills come into conflict, and lead to a dulling of the effectiveness of governance oversight of the organisations. As the Proust Review into Defence Management noted: “The current structure is confusing....it is necessary to get the structure right, by which we mean that the structure should be aligned with accountabilities and responsibilities.” Any reform of DMO should thus proceed in harmony with reform of the higher Defence organisation.

The principal objectives of this analysis are to:
  • Identify the systemic problems that have been allowed to develop within Defence/DMO capability and acquisition management over the past two decades. In doing this, some detail and repetition have been necessary in order to demonstrate that the problems identified are systemic and not 'isolated' or 'legacy' ones.
  • Emphasise the urgency for Defence/DMO reform if Australia is to achieve and maintain a force structure and military capabilities, in time and space, that best meet our strategic objectives.
  • Identify the need for reform of Defence/DMO to proceed in harmony with reform of the higher Defence machinery, so that accountabilities and resource allocation align with responsibilities.
In short, to develop an acceptance of the need and urgency for change, to restructure the Defence and Service Organisations, re-value the Services, and reimpose effective civil (rather than civilian) management and governance within and over defence matters.

However, the scope and the nature of the challenges identified in this analysis will call for strong, focussed and persistent cross-factional action, as national interests will have to be put before political considerations and executive objectives if Australia is to recover the professional military capabilities and competencies needed to face current and emerging threats.


INDEX
 

1.THE EVOLUTION OF DEFENCE/DMO MANAGEMENT PROCESSES

BACKGROUND

Over the past decade or more, a steady stream of Defence acquisition problems has passed through the press, ranging from minor, but operationally important, military purchases to major capability acquisitions. The higher end of the scale included such acquisitions as:
  • The 'Manoora' and 'Kanimbla' purchases.
  • The Super Seasprite Project.
  • The FFG Update Project.
  • The ARH (Tiger) Project.
  • The Collins Submarine Project.
  • The AEW&C Project, and so on.
In addition, capabilities eventually introduced have often lacked defensive or other systems which have prevented their use in combat situations; that is, they were, and some still are, not combat ready.

Each of these deficiencies has impacted adversely Australia's force structure, operational capabilities, and national security.

With each major acquisition problem, Defence has often claimed that the problem lies in the past, a 'legacy' problem, and has emphasised that things are now on track and that such problems will be avoided in the future. However, as time goes by, the same problems are seen to arise, with the same adverse impacts upon Australia's force structure, capabilities and security.

Pre-Defence Reform and Commercial Support (DRP and CSP) Programs, Australia's capability acquisition procedures were recognised and respected widely for their effectiveness, efficiency and economy. The management systems employed were based upon transparency and public accountability, and were able to provide clear and consistent information on the status of each project. RAAF, in particular, had developed extremely sound and competent policies, systems and procedures.

These skills and competencies persisted until about 1998 when they were abandoned in favour of those that gave rise to the major equipment acquisition problems that soon followed, that have persisted to this day, and that DMO has warned us to expect to continue for another decade or more.

It is thus timely to revisit the development of Defence/DMO procurement methodologies in order to identify why and how Defence abandoned the established, professional management of major capability acquisitions and adopted the problem-entrenched, executive-driven, business/commercial approaches seen today, and determine how this situation may be reversed.

THE IMPOSITION OF EXECUTIVE DIRECTION OVER PROJECT MANAGEMENT

The following observations have been taken from a submission made to the Joint Standing Committee on Foreign Affairs, Defence and Trade (JSCFADT) on ADF Regional Air Superiority to 2020, by Air Commodore Garry F. Bates, AM, in 2006 (4). Over the period being reviewed, Air Commodore Bates was Director General, Aerospace Combat Systems Branch in the Defence Acquisition Organisation (DAO), which later became the Defence Materiel Organisation (DMO), from 1997 to 2001. His responsibilities included Chair, Hornet Upgrade Project Board, F-111 Block Upgrade Projects Board, and the P-3C Update Project Board, as well as Acquisition Member, AIR 6000 Project Board (later to become the New Air Combat Capability Project – the JSF Project). His observations thus come from the centre of Defence/DAO/DMO higher management:

“The concerns I raise to this inquiry are my observations that the existing procedures were reconstructed by executive direction using externally sourced personnel who lacked domain exposure and the knowledge of past practices and pitfalls in acquisition. Worse, established processes were by-passed with limited regard to the enormous body of knowledge and the lessons learnt that had previously been incorporated.

I was very proud to be the Acquisition Member on the inaugural Project Board for Project AIR 6000, In accordance with the Project Management Method (PMM) which DAO was committed to at the time, the Project Mandate required an open mind to the pursuit of all capabilities, airborne and other, that could contribute to meeting the New Air Combat Capability requirement. Progressive Project Boards were fully minuted and inclusive of external contributions including the integral utilisation of a DSTO combat modelling expert seconded to the Capability staff, a system that had proved very effective with the source selection of the AEW&C contenders.

This process changed with the release of the Defence White Paper 2000 which declared the intention to acquire up to 100 fighter aircraft to replace the F-111 and F/A-18 capabilities. I was never party to the development of this position but understand that it was the position agreed by Defence with the Minister of the day. It was certainly a surprise to most members of the Project Board. The Board process began to disintegrate as the Chair (person) moved to another position, having been promoted at the behest of the Minister to the AEW&C project.

The incoming Chair for project 6000 was an officer returning from overseas tour of duty and unfamiliar with the acquisition reforms of recent years. He expressed the view that the PMM process was too arduous and was not supported by the DMO executive. Both views were true as the incoming USDM (Under Secretary Defence Materiel) did not accept the decision of the former Deputy Secretary that DAO would adopt PMM following successful trials and development of the process based on PRINCE 2 over a large number of projects for an extended period. This rejection was strongly supported by a civilian Division that was subsequently assigned leadership of developing yet another project management process.

In addition, the use of Project Boards, usually at one star level, did not sit comfortably with USDM. He introduced Governance Boards of external and internal consultants to advise him directly on project performance. It took considerable effort by the Aerospace Division, using an organisational concept model I devised, to convince USDM to retain the Projects Boards which managed the progress of projects and reported to the domain Division Heads (two star level) as complementary to the higher level Governance Boards which advised USDM.

However, AIR 6000 Project Boards did not continue under the new arrangements and I was not an active participant in the subsequent developments other than to establish a core project management team and second them to the capability staff as part of an Integrated Project Management Team that I had advocated for some years. My original project manager, a PMM expert, resigned from Defence in disgust at the relegation of his position and expertise. In late November 2001, capability staff presented an Industry Brief, a role usually undertaken by acquisition staff. The brief outlined the intention to seek information from industry as part of a three tiered approach: Capability Staff sponsored a Force Mix Market Survey of potential aerospace systems; DMO sponsored a Request for Information from international aerospace industry; and DSTO sponsored a Technical Data request to populate their modelling. All requests required responses by 1 Feb 02. My incoming project manager attended the brief and reported that it was not well received by industry with many attendees expressing grave concerns. I expressed my concerns to my superior and to CAF (Chief of Air Force) in an e-mail in December 2001 prior to my departure from the DMO.

Another factor which changed the face of the Air 6000 process was the public presentation made by then Minister Reith at the Defence and Industry Conference in June 2001. He introduced the identification of defence acquisition as a monopsony. My simple interpretation of the impact of this economic term is that contract savings are not necessarily best achieved via competitive tendering in the single buyer, single supplier relationship that represents most defence specialised acquisitions. This concept had been studied by RAND (ISBN: 0-8330-3009-4 MR-1362-OSD/JSF, 2001 http://www.rand.org/publications/MR/MR1362) and was put forward by Dr Paris Genalis of the USA during the ADAC 4 meetings in Canberra and later echoed by Dr Learmonth, Head Industry Division DMO at the Partnering and Alliances Seminar in November 2001. Dr Learmonth was subsequently assigned leadership of the JSF SDD study by USDM.

The consequence of this set of public statements was that there was no apparent commitment to competitive tendering for Air 6000 within the higher echelons of Defence at the time that the three tiered requests to industry were dispatched. I recall a comment previously made by USDM to Aerospace Systems Division star rank officers to the effect that he did not believe in spending funds on F-111 and F/A-18 upgrades and wanted to direct the money to early acquisition of the JSF. He also advised that he expected his staff to present counter-positions to capability staff to allow him and the Secretary to consider a range of possibilities rather than the position of capability staff. The time-frame of these comments was mid 2001. I was very uncomfortable with this position as it was totally counter to the Capability based Management System and its inclusive Integrated Project Management teams that had been in development for many years with the specific endorsement of the Defence Committee. I had been an early contributor to this model. My point here is that there was no agreed approach to the process management of Air 6000 within the hierarchy and clearly USDM was not going to take advice from those of us working within the previously endorsed acquisition process.”

SUMMARY

Air Commodore Bates' experiences show:
  • Domain experts, working to tested and proven Project Management procedures, were replaced, under executive direction, by non-experts possessing no knowledge or experience in the technologies concerned or the management systems involved.
  • Project Management Boards were replaced by Governance Boards comprising external and internal consultants to advise the executive heads, and hence Defence/Government/Parliament.
  • The move towards supposedly commercial/business orientated solutions to capability acquisitions may be traced to June 2001.
Governance Boards may play some useful role in some public service bureaucracies, but are wholly inappropriate for procuring specialised and usually complex (for non-domain experts), and expensive military capabilities, or the management of them, as in the case of the Defence Scientific and Technology Organisation (DSTO). This has been demonstrated repeatedly over the past decade or so in failed or deficient projects managed by the Defence/DAO/DMO organisations. Governance Boards have persisted since early 2000, increasing in their level of oversight but not possessing the domain expertise or management systems and competence that are central to effective project management. That the Air Warfare Destroyer and Replacement Submarine projects have Governance Boards at the highest level bodes ill for both projects.

While Defence/DMO have been urged repeatedly to adopt a more (invariably ill-defined and not justified) commercial/business approach, the adoption of a contract-centric focus, as opposed to a capability and evidence-based project management system, will never provide the unrealistic benefits proffered by the seductive marketeer, or meet the unrealistic expectations of Defence/DMO.

2.THE INTERVENTION OF PARLIAMENT
In March 2003, the Senate Foreign Affairs, Defence and Trade Reference Committee reported on materiel acquisition and management in Defence, finding that there was relatively poor visibility on the progress of major projects as far as Parliament and the public are concerned, and called for the Auditor-General to produce an annual report on the progress of major Defence projects. The first of such reports (a trial) was issued on 20th November 2008, covering nine projects.

It is interesting here to note that the DMO MPR 2007-08 (Page 13) spoke of:

“These projects are often expensive, technologically advanced and managerially challenging, and require DMO to manage contracts that:
  • are inherently complex and require sophisticated management processes;
  • often specify substantial payments prior to Defence being able to test and operationally evaluate the capability being acquired; and
  • involve significant risks and issues that may only be identified during the latter phase of development tests and evaluations”.
Two important observations arise here:

Firstly, the Auditor-General refers to “DMO to manage contracts that:”, which implies that the contract manages the three factors listed above, whereas no contract can, and should not attempt to do so.

The three factors listed fall wholly within the scope of Project Management where operational and engineering domain experts work within established policies, systems and procedures designed specifically to:
  • manage complexity, using established, capability-based, management systems,
  • ensure that all work is done in accordance with the requirements specified by Project Management before being approved for payment under the contract, and
  • manage all risks (almost invariably operational/engineering/schedule in nature) so that they are mitigated to the extent possible through pre-emptive project management action.
The Auditor-General's focus upon contract management rather than Project Management is probably a natural result of the move from the capability and evidence-based management system witnessed by Air Cdre Bates.

The second report, tabled by the Auditor-General on 24th November 2009, was “ ...designed to provide improved transparency and public accountability for these (15) major acquisition projects through the presentation of clear and consistent information on the status of projects”.
This report marks about a decade of DMO management following processes based upon inadequate domain expertise, and following executive governance/commercial management processes rather than capability-based project management systems.

The key question that arises is whether the changes in management that were introduced in 1998-2001 have been successful or not, following a decade of evolution and application. The answer to that question has been sought through the analysis of the fifteen major projects covered by DMO MPR 2008 – 09 that follows, and the individual project analyses included as Annex A.

3.ANALYSIS OF DMO MPR 2008 - 09

THE AUDITOR-GENERAL’S REVIEW


In the analysis that follows, references to DMO MPR 2008-09 are identified by the document page number, for example (Page XX), or to tables in specified sections of the report.

In his Independent Review Report at Page 133 of the MPR, the Auditor-General recorded his inability to assess the information contained in both:
  • Major Challenges in Table 1.2 and Table 4.1 (Major Project Risks) and Table 4.2 (Major Project Issues).

  • Future dates that are 'forecast' regarding a project's expected achievement of delivery schedules and capability where included in Sections 3 and 4 of each PDS.
The Auditor-General went on to say:

“DMO's systems and processes are not sufficiently mature without considerable additional examination by the Australian National Audit Office (ANAO) to provide assurance in relation to the completeness and accuracy of the above mentioned information. In view of these factors, it was not feasible to obtain an appropriate level of assurance for the purpose of this review in respect of the information presented”.

The Auditor-General's Review can only be seen as strongly qualified, due to the lack of consistent project status data; data that should flow directly from any robust project engineering data base at little, if any, cost. The Government and the Australian Parliament might also ask why such a highly qualified review by the Auditor General for Australia has been buried so deeply within the report and, more importantly, why such core deficiencies remain after a decade of DMO management.

DETAILED COMMENTS ON THE BODY OF THE REPORT

General

The central aim of the MPR is given as 'improving transparency and public accountability through the presentation of transparent, clear and consistent information on the status of projects'.

However, Defence/DMO have been unable to provide this in either of the reports issued to date.

It should be recalled that DMO's tasks were undertaken previously by the three Service Departments without, as a general rule, encountering the serious capability and schedule problems and cost overruns that the DMO has encountered over the past decade. RAAF, in particular, pre the DRP/CSP, specified, evaluated, selected, procured and usually introduced all of its aircraft and supporting systems and facilities to capability, schedule and cost with little fuss or bother, while ensuring also that all lines and levels of maintenance, in both the Service and Industry, were in place by the time the prime equipment arrived in Australia - a benchmark far more appropriate than the failed US and UK acquisition systems, and inappropriate civil construction contracts, currently used by the DMO.

The RAAF certainly struck problems and had to review frequently its new project management policies and procedures, and modify them in line with changing circumstances and technologies. However, it was able to do this from the professional operational and engineering competencies that had been built up over some 70 years.

Dr Alan Stephens, in his centennial history of the RAAF (5), noted that the appropriateness and effectiveness of the RAAF's functional organisation were tested severely by the continuing operational demands and technological challenges that followed WWII. The F-111 saga proved to be particularly testing. The weight of the political implications, as well as the extremely complex engineering problems associated with the aircraft, were carried jointly by the Minister for Air (Malcolm Fraser) and the RAAF's Air Member for Technical Services (AVM E. Hey). As recorded by Dr Alan Stephens:

“That highly successful outcome said a great deal about the RAAF. Malcolm Fraser's strong performance in his negotiations with Secretary Laird rested essentially on the Air Force's profound technical expertise, and that expertise had not been acquired by chance. In the first instance it was attributable to the far-sighted men who in 1948 established a Technical Branch with a core of tertiary qualified engineers. And in this instance it also owed a great deal to the astute leadership of Ernie Hey, who held his nerve in hard times and ensured that the RAAF received the right answers by personally selecting the Branch's best and brightest to manage the F-111 program. Consequently, when Australia's Defence Minister argued his case with the Americans, he did so from a position of authority.”

This situation persisted until the Services were starved for funds after the Whitlam Government's reorganisation of the Defence Departments in 1972, followed by the imposition of the Defence Reform and Commercial Support Programs which led to the extreme downsizing and de-skilling of the technology-based Services, the RAAF and RAN. The Sanderson Report of 1989, which led to the disbandment of the RAAF's Technical Branch, was the final straw. The current inability of Australia to argue its case in regard to the JSF from a position of authority may be traced to these changes. Further reading on the importance of the professional competencies that once existed in the RAAF is included at references 6 and 7.

Summary, Table 2 – Project Complexity (Page 17).

Much is still being made of the complexity of DMO's projects, and the effects of complexity upon schedule, but no lessons appear to have been learned on how best to manage complexity. Categories of complexity are still being measured by DMO in terms of their 'project and schedule management complexity', and their 'technical difficulty'.

Schedule, however, forms but one of the three major elements of project management, and the solution to DMO's perceived problems with project and schedule complexity, as well as technical difficulty, lie wholly, as they always have and will, in having in place:
  • robust project management systems, supported by
  • sound, capability and evidence-based, operational and engineering competencies in the technologies to be managed.
These two functions must be in place and effective in all projects, major or minor, as they are the drivers of contracts, not the slaves. A reading of the MPR shows clearly that the DMO fails to identify let alone recognise the role and importance of these functions or their proper relationship with the contract management function.

DMO's appeal to complexity seems to be more of a universal excuse than the result of any reasoned and objective analysis of specific project problems. Although 'complexity' is now seemingly accepted uncritically by Government and governance mechanisms, with sound project and engineering management complexity would soon come under control and cease to be the ubiquitous ogre painted by DMO.

ANAO records DMO's position as “The more complex the project, the greater the risk in delivering within budget, on schedule and to the required capability. DMO's experience supports the view that the more developmental in nature a project, the more susceptible a project is to schedule delays compared to MOTS (Military-off-the-Shelf) solutions.”

This elemental statement highlights a fundamental management misconception. There can be no comparison between inherently complex projects and MOTS solutions unless the MOTS solution meets Australia's requirements fully in capability, schedule and cost, in which case the MOTS solution should have been considered in the first place. It would be most unwise, as implied, to favour MOTS solutions to meet Service requirements simply because of DMO's inability to manage projects, especially complex ones. MOTS solutions will often result in a progressive deterioration in a country's military capabilities by forcing seemingly unimportant changes in force structure, operational concepts, support policies and the Defence Industry base. The end result is to be prepared to fight someone else's threats in someone else's threat environment and geography, not one's own. This is a classic ‘tail wagging the dog’ situation. The only remedy is to recover lost project and engineering management and technical competencies, and develop an acquisition organisation that can handle complexity as a matter of course.

Summary, Projects’ Performance (Page 19).

Maintaining schedule is seen by DMO as the major challenge to both DMO and Industry, and so assumes primary DMO focus. This gives the impression that schedule may be regarded as an independent element that can be managed separately from capability and cost. However, project management processes are designed specifically to pull capability, schedule and cost together and manage them in an integrated and iterative manner so that all three objectives of the project are met optimally. The Project Management Plan and its sub-plans are designed to ensure that the three elements are managed in harmony, and any attempt to give priority to one element will normally impact the other elements adversely.

In real terms, cost and schedule are transitory elements of a project, while capability, the core purpose of any project, is enduring.

Finally, DMO's averaging or summing schedule over-runs is largely meaningless, and potentially misleading. Each project over-run has its unique impact upon particular area(s) of Australia's force structure and military capabilities, and hence our national security. The sum of any number of schedule over-runs must be dimensioned not in DMO's simple, numerical terms but in their individual and combined effect on the force structure required to underpin Australia's defence capabilities and security – an exercise that Defence/DMO seems reluctant or unable to do.

Summary, Schedule Performance (Page 30).

As highlighted earlier, concentrating upon schedule will, invariably, impact capability and cost adversely . DMO's simplistic reasons given for schedule slippages, such as 'technical factors such as design problems, difficulties in integrating different systems...or emergent work associated with upgrades.' are disingenuous and inadequate.

Such generic factors have to be qualified and quantified on a project by project basis to be of any use, and then managed properly within a strong, capability-based, project management framework. There should be few surprises of this type in a well-managed project, and the DMO should, by now, after a decade, be well past the generic risk/problems/issues stage.

Summary, Capability Performance (Page 33).

The JCPAA's continuing interest in data that shows clearly each project's progress towards delivering on the key measures of equipment capability identifies a core problem. Put simply, if capability requirements are not met, then the project will generally be a waste of time, money and effort and will leave a gap in Australia's force structure, military capabilities, and ultimately national security. In addition, projects which fail due to poor capability definition and/or poor project management, incur a high 'opportunity cost' by absorbing resources which could be put to better use.

However, the proper specification and management of capability within the Project Office demands a solid basis of skills and competencies in the technology involved. Such skills and competencies then need to function within robust project and engineering management systems so as to bring together and control system capability, schedule and cost in a coordinated way. Only the adoption of sound governance of a robust management structure can provide the timely, accurate and consistent status reports, across all projects, sought by ANAO and JPCAA. Analysis of MPR 2007-08 and 2008-09 shows clearly that such systems are not in place within the DMO, but have been resisted by Defence and the DMO for more than a decade.

Summary Acquisition Governance Issues Arising from the Review (P35).

Three areas of concern have been highlighted by ANAO:
  • The nature of the information in the MPR.
  • The maturity and reliability of the systems available for its production.
  • The resources available for the review.
In particular, ANAO has had to exclude some information, such as risk identification that relates to events/circumstances that may or may not occur. This problem, in the past, now, and into the future lies in DMO's approach to the management of risk. Within a competent Project and Engineering Management framework, that is, one that treats project and engineering elements of the project in an objective manner, risk will be qualitatively and quantitatively assessed in objective terms by system and sub-system and in relation to the project as part of established procedures. Risk reports are then generated on a continuing basis so that project status (in terms of capability, schedule and cost) is visible, and risk mitigation measures are identified early, clearly, objectively and, most importantly, implemented.

Within DMO, this approach is missing. For example, with the risk management General Guidance contained in the DMO document titled 'Liability Risk Assessment For (Insert Details)', issued by the Office of the Special Counsel, Procurement and Contracting Branch. This defines 'risk' as “The possibility of an event occurring” , and the word 'liability' is identified as “the obligation to pay for, or to pay to rectify the consequences, of the event when it occurs”. The guidance then includes DMO's standard risk management processes, emphasising the need for the treatment of 'extreme' and 'high' levels of risk to be cost effective, and 'low' and 'medium' levels of risk to employ analysis, intuition and experience in reconciling them.

This approach cuts across accepted project and engineering management functions and accountabilities, and suggests that contract liability is the primary focus and driver of risk management within DMO. The guidelines attempt to control contract liability rather than the (usually project/technical) risks that arise throughout projects. DMO's risk management thus appears as a contract/legal-centric process, rather than a project-centric management procedure.

Such confusion and intrusion only adds to project risk, prevents the timely identification of the risk, and may explain partly the continued 'dumbing down' of internationally-accepted risk management standards and procedures within DMO – in effect, by replacing objective, outcome-focussed management with subjective, inwards-looking process.

In highlighting the need for reviewing DMO's project management and reporting frameworks and the number of different frameworks in place that contribute to (or actually impede) good governance of the acquisition process, ANAO identifies a core problem also seen in this analysis. As cited in the MPR, DMO's governance over acquisition processes is:

“guided by policies and procedures that are regularly updated to support developments in project management and DMO's own experience. Projects are assigned to project offices that have the responsibility to manage the acquisition process. A range of systems and processes provide support to the project offices and allow for centralised reporting of key project information”.

This statement raises very important points that beg analysis. There are three main reasons behind Defence /DMO's being unable to manage major projects, or indeed any military capabilities, competently. These are:
  • Firstly, the abandonment of established, capability and evidence-based, project management systems and procedures in favour of executive direction and quasi-commercial processes over the period 1998 to 2001.(6) While some early DMO projects started out promisingly by following established and standard management systems, those systems and procedures found little understanding or patience within Defence, or its re-invented acquisition bureaucracy, the DMO. The result was an arbitrary abandonment of professional standards, and with them good governance, a core problem that remains to this day.
  • Secondly, this situation was aggravated by the erosion of professional Service competencies resulting from the Defence Reform Program (DRP) and the Commercial Support Program (CSP), under which:
  • The Services were downsized and deskilled, lost their Engineering (as well as other professional) Branches and their Support Commands, and a General List of officers was introduced. Together, these changes destroyed, in the case of the RAAF, some seventy years of hard won and very successful engineering and project management expertise.
  • Professional military competencies were then replaced by generalists at the higher levels of the Services and Defence/DMO who possessed wholly inadequate operational expertise, management skills, and engineering competencies.
  • Finally, the post-DRP higher Defence machinery required Service members to work in a subservient role within a Defence/DMO bureaucracy, a move which ignored the functional incompatibilities that exist between these groups. The interface between the military and the bureaucracy was analysed at (7).
In short, where professional management of the military, and the technology it operates and sustains, meets a civilian bureaucracy lacking in operational, technological and project management competencies, a major fault line must inevitably develop that will work against the timely and effective management of military capabilities. Such a situation soon becomes a fertile breeding ground for a culture of 'who is right' rather than 'what is right', a predominance of form over substance and objectivity, and the hubris of internal politics that distorts military reality. The root cause for this is analysed in more detail at Annex B.

In the absence of professional project and engineering management systems founded upon good governance, all policies, processes and reviews that have developed since Defence's arbitrary change in methodology about a decade ago have only reinforced the deeply flawed processes adopted by Defence/DMO, both organisations consistently resisting all criticism and pressures to return to a professional, competencies-based management structure. There is, thus, little wonder that ANAO and JPCAA are so concerned with Defence/DMO management frameworks and the lack of assurance on the project status data that is being provided.

At the heart of this is the practice of allowing project offices to follow a range of different acquisition systems and processes. In the absence of an overarching project management discipline, this works against accurate, timely and repeatable project management information and data being available across projects. On the other hand, project and systems management methodologies have been used internationally across all manner of projects - simply tailored in scope and depth to meet the unique requirements of each project.

Until DMO's projects come under sound, capability-based, project/engineering management, supported by the operational and technical skills and competencies required, and with contract management taking its proper role and place in the acquisition process, the problems being encountered by ANAO/JPCAA will persist.

Summary, DMO’s Enterprise Risk Management Framework (ERMF) (Page 37).

The management of risk has presented a continuing problem for Defence/DMO, as indicated previously, because of a focus on risk in terms of contract liability rather than on project/engineering/technological risks that have to be identified, assessed, analysed and mitigated within those disciplines before their likely impact upon contract liability can be estimated with any confidence, and then mitigated.

DMO's ERMF fails to recognise the basic tenets of modern day risk management, such as:

  • Risk Identification:
    Risks should be clearly, concisely and accurately identified and described discretely. The description of an identified risk should not include references to the likelihood of or the consequence/s arising from the risk materialising. The use of such terms as ‘risk’ or ‘possibility’ or ‘likely’ should be avoided in the description of an identified risk.

  • Risk Assessment:
    The likelihood of Risk Arising x Consequence/s if Risk Arises = Level of Risk-laden

Levels of risk that are at the ‘Extreme’ or ‘High’ levels are not to be trifled with, since, if left untreated, they will assure that the activity in which such levels of risk reside will fail, and likely in a catastrophic way.

  • Analysis of Assessed Level of Risk:
    'There is no gain without pain, no advance without risk'.
    'Risks, properly managed, equal opportunities'.
    Extreme Levels of Risk, by their nature, require a halt to the related or effected activities until the level of risk can be reduced to an acceptable level, or a different direction taken that reduces or removes the level of risk. To do otherwise, assures that the risk will materialise and failure will be the outcome.

  • Risk Treatment:
    Risk treatment is an activity that should embody active and pre-emptive tasks that are defined and have measurable effects and outcomes.

    Monitoring or just watching is NOT a valid risk treatment.

Even a casual browse through the DMO Project Data Sheets reveals anomalies and inconsistencies with project management norms and common sense, as well as standard risk management practices. Risk in relation to the 15 projects covered by MPR 2008-09 is analysed further at Annex A.

Summary, Presentation of Project Financial Information (Page 40).

Until cost becomes an integral part of an effective, capability-based, project management system, the nexus between capability, schedule and cost will be weak or non-existent. Visibility of project costs should flow directly and traceably from project management status reports. Obviously, this is not the case within the current practices or processes employed by DMO.

DMO Report, Project Longitudinal Analysis (Page 111).

“This chapter provides a longitudinal analysis that compares and contrasts performance across the 15 projects and includes an analysis of key aspects of this performance. As the number of projects included in the DMO MPR increases over time, the longitudinal analysis is expected to indicate trends and enable more systemic observations on DMO project performance”.

Firstly, there are two types of longitudinal analysis:
  • Retrospective analysis (RLA), which looks into the past and analyses what happened so that past problems and mistakes may be avoided to the extent possible in both current and future projects. This accords with sound management system information feedback practice.
  • Prospective analysis (PLA), which simply accumulates 'data’ from projects over time, and, in the DMO's case, is expected 'to indicate trends and enable more systematic observations on DMO project performance'. In short, this approach accepts that mistakes will be made, and that these may be left to accumulate until sufficient have been made so that 'more systemic observations can be made which provides a more holistic picture’. Such an approach makes no provision for identifying and correcting mistakes promptly, or ensuring that they are not made on other projects. That is, they create an historical record, but do not form part of the management information feedback loop critical to each and every project.
DMO's approach is thus based upon a methodology that continues to accept mistakes simply to populate a generic, 'lessons learned' data base. This perception is reinforced by DMO's evidence given before the JPCAA Hearing into the 2007-08 MPR on 19th March 2009:

“They (projects) are very complex. I would hope that over the next five or 10 years of this report you see a transition where things do not happen anymore or when we mitigate some of these risks”.

This approach is also reflected in DMO's comments on the Alliance contracting methodology to be used on the Air Warfare Destroyer Project, when the CEO DMO stated that 'this new contracting approach will have to be used on a number of projects before all lessons will have been learned'.

Furthermore, all longitudinal analyses depend for their accuracy and usefulness upon having project management and reporting systems in place that provide objective and well-defined data items that are common and repeatable across all projects. Only in this way is it possible to accumulate repeated observations of the same data items over time. DMO cannot meet these requirements, because:
  • each DMO project is managed differently, so that each project data base is unique. This problem was also evidenced in the 2007-08 MPR, and
  • key data items, such as capability and schedule status, are based solely upon individual project manager's subjective assessments, not upon traceable, objective data items.
In summary, DMO's prospective Project Longitudinal Analysis can do no more than continue to record the type of low value, vague and generic lessons that might, in the fullness of time and the allocation of appropriate resources, be 'learned', but even then will draw only subjective conclusions.

DMO's Project Longitudinal Analysis is more about 'reinventing the wheel, less a spoke or two each revolution', on a repetitive basis, 'dumbing down' its systems and processes as it goes. Such a process has no end and will never yield the right answers.

The principal value true RLA is that it focusses upon the root causes behind the situation being encountered TODAY, from which changes in the management systems and procedures being used may be identified, specified, implemented and evaluated. The nexus between problem, cause and correction is kept as short as possible so that the effects of problems are headed off on current and future projects. This is the approach used, to the extent possible, in this and the many other analyses provided over many years to Defence and the DMO, as well as to successive Governments and Parliament, but to no apparent effect.

COMMENTS ON DMO REPORT

DMO sees the key benefit of the MPR as 'reporting the evolving history of each project'. However, the objectives of Government. JPCAA and ANAO are, understandably, focussed upon obtaining clear, concise, accurate, consistent and auditable visibility of the status of projects against specified requirements. Such documentation is an output of those traditional project/engineering management methodologies rejected by Defence/DAO/DMO. The information sought is thus unlikely, if ever, to become available from DMO's PDSs, as DMO's contracting and project management approaches appear designed to learn principally from generic mistakes, and the majority of mistakes made to date have been of the most elementary, if often very costly, types. This was demonstrated clearly at the JPCAA Hearing into the 2007-08 DMO MPR held on 19th March 2009 (Page 11, Lessons Learned and the Time Needed to Learn Them):

Mr Georgiou: “My difficulty is just the notion of lessons learned. I have just skipped over three or four of them and I could just about write a template for 'lessons learned' that would cover most of the lessons learned that are outlined here.” Mr Georgiou was right – the lessons learned were so simplistic and subjective that even the most junior and inexperienced of project managers should have known and avoided them.

DMO's response to Mr Georgiou's observation was largely an appeal to the large, complex and systems integration tasks involved: “They are very complex. I would hope that over the next five or 10 years of this report you see a transition where things do not happen anymore or when we mitigate some of those risks.”

So, after a decade or more of these generic and ubiquitous risks, the best DMO can offer is yet another decade or so of 'learning lessons' from 'experience' in the expectation that somehow things will improve.

KEY CHANGES TO THE DMO’s BUSINESS PRACTICES

Both the JCPAA and the ANAO have suggested changes to the DMO's modus operandi to provide useful information to allow them to assess accurately the progress of major projects.

In response, the changes to DMO's business practices (Page 51) which have taken place are seen, by DMO, to have arisen from:

  • The Mortimer Report, and
  • The Strategic Reform Program.

The Mortimer Report.

“Reforms that flow from the Government's response to the Mortimer Review cover the complete life cycle of capability systems, and follow two themes”:
  • Improving commercial discipline on Defence procurement and sustainment process, and
  • Making DMO more businesslike.
The Mortimer reforms implemented are seen by DMO as ensuring that high level strategic and capability advice is provided to Government, and to increase the rigour with which projects are assessed for inclusion in the Defence Capability Plan. Changes are also aimed at strengthening the accountability of capability managers, to quote: ' Defence will implement a framework through which the Capability Managers provide greater oversight and coordination of all elements necessary to introduce a capability into service. Mechanisms (processes) will be included to ensure that DMO, the Capability Managers and the Capital Development Group work together to agree the baseline scope, risk and schedule against which delivery of equipment can be measured'.

Comment:

A key characteristic of this approach is the further diffusion of accountability. Mortimer, like so many others who recommend that DMO adopt a more 'business-like' approach, did not specify clearly what he meant or specify what specific commercial/business changes were thought necessary to achieve what ends. The two themes of the Mortimer 'reforms' are wholly commercial, that is, contract management centric, and ignore the core, capability and evidence-based, project and engineering management systems, skills and procedures that are critical to ensuring that the contract is made as low risk and successful as possible. The problems identified by ANAO arise directly from the absence of these project related systems, and will persist so long as these systems are not in place. It is not seen how the Mortimer 'reforms' can result in any improvement in DMO's ability to provide high quality advice to Government, or increase the rigour with which projects are assessed. As a result, Defence's Capability Development and Acquisition phases will continue to be inadequate, with their inevitable adverse effects on the defence and security of Australia, as well as the public purse.

'Strengthening the accountability of the Capability Managers' will also provide no improvement as the Service Chiefs are neither organised to accept such accountability nor do they have the skills and competencies base required. This has been reflected directly in the poor requirements baselines that have plagued projects, as well as the intractable problems associated with understanding the technology involved and its integration. Hence, DMO's reforms will merely become additional processes, serving only to make the current situation worse, as well as slower and more costly.

No recognition is made in the Mortimer Review of the need for capability-based project management and expertise relevant to the technologies being managed. No mention is made of the Project Management Plans, the Systems (and Integration) Management Plans, the Configuration Management Plans, the Maintenance Analysis Plans, the Risk Management Plans, the Test and Evaluation Master Plans, or the Support Analysis Plans, and so on, that form part of and drive every project. Contractors may provide input to them, but these plans cannot be outsourced to contractors without incurring the very generic 'Risks', 'Issues', and 'Lessons Learned' that populate the DMO's PDSs.

The Flight to Process

The improvements promised at Para 1.28 (Page 52) of the MPR represent wishful thinking, as they are all built upon adding processes to an already process-bound organisation. Every review into Defence/ DMO to date has led only to more process, and now DMO plans to add the following:
  • 'A Project Directive will provide top-level direction from CDF to the Service Chiefs to introduce the full operational level capability into service by the date agreed upon by Government'. Firstly, wasn't this a primary DMO function? Secondly, as stated above, the Service Chiefs are no longer able to do it; they no longer control the activities or the budgets or have the competencies needed to discharge that responsibility.
  • 'The Memorandum of Arrangements between the Secretary, CDF, and the CEO DMO, as well as Materiel Acquisition Agreements, will be reviewed in terms of responsibilities and accountabilities'. This may make the deck chairs look more orderly, and identify who sits in them, but will achieve nothing of importance.
  • 'DMO will introduce two delivery milestones – Initial and Final Materiel Release'. DMO has already departed from internationally-accepted definitions for these important milestones for no justifiable reason, and no reason has been given for departing further from them.
  • 'DMO will increase the commercial acumen of its staff by employing more commercially experienced and skilled personnel at SES level and has started recruiting an SES3 General Manager – Commercial'. The deficiencies in DMO's competence to manage major projects lie in failures in Project and Engineering Management (both recognised as being the controllers of complexity), Risk Management, and IV&V, and so it is difficult to see how this move will result in any better functional performance.
  • 'A Project Charter will also be raised for 'complex and demanding projects' detailing the authority, responsibility, and accountability of the Project Manager. This Charter will hold him to account for meeting both financial and non-financial performance targets. Charters will also apply to the System Sustainment Managers'. This again is a counter-productive (process-driven) approach to management which will have no meaningful effect so long as both Project Managers and Sustainment Managers have inadequate project and engineering management skills and competencies, and do not possess the requisite authorities and resources needed to discharge their responsibilities.
  • 'Gate Review Assurance Boards will be instituted to examine complex and demanding projects before critical points in the project's life cycle'. Unfortunately, no indication is given as to the skills and competencies that these Boards will bring to project management, or just how they will be effective, The activity is well covered by standard project management procedures and should have been in place in 2000, as recommended by Industry. The fact that a ‘Gate Process’ was not implemented on formation of the DMO indicates that the DMO has little, if any, basic project management understanding.
Comment

The practice of adding additional and higher layers of process simply passes problems up the line to people who have even less competent knowledge of the problem and its solution than those below, ensuring that judgements and decisions are taken at the higher level on grounds other than what is best for the project. Woven throughout this maze is the principle of non-acceptance of mistakes and the diffusion of accountability. Such a construct has proven to be inappropriate for the management of Australia's military capabilities and national security. Unfortunately, Australia's governance mechanisms, seemingly due to limited comprehension, or lack of political will, are not up to the task of identifying problems and turning them around.

None of the changes in process listed above will redress the effects that flow inevitably from the failure of Defence/DMO to follow standard project, engineering, risk management, and IV&V systems and procedures, or add to its ability to manage projects through contracts. The management approach reflected in the 2008-09 MPR has failed and will continue to fail until major structural and management changes are imposed.

The Question of Accountability

Several of the processes listed above, and included in the Defence Strategic Review, relate to accountability in Defence/DMO. Accountability was also a central theme in the Proust Review into Defence Management (8), for example (Pages 32, 33):
  • “...Defence had become unwieldy to manage..., as the number of groups had grown – each with its own leadership, reporting and administration overheads.”
  • “The current structure is confusing.”
  • “While it might be disruptive in the short term to attempt another reorganisation of Defence, it is necessary to get the structure right, by which we mean that the structure should be aligned with accountabilities and responsibilities.”
Then, in the Report Overview (Page vii): “We found that the organisation has confused its accountabilities, and there is a lack of alignment between responsibility and accountability in key areas of the organisation.”

The question of accountability is also raised, both directly and indirectly, in this analysis. To a great extent, accountability is an output of sound management. Put simply, sound management requires:
  • A corporate plan that defines the objectives of the organisation. (Planning)
  • An organisational structure appropriate to achieving those objectives. (Organising)
  • Directives (policies, systems, and procedures) for managing the functions of the organisation in the most efficient and effective manner. (Directing)
  • A control mechanism in the form of management feed-back loops so that the activities being undertaken are monitored and measured continually against the directives (and in turn the objectives of the organisation), and corrected promptly as and when necessary. The Control Loop is the most critical of the four core management elements. (Controlling)
In addition, three important characteristics of sound management design are:
  • Clear, accurate and concise statements of responsibility (and hence accountability).
  • The matching of authority and resources with responsibilities.
  • Ensuring short and direct lines of communication and control throughout the organisation.
However, no organisation, no matter how well designed,, can function in a general absence of ethics.

Over recent decades, sound management has not been a feature of government bureaucracies. Accountabilities have become diffused to the point of avoidance, which, when coupled with outsourcing policies that have led to a dramatic lack of expertise in the matters for which they are responsible, have resulted in departments being unable to provide sound advice to their ministers, and hence government and parliament, or to implement government policies competently. As Proust stated in her Summary of Recommendations: “Another threat to successful change is that Defence may passively and outwardly accept the Review but be less than rigorous in its implementation.” Defence shows symptoms of an organisation without a sound management structure, which means that it has no Control Loop, and so is incapable of measuring its own performance, or reforming itself.

The Strategic Reform Program

The Defence Strategic Reform Program (9) is 'a response to government seeking greater accountability transparency in the way Defence manages its budget, and the quality of advice it provides'. A need is also seen for substantial reform. However, the review will proceed on the basis of eight internal, 'Companion Reviews', all of which have been sheltered from any transparency and open debate, or the consideration of ‘evidence-based’ observations and analyses that have been provided by independent experts. Furthermore, the review will be managed by a Defence Deputy Secretary of unknown qualifications working to unknown Terms of Reference. The review will thus be driven internally and be responsive only to, and controlled by, Defence executive imperatives.

In addition, the Department has already identified exactly how much can be saved in eight specified areas of activity, which means that management in those areas will be required to trim their budgets (i.e. activities) to meet the financial objectives that have been set. Because of the maze of organisational, functional and financial interfaces that now exist within and between the Services and the Department that supposedly supports them, there is little probability that the cascade effects of these 'economies' on other areas will be recognised – the Services will just be told to live with them.

Pre-DRP/CSP, such economy drives were known as 'Money must be saved at any cost!' exercises. Fortunately, at that time, the Services were organised and resourced such that they were able to retain core capabilities, competencies and standards while trimming activities to meet the passing financial constraints imposed. Today, neither Defence nor the Services have the ability to do this, so the Strategic Review will carry an extreme risk of further debilitating Service capabilities and degrading further Australia's defence and security.

A similar strategic review was conducted in the UK (10) with disastrous results, when it was found that “In defence, since there is no measurement of output, a cut in one area often cascades throughout the operation with unseen negative consequences”. The UK has also learned that such consequences can lead directly to loss of life and equipment. This was highlighted in the subsequent loss of an RAF Nimrod aircraft and its entire crew in Afghanistan (11). Some of the findings from the very thorough review that followed were:

Organisational Trauma 1998-2006.
The MoD suffered a sustained period of deep organisational trauma over this period, beginning with the 1998 Strategic Defence Review. Financial pressures and cuts drove a cascade of multifarious organisational changes, which led to the dilution of the airworthiness regime and culture within the MoD, and distraction from safety and airworthiness issues as the top priority”.

While the RAAF may have an Airworthiness Board, Australia's problem lies in the loss of the three Services' Technical Services Branches which planned and implemented those engineering and maintenance policies, systems and procedures which ensured that the activities that underpin airworthiness / seaworthiness / battleworthiness were in place and working properly across the whole Service.

Returning to the Nimrod Review:

“There was a change in culture and priorities in the MoD towards 'business' and financial targets, at the expense of functional values such as safety and airworthiness. The Defence Logistics Organisation, in particular, came under huge pressure. Its primary focus became delivering 'change' and the 'change programme' and achieving the 'Strategic Goal' of a 20% reduction in output costs in five years and other financial savings. Airworthiness was a victim of the process started by the 1998 Strategic Review”.

Two senior personnel who presided over the Defence Logistics Organisation during the critical period were appropriately named in the UK review as bearing particular responsibility.

'The switch to financial and business targets at the expense of functional values' mirrors well the situation that has been allowed to develop in Australia and now drives the Defence/DMO organisations.

“There is no doubt that the culture of the time had switched. In the days of Sir Colin Terry you had to be on top of your airworthiness. By 2004, you had to be on top of your budget if you wanted to get ahead”. (Air Chief Marshal Sir Colin Terry, Chief Engineer, RAF, 1997 – 1999). Here we see money being used for functional control, rather than a functional resource, and then only one of many of equal or greater value.

Dilution

“The continuous organisational change during the period 2000-2006 led to a marked dilution of the safety and airworthiness regime and culture in the MoD for three reasons. First, during this period there was an inexorable change in the MoD from a safety and airworthiness culture to a 'business culture'. Second, the organisational changes in the MoD led to a safety and airworthiness regime which was organisationally complex, convoluted, confused and 'seemingly dysfunctional'. Third, meanwhile, there was also a steady dismantling of some of the important features of the safety and airworthiness regime which had previously existed: (Selected items)

Abolition of the 'Chief Engineer RAF';
Dilution of air technical support services;
Dilution of aircraft engineering skills.”

The Nimrod disaster review should have been made compulsory reading for all Defence executives and senior Service officers involved with the Defence Strategic Review.

Project Maturity Scores. (Page 55)

DMO's Project Maturity Score “quantifies the maturity of a project by way of an objective score based on the Project Manager's judgement at defined milestones in its capability development and acquisition phases. This score is then compared against an ideal or benchmark score for that milestone”.

These scores attempt to replace the milestone reports that form the backbone of sound project management, but without the auditable, objective data items that project management reports require and provide.

The soundness of DMO's Project Maturity Scores rests wholly and solely upon the Project Manager's judgement, which, by definition, can only be subjective (that is, 'based upon personal feelings, tastes, or opinions' – Oxford English Dictionary). It is impossible for any subjective assessment to become an objective measure as DMO suggests (that is, one 'not influenced by personal feelings or opinions' – Oxford English Dictionary). DMO's Maturity Scores are thus not objective scores as stated, but simply unsupported opinion, likely open to cognitive or other bias in the interests of self and/or career preservation. To then compare them with an 'ideal benchmark' is merely to invoke further magical thinking.

DMO uses US and UK Defence acquisition systems as benchmarks, but both have extremely poor records; records that have been assessed as gross failures. The US system, for example, has been the subject of trenchant criticism since the mid-1980s when Congress decided that its Department of Defense and its Defence Procurement Organisation were broken, and undertook a range of reforms under the Goldwater-Nicholls Act of 1986. Regrettably, these reforms were sabotaged by very strong vested interests, and so the problems remain even more deeply entrenched today. However, the US national security situation has now reached the point where the President's State of the Nation Address declaration “Well, I do not accept second-place for the United States of America” has a decidedly hollow ring. The final confrontation between the US Department of Defense and Congress will most probably arise in the near future as highly critical JSF Project governance reports are tabled.

That DMO should see the UK Ministry of Defence (MoD) procurement organisation as a suitable performance baseline is surprising, as that organisation has been the subject of severe criticism for repeated failures to bring capabilities in on time, capability and cost. In the sustainment area, the organisation has been similarly criticised, with the MoD being accused of causing the unnecessary deaths of many UK Service personnel.

A recent paper titled 'UK Defence Needs' (10) records the MoD acquisition organisation as having an average equipment programme over-run of five years (many going over a decade), and very high cost over-runs. Current projects are some £35bn over budget. The paper gives details and references to other substantial analyses undertaken into the UK MoD procurement organisation, to little, if any, effect.

In comparison, of the 15 DMO projects covered in the 2008-09 MPR, eight project schedules slipped by a total of 378 months: including HF Modernisation (74), Collins RCS (72), FFG Upgrade (65), Wedgetail (48), ARH Tiger (42), Armidales (33) and Bushranger (26).

If DMO is prepared to accept the US and UK procurement organisations, and some Australian civil engineering projects, as performance baselines, it is difficult to accept that the organisation could ever construct an 'ideal score'. The bar is being set far too low for the defence and security of Australia.

The development of Project Management Scores may be seen as an inevitable result of putting generalists in management positions that require professional competencies and skills in Project and Engineering management. The scores and their benchmarking give an illusion of project management, but, in reality, have the substance of vacuous smoke and mist.

An Independent Performance Assessment

A recent report released by global consultancy firm Mc Kinsey & Co (12) ranked Defence management practices across 33 Departments and Ministries around the world in three key areas: personnel, maintenance, and weapon buying. Brazil, Poland, and Russia were rated 1, 2, and 3 respectively.

The United States and Australia were found to be the lowest performing countries with regard to equipment output for every dollar spent, ranking 33 and 32 respectively. While Defence has challenged the Mc Kinsey findings, the reasons given are not persuasive that the over-all approach and relative rankings are significantly wrong.

Project Lessons Learned from the DMO MPR. (Pages 58 and 122)

In this section of the report, DMO records the many and various ways in which lessons learned are absorbed and used, but despite defining project complexity in terms of technical difficulty, this critical aspect does not rate a mention. The responses of the CEO of the DMO to 'lessons learned' are expressed wholly in contract or process terms.

Of particular interest is the statement that 'the effects from any major change to contracting practices would not be evident until a range of new projects has used the new practices and sufficient time had elapsed to assess the effectiveness of the changes'. Common sense management would suggest that if you do not know what you are doing, then do not do it! However, Defence seems content for DMO to use capability procurement on a trial and error basis, in an effort to identify what systemic changes to contracting processes might be needed sometime in the future.

The following corrective actions have been put forward as responding properly to the lessons learned from the DMO MPR, but they will merely add further layers of process and will not get to the core management problems that caused the problems in the first place:
  • Management Action Plans.
  • The Central Audit Recommendations Management System.
  • The assignment of DMO officers to reporting implementation progress.
  • The use of Councils to 'consider business improvement initiatives in response to business needs. (no mention of capability needs).
  • The Quality and Environment Management System.
  • The series of Check Lists based on lessons learned to assist staffs.
  • Modifying manuals, training courses, and processes, effectively 'dumbing' them down.
  • Regularly updating contracting templates.
In summary, DMO still focuses upon 'lessons learned' that are, in fact, only common, generic project management problems. No real lessons and their root cause(s) have been identified and corrected after ten years of DMO's purported 'lessons learned' approach.


Tiger ARH (©2007 Carlo Kopp).



  C-17A Globemaster (©2007 Carlo Kopp).



  Bushmaster PMV (©2007 Carlo Kopp).



F-35A mockup displayed at Avalon Airshow (©2007 Carlo Kopp).

4.IN RETROSPECT – THE NACC/JSF PROJECT

In retrospect, the New Air Combat Capability / Joint Strike Fighter (NACC/JSF) Project, should have been included in ANAO's DMO MPRs, or made the subject of a separate study, as this aircraft probably represents the most risk laden project ever undertaken by Australia; one which, if it proceeds as currently planned, will entrench Australia, at best, as a second-rate, minor military power both in our region and on the world scene, a nation in which coherent and competent defence planning is clearly beyond its competencies.

As this project ran into inevitable capability, schedule and cost problems over the years, we have seen the aircraft's capabilities extolled well beyond the limits of physics, credence, or even common sense. The aircraft was designed to face threats that are now more than a decade old, and by the time the aircraft enters service it will be at least two full generations behind the threats it will have to face.

However, successive Parliaments, Governments, Defence Ministers and the DMO have chosen to ignore, even denigrate, the substantial operational and technical criticism of both the aircraft and the way in which it is being mismanaged that has been provided by local expert analysts as well a series of highly critical US governance reports.

In brief, the aircraft to which Australia has mortgaged its military future, and that of many of its allies, is: (13)
  • Not a competitively stealthy aircraft,
  • Not a 5th Generation aircraft.
  • Not survivable against current referenced threats.
  • Not a lethal aircraft system in face of these threats.
  • Not an affordable aircraft system.
  • Not a supportable aircraft system.
Defence/DMO should measure their enthusiastic, 'expert' assessments of the JSF against the 2009 report of the US Director Operational Test and Evaluation into the F-35 Lightning Joint Strike Fighter (JSF) (14), as well as the GAO testimony given before the Committee on Armed Services, U.S. Senate that was released on 11th March 2010 (15), as well as a string of previous GAO and SAR (Selected Acquisition) Reports, and then review the warm and unqualified assurances that have been given to Government, Parliament and the Australian people.

That Defence should continue its unqualified support for the JSF, and recommend the purchase of 14 aircraft from the initial production run, in the face of official US Government warnings on project cost, schedule and capability, indicates poor or absent due diligence on the part of Defence/DMO, and an equally poor or absent governance oversight.

The JSF Project thus begs an urgent and independent management audit.

5.RECENT DEVELOPMENTS IN DEFENCE CAPABILITY AND ACQUISITION MANAGEMENT


To a large extent, Australia's Defence/DMO organisations follow policies, structures and processes similar to those in use in the UK and USA, and use those countries as capability procurement benchmarks. However, recent developments in both countries have further emphasised the reasons behind the failure of the defence bureaucracies in all three countries to manage capability requirements and procurement matters effectively, efficiently, timely and economically.

The UK Experience

Following its disastrous 1998 Strategic Defence Review, the UK, under severe financial pressure, has now embarked upon another Strategic Defence and Security Review which is due to report during October 2010. This, as for all such reviews, carries both an opportunity to correct what is wrong with the management of UK defence, and the risk that the Review will only make matters worse.

Defence Secretary Liam Fox has announced that Service chiefs are to be given more control over the armed services as part of a package designed to 'decentralise' the Ministry of Defence. This would provide a much-needed strengthening of the functional backbone of the Services, if done properly, but there is a definite fear that the review will merely entrench further problems because it will be far too rushed and is seen to be 'Treasury-led'. There is thus a high risk that the review will turn out to be just another cost-cutting exercise, not one based upon clear, functional military needs, and will simply inject yet another cascade of multifarious organisational changes and processes that will work against professional military capabilities, competence and morale, and only further erode UK and NATO security.

The US Experience

There have been three major recent developments that will impact Australia directly:
  • The release of the 2010 US Quadrennial Defense Review (QDR) plus, and most importantly, the Final Report of the Independent Panel's Review of the QDR.
  • Secretary Gates has embarked upon a reform program as a result of financial pressures similar to those behind the UK Defence Review.
  • The issue of Government Accountability Office Report (GAO 10-789) covering Tactical Aircraft Capitalisation.
During the mid-1980s, the US Congress decided that its Department of Defense and its Defense Procurement Organisation were broken and a range of reforms were undertaken under the Goldwater-Nicholls Act of 1986. However, Defence, political, lobbyist, and Defence Industry vested interests combined to block all meaningful change. As a result, the problems that prompted the Goldwater-Nicholls Act have persisted and festered to this day. The Independent Panel's Review of the 2010 QDR throws important light upon the reasons behind the long decline in the performance of the US DoD and its procurement organisation, as well as its continued inability to reform itself:

“The modern QDR originated in 1990, conducting a bottom-up review to free the Department from the constraints of existing assumptions and refresh the intellectual capital of the political leadership in Congress as well as the Executive branch.”

“The initial Bottom-up Review was considered a success. Unfortunately, once the idea became statutory, it became part of the bureaucratic regime. The natural tendency of bureaucracy is to plan short term, operate from the top down, think within existing parameters, and affirm the correctness of existing plans and program of record.”

“That is exactly what happened to the QDR process. Instead of unconstrained, long term analysis by planners who were encouraged to challenge pre-existing thinking, the QDRs became explanations and justifications, often with marginal changes, of established decisions and plans.”

“This latest QDR continues the trend of the last 15 years...but for the reasons stated, it is not the kind of long-term planning document which the statute envisions.”

“The fourth chapter of our Report takes on the issue of acquisition reform. We commend Secretary Gates for his emphasis on reducing both the cost of new programs and the time it takes to develop them. But we are concerned that the typical direction of past reforms – increasing the process involved in making procurement decisions – may detract from the clear authority and accountability that alone can reduce cost and increase efficiency.”

“The issues raised in the body of this Report are sufficiently serious that we believe an explicit warning is appropriate. The aging of the inventories and equipment used by the services, the decline in the size of the Navy, escalating personnel entitlements, overhead and procurement costs, and the growing stress on the force means that a train wreck is coming in the areas of personnel, acquisition, and force structure.”

“The potential consequences for the United States of a 'business as usual' attitude towards the concerns in this Report are not acceptable. We are confident that the trend lines can be reversed, but it will require an ongoing, bipartisan concentration of political will in support of decisive action.”

The Independent Panel also found:

On the Core Issue: The Panel saw the core issue as follows:

“The Panel's examination of the QDR found that the Department of Defense has been in a near constant state of acquisition reform since the implementation of Goldwater-Nicholls in 1986. These efforts have generally produced more structure and more process, but have not produced notable improvement in delivering required capabilities when needed at the expected cost.”

Improving Acquisition Performance: The Panel saw the critical path to improving acquisition performance as requiring:

“....replacing the current diffused, fragmented assignment of responsibilities without accountability, with authority and accountability vested in identified, authoritative individuals in line management.” To which should be added: 'possessing the appropriate project and engineering management competencies'.

On Contract Management. The Panel highlighted:

“The key to effective execution of any contract is not the quality of the contract, it is the quality of the program management responding to clear assignment of authority and accountability for each program”.

The Independent Panel's Report on the 2010 QDR raises many matters that demand urgent and effective action if the US is to 'keep up its global leadership role', as stated by the President, and so be capable of providing for the security of its allies in accordance with its many treaties, including that with Australia. However, there is little, if any, evidence that any of those responsible, either in Government, the Defense Executive, or industry, are prepared to undertake the major structural and accountability changes required.

The Gates' Reforms

Secretary of Defense Gates' reform program is supposedly aimed at fixing what has been chronically wrong with Defence and its procurement organisation for two decades or more. However, his Marine Corps Force Restructure merely nibbles around the margin, and his other reforms are aimed primarily at reducing costs by eliminating 6,324 military, civilian and contractor positions and abolishing the Joint Force Headquarters. At the same time, the DoD is adding 20,000 acquisition workers, comprising new staff or converting contracted workers to in-house workers. He has studiously avoided the hard decisions on the structural and management changes recommended by the Independent Panel, so nothing of substance can be achieved – it will be, once again, 'business as usual', despite the QDR Panel's dire warning, and will only result in an unfair load having to be borne once again by a downsized, overstressed and de-skilled military.

GAO Report 10-789.

Report 10-789 highlights the continuing erosion of the US's tactical air force capability which will soon drop (if it hasn't already dropped) below the required levels, and warns that capability shortfalls will persist for years because of continuing financial constraints.

This situation has arisen against a background where there has been no comprehensive, long-term planning by the US DoD to identify the evolving threat environment, specify the force structure needed and the capabilities required to meet identified gaps, and enable resources to be allocated in accordance with clearly-identified priorities. All recent research in this area has been funded privately and solely by think tanks, not by DoD.

In the absence of any long-term planning input, both the QDR and the GAO Report findings and recommendations, as well as DoD's Force Structure and Procurement Plans and priorities, must carry a high level of uncertainty, and thus risk to the security of both the US and its allies.

QDR and Independent Review Panel Lessons for Australia

The QDR, and particularly the independent review, contain many observations that apply directly to Australia, particularly:
  • Defence bureaucracies will inevitably plan short term, operate from the top down, think within existing parameters, and always affirm the correctness of existing plans and programs. They will also resist long-term analysis as well as any challenges to their thinking, and focus upon self-justification.
  • Instead of real capability and acquisition management reform, they will merely increase the number and levels of the sub-processes comprising their decision-taking process, the intent being to further diffuse responsibility for inevitable shortcomings in performance.
  • Despite what happens, 'business as usual' will prevail under increasing layers of process, and the primary objective of the organisation will continue to be to increase whenever possible the power of the bureaucracy.
Australia also needs to recognise the brittleness in relying too heavily upon foreign contractors for the preparedness and sustainment of our defence capabilities under any and all circumstances which, from experience, can arise in new forms and at very short notice. A similar high risk is inherent in having to rely too heavily upon security treaties with nations that may be reluctant or unable to provide support as and when it is needed, particularly if we are seen not to be pulling our independent weight.

There is only one effective and reliable form of insurance against such risks, and that is the establishment and maintenance of technologically-skilled and highly professional military services, supported by a robust and flexible defence industry. Neither now exists, while Defence/DMO policies continue to erode what little remains.

Government and governance bodies are must consider carefully the analysis of DMO MPR 2008-09 contained in this paper, as well as the wider matters raised, as the primary risk to Australia's long-term security has now moved from external threats to the cumulative effects of Defence/DMO policies and processes.

As the QDR Independent Panel's Report so correctly observes: “We are confident that the trend lines can be reversed, but it will require an ongoing, bipartisan concentration of political will in support of decisive action.”

6.SUMMARY


For over a decade, the performance of Australia's Defence Department and its Capability Acquisition Organisation have been subject to criticism politically, in the Press, and by defence analysts, for poorly selected, late, capability deficient and failed major projects that has resulted in a distorted and deficient force capability and the waste of billions of dollars.

While the root cause may be traced to the widespread reorganisation, downsizing and de-skilling of the Services, particularly the disbandment of their Technical Services Branches and the loss of their Support Commands, the acquisition organisation's move from a capability and evidence-based project management methodology to commercial/business-orientated processes greatly aggravated the effects of the loss of core competencies.

After some seven years of acquisition problems, The Auditor-General was called upon to produce an annual report on major Defence projects. The second report in this series, which covered 15 major projects, was presented in November 2009, but was strongly qualified on the grounds that it was not possible to obtain an appropriate level of assurance in the information presented by DMO.

Analysis of this report indicates that there is little, if any, evidence that long-standing, critical deficiencies in the project, engineering, risk management, and Independent Verification and Validation areas have been recognised or corrected. In brief:

  • The proper management of risk still eludes DMO, with the organisation still focussed upon contract management and 'business'/commercial approaches rather than managing the technical and project risks that usually arise, a result of the switch to financial/business targets at the expense of functional values.
  • 'Lessons Learned' remain simplistically generic and fail to identify and correct the underlying causes.
  • The measurement of of the critical performance elements of capability and schedule rest solely upon Project Maturity Scores which have no objective basis.
  • The many changes made within DMO over the years have led solely to the addition of processes and further levels of executive oversight, rather than acquiring much-needed technical skills and project management competencies.
  • Problems will thus remain in obtaining the traceable, repeatable and objective project status data sought by the Auditor-General for Parliament, and required for the effective management of DMO major projects.
Defence Industry development also needs to be included in Defence/DMO plan, programs and reporting to prevent the continuing erosion of Australia's self-reliance.

In addition, organisational fault lines have been identified within the Defence organisation, identical to those now identified in the US, which make the organisation resistant to change, and dictate against Defence and DMO becoming efficient and effective organisations. There is thus a need to reform DMO in harmony with reform of the higher Defence organisation.

In the end, “it all comes down to a tale of war – a war not run by generals, but by comfortable men in air-conditioned offices who are indifferent to human suffering”.(16) Today, under civilian rather than civil control, Australia's military is run, at all levels, by well-entrenched, comfortable men in their air-conditioned offices.

This analysis aims to develop an acceptance of the need for urgent reform of Defence/DMO by restructuring the Defence and Service organisations, re-valuing (re-skilling) the Services, and re-imposing sound management and governance within and over Defence/DMO functions, realigning accountability with responsibilities and resource allocation.

However, the scope and the nature of the challenges identified in this analysis will call for strong, focussed and persistent cross-factional action, as national interests will have to be put before political considerations and executive pressures if Australia is to recover the professional military capabilities and competencies needed to face current and emerging threats.

References

  1. Australian National Audit Office (ANAO) Defence Materiel Organisation Major Projects Report (DMO MPR) 2007-08, dated 27th November 2008.
  2. ANAO DMO MPR, 2008-09, dated 24th November 2009.
  3. The Defence Materiel Organisation (DMO) was formed on 1st July 2000 from an amalgamation of the Defence Acquisition Organisation (DAO) and Support Command Australia (itself an amalgamation of the three Services’ Support Commands). DMO was formed as a Prescribed Agency responsible for the management of new capabilities and capability sustainment (in-service support), reporting to the Minister for Defence for performance and finances, and accountable to the Secretary under the Public Service Act.
  4. Air Cdre Garry Bates (Retd). See also Bates Garry, ‘What a Difference a Decade Makes: Risking the Sustainment Capability of the Air Force’, Australian Defence Association Magazine ‘Defender’, Autumn 2007.
  5. Alan Stephens, ‘The Royal Australian Air Force’, The Australian Centenary History, Volume II, Chapter 15-Defence Self-Reliance.
  6. E.J. Bushell, R.G. Green, B.J. Graf, ‘The Decline in the Management of Defence, and Defence Capability Development, Acquisition, Preparedness and Sustainment’, Air Power Australia Analysis 2009 – 03, 5th September 2009.
  7. E.J. Bushell, ‘Rebuilding the Warrior Ethos’, Air Power Australia Analysis 2008-10, 27th December 2008.
  8. Report on the Defence Management Review (The Proust Report), 30th March 2007.
  9. The Strategic Reform Program 2009 – Delivering Force 2030.
  10. Air Commodore Andrew Lambert and Allen Sykes, ‘UK Defence Needs’, published by the United Kingdom National Defence Association, February 2009.
  11. Charles Haddon-Cave QC, ‘The Nimrod Review’, 28 October 2009, ISBN: 9780102962659.
  12. Mc Kinsey Report, John T, Bennett, ‘U.S. Last in Combat Gear Output Per Spent Dollar’, 15 March 2010.
  13. ‘F-35 JSF Program: Assessment of Top Level Programmatic Risks’, Airpower Australia NOTAM 19th November 2009.
  14. DOTE-F-35-JSF-2009-Annual Report.pdf.
  15. GAO Report GAO-10-520T. The status of the JSF Project was more recently analysed by GAO in its report GAO-10-382, 19th March 2010.
  16. Gordon Thomas, 'Gideon's Spies', Thomas Dunne Books, 2009, Page 190. This paraphrase comes from ex-Israeli Intelligence Officer Ari Ben-Menashe's book 'Profits of War' which recounts the machinations of intelligence agencies driving government policies during the 1980s. He described his book as: “a tale of government by cabal – how a handful of people in a few intelligence agencies determined the policies of their governments, secretly ran enormous operations without public accountability, abused power and public trust, lied, manipulated the media and deceived the public. Last, but not least, it is a tale of war - a war run not by generals, but by comfortable men in air-conditioned offices who are indifferent to human suffering”. Menashe's observations highlight well the range and type of dangers that can be expected to arise when government departments have authority without accountability, and governance mechanisms are not functioning as they should.
LIST OF ANNEXES:

A. Analysis of Project Data Sheets

B. Evolution of Management Approaches.

ANNEX A: ANALYSIS OF PROJECT DATA SHEETS


Analysis of the Project Data Sheets for each of the fifteen projects covered by DMO MPR 2008 – 09 follows.

(The data contained in the following Major Project Analyses, and many of the points made in the body of this analysis, were contained in the author’s submission to the Australian National Audit Office and the Joint Committee Parliamentary Accounts and Audit in relation to DMO MPR 2008 – 09.)

Attachment 1 to this analysis provides a summary assessment of the consequences of the risks met, or likely to be met, in regard to each project in terms of both DMO and Australian/International Standards.

AIR WARFARE DESTROYER BUILD

General.
  • Operational analysis of longer-term threats suggests that the project’s capability expectations are overly ambitious when emerging regional air power and anti-shipping missile (supersonic cruise and ballistic) capabilities are factored in.
  • The solution seen by Defence/DMO as ensuring that this project avoids the fate of earlier major projects has been the establishment of yet another layer of bureaucratic review – the establishment of an Air Warfare Destroyer Alliance Principals’ Council.

Summary Comment on 2008-09 MPR:

The Air Warfare Destroyer Project is in extreme danger of failing for the following reasons:

  • The decision by Defence/DAO/DMO to abandon due diligence in the management of major projects, which led to proven Project/Engineering/Risk/IV&V Management systems and procedures being avoided, ‘dumbed down’, and made subservient to contract management, left Defence/DMO open to pressures from manufacturers and to subjective decisions taken at ministerial and departmental levels. This is a generic problem common to all projects now managed by Defence/DMO.

  • The adoption of a new contracting methodology that has not been proven to be appropriate for high technology military systems. This will introduce yet another data base from which DMO and ANAO will have to try to retrieve meaningful project status data.

  • The project’s critical dependency upon developing and commissioning a new dockyard from a ‘green site’ to meet project milestones.

  • The absence of an agreed Certification Basis and agreed Certification Plan at the time of contract signature, without which no project should proceed to contract signature.

  • The reliance upon a higher review organisation lacking in required competencies.

Comment on PDS Content.

Although listed as an Acquisition Category 1 project, which means that it will require extensive project and schedule management, and have very high levels of technical difficulty, there is no mention of just how these challenges will be managed, resolved, and reported in an objective, accurate, timely and consistent manner.

Defence/DMO has chosen to trial an Alliance-based contracting methodology for this project on the basis of its having been used successfully on a museum and a highway project. Just how this approach is seen to be appropriate for a high-technology naval capability, with its unique systems engineering integration challenges, is not explained. The additional organisational and functional interfaces that will have to be negotiated must only add to project risk

Alliance contracting has become a popular funding model for government infrastructure projects, aiming to share the risk of cost over-runs between government (the taxpayer) and the private sector. In practice it seems to simply financial risk back to the taxpayer. A recently-commissioned Victorian Treasury study found that so-called alliance contracting costs blew out between 48% and 55%, the highest of the three funding models studied.

This finding is not surprising, as any contract, particularly those involving high technology systems, are prone to cost over-runs if the customer does not have the required project and engineering management competencies appropriate to the technology involved. Governments, and DMO, fall into this bracket, as they focus upon the wrong risks. Not having the project and engineering competencies required, they fail to manage properly the risks associated with capability, schedule and cost. They concentrate upon the risk (liability) to the contract without understanding that risks are embedded mainly in the technical (capabilities) and schedule areas.

This new contracting approach will, as stated by DMO, ‘have to be used on a number of projects before all lessons will have been learned’, so will be in a trial mode. This approach must be assessed as only adding new and unidentified risk to those projects upon which it will be used.

Within the complex management structure devised for this project, it is difficult to identify the project and other management skills and competencies that DMO will bring to the table.

Major Challenges.

The six major challenges listed are so elementary as to be hardly worth mentioning. DMO should, at this stage of the project, and after some ten years of experience, be able to better qualify and quantify those major areas requiring project, engineering and risk management focus.

Schedule Performance.

DMO states: “Progress to achievement of planned in-service dates for the three ships and their support system is as scheduled”.

Capability Performance.

DMO states: “The current status is that planned capability will be achieved”.

Both schedule and capability assessments must be considered as being prematurely optimistic, subjective, and of little value, particularly in view of the significant, if unqualified and unquantified, problems lurking under ‘Major Project Risks’.

Major Risks, Issues and Linked Projects.

The items included in this section are too broad and are seen primarily from a contract aspect. Most are merely generic risks, lacking in capability, schedule and cost focus. The project would be difficult enough to manage if it were built in an experienced, working shipyard, but to expect a new and competent shipyard becoming available from scratch to meet project milestones, is beyond optimism. The skills and competencies that must be put in place have been greatly underestimated.

Finally, any project that ‘does not have an agreed Project Certification Plan and Certification Basis’ should not proceed as it embeds certain, if indefinable, risks. The absence of such basic pre-requisites for executing a minor, let alone major capital equipment procurement contract was the root cause of the SEA 1411 Super Sea Sprite debacle, the Collins Class Submarine misadventures and the continuing challenges within the Wedgetail AEW&C Project, to name but a few.

Have these lessons been learned? Clearly not.

Key Lessons Learned.

While hardly classifiable as a ‘lesson’, this section is important in that it warns “that it takes time and effort to develop the culture necessary to achieve improved outcomes”, and advises that an invaluable external facilitator has been engaged to help make the Alliance work. This attitude seems to fit into DMO’s ‘..hope that over the next five or 10 years of this (MPR) report you see a transition where things do not happen anymore or when we mitigate some of those risks”, as given in evidence at the JPCAA hearing into the 2007-08 MPR on 19th March 2009. Australia can hardly be expected to accept a continuance of these ‘things’ which impact directly the country’s security.

Project Maturity Score and Benchmark.

See body for analysis for an assessment of the usefulness of these scores.

Risk Consequences.

While DMO sees both capability and schedule being achieved as planned, potential consequences of the risks inherent in this project are assessed as approaching Severe/Catastrophic.

BRIDGING AIR COMBAT CAPABILITY

(Refer also body of this analysis and Annex B)

General.

The background to this project re-writes some of the history of its birth, but does classify it as a ‘directed government solution’. However, it is important that this project be seen as part of Australia’s New Air Combat Capability (NACC).

The Super Hornet, an aircraft rejected by other Western nations as a fighter replacement, was marketed robustly by the maker to Defence and Government and was purchased abruptly by the Minister of the day as a ‘no brainer’, against the advice of Air Force. Both Government and Defence had been provided with substantive data and analysis that shows that the aircraft will not be able to guarantee regional air superiority, as required by government policy. However, Manufacturer’s promises were accepted without question, even where they conflicted with the basic laws of physics.

As a result, the decision was criticised severely by the Leader of the Opposition and the shadow Minister for Defence before the last election, both promising to rectify what was wrong, and many hoped that this would occur.

The new Minister for Defence, Fitzgibbon, soon initiated a two-part review of Defence/DMO New Air Combat Capabilities planning – the first covering the decisions taken in regard to the unnecessarily early retirement of the F-111 and the decision to purchase the Super Hornet – Part 2 would later review the Howard Government’s JSF decisions.

Part 1 of the review was conducted internally by Mr Neil Orme of Defence. Mr Orme was provided with substantial personal and written submissions by individuals and Air Power Australia, which analysed and detailed the areas of risk associated with the New Air Combat Capability decisions being taken by Defence, including the JSF Project. He also had the many US governance reports issued over the years to guide him. These submissions and reports were simply unacknowledged and totally ignored, despite continuing assurances that Defence welcomed open discussion of its plans and programs. The failure of Defence/DMO to take note of the extreme risks associated with the aircraft may be measured by comparing the content of the Orme Report and the presentations made to him.

Since then, Government, Defence, and DMO have been kept abreast, in detail, of the many areas of risk associated with the JSF Project, but no risks were admitted by Defence/DMO or apparently by the over fifty ‘specialists’ involved with the JSF and NACC projects. All Defence/DMO statements have merely repeated the clearly discredited statements emanating from the manufacturer and the US Project Office which have always marched in ‘lock step’. The JSF Project is analysed in more detail in the body of this analysis.

The Minister’s decision to remain ‘Hornet Country’, an aircraft ignored by other nations in search of a high capability air combat aircraft, and Defence/DMO’s blind commitment to the JSF will not only enshrine Australia’s inability to exercise air supremacy in our region, but will condemn Australia’s management of defence capabilities in the eyes of the world.

Lessons Learned.

The Key Lessons Learned simply record basic knowledge that the RAAF possessed some 70 years ago.

Collateral Damage.

The Super Hornet Project continues the destruction of Australia’s military capability skills base:

  • First, Australia lost the high technology skills and competencies base that had been built up at Amberley, QLD, to support the F-111 Fleet, including systems integration and embedded software/hardware development, when the aircraft was prematurely retired. These core skills, which were critical to the support of Australia’s future air combat capability, were abandoned by Defence.

  • Second, similar albeit much less developed support facilities and competencies established at Williamtown, NSW, for the F/A-18 Fleet will be largely disbanded as the Super Hornet, with its overseas support contract, comes into service and the current A/B models deteriorate. This will leave another hole in Australia’s air combat support capabilities.

  • Finally, when the AP-3C Orion is replaced by an aircraft supported by overseas contractors, Defence will direct that the Edinburgh, SA, support facility be disbanded, and with that will go the last of Australia’s major local support capabilities.

This is the inevitability of Defence/DMO policy to let supply and long term support contracts to foreign companies having no business reason to set up support facilities in Australia. Defence/DMO policies are based upon a perceived reduction in cost and risk, which will, in the end:

  • Greatly increase cost.

  • Result in the further debilitation and de-skilling of the Services and Australia’s defence industry.

  • Lead to the collapse of Australia’s defence self-reliance.

  • Greatly increase the risk to Australia’s security.

Risk Consequences

This project still carries capability risks with the potential consequences that could reach a level of Severe/Catastrophic.

MULTIROLE HELICOPTER

General.

This project was entered into too early and planned inadequately due to a lack of in-depth experience with helicopter technology and an absence of sound project management disciplines. This parallels problems observed in Europe with the same basic NH90 design, detailed in the February, 2010 disclosure of the highly critical BundesWehr Luftlande- und Lufttransportschule (Airborne and Air Transport School) operational suitability assessment report, which recommends “using alternative aircraft whenever possible in an operational scenario”.

This is highlighted in, for example:

  • The immaturity of the design, requiring costly retrofit to achieve the required capability.

  • The aircraft’s questionable ability to conduct the required range of operational tasks.

  • A lack of understanding of the role and functions of configuration management in identifying and quantifying support requirements accurately and timely, for example, spares and technical documentation.

  • An inability to coordinate project milestones, evidenced by the inadequate flying effort achieved, which has also impacted Service flying training.

  • Failure to analyse system and equipment reliability against the configuration baseline as it matures, a basic function of project management which drives requirements for manpower, skills and training, technical documentation, repairable items and breakdown spares, facilities, tools, test equipment, and so on, to meet the unique support needs of the Service operating the equipment.

This project seems to have been driven by schedule at the expense of capability and sustainability.

Maturity Scores and Benchmark.

A total score of 57 against a benchmark of 57 hardly reflects reality. The question of the value of these scores is analysed in the body of this paper.

Risks.

Although mostly identified as “There is a chance...”, the major risks listed do give a better insight than many projects. However, they are still at too high a level and are all problems that should have been identified and managed (fixed) under standard project engineering procedures.

Each ‘risk’ begs the question:

‘How and why did this happen?”

The answers to that question would have led to the real ‘lessons learned’ being identified and the appropriate corrective action taken for that project and for future projects.

Finally, the remedial actions given for each of the ‘risks’ identifies a lack of understanding and application of the standard project/engineering management procedures that must be applied rigorously and consistently on every project. In the absence of responses to the question raised above, the lessons learned provide no insight as to why these ‘risks ’ and ‘issues’ occurred and how they might be corrected for the future.

Risk Consequences

This project still carries risks with potential consequences of Severe/Catastrophic.

AIRBORNE EARLY WARNING AND CONTROL AIRCRAFT

Capability Performance.

“Integrated system performance is currently not meeting specification. … However, remediation of all radar performance shortfalls is not expected to be achieved by final delivery of the system”. The project is some four years behind schedule, so should be able to provide many useful ‘lessons learned’.

Major Project Risks.

As for so many other projects, the risks recorded are generic and have not been tested by the simple question: “How and why did this happen?” so as to get to their root cause(s) and identify the corrective action needed.

Lessons Learned.

All ‘lessons learned’ fail to provide answers that can be used to modify management systems and procedures to prevent future occurrences, as they are couched in subjective, generalist terms rather than objective, management terms. The project still reflects inadequate management systems and technical expertise.

Contractor Support - Good Value for Money?

Considerable concern is felt in regard to the Minister’s recent statement regarding the Wedgetail Through Life Support Contract. The fundamental question relates to how DMO determined that the contract provided good value for money. When the RAAF managed its own operating, intermediate and deeper maintenance facilities and contractors, determining contract scope and value for money was relatively straightforward and accurate. As DMO cannot fall back on that hands-on experience, the way in which DMO determines whether Australia is getting value for money needs to be better explained and made transparent.

A first look by an ex-RAAF engineer officer with project and maintenance management expertise points out:

  • The contract for $A800 million for provision of “Services including logistics, training, spares management, aircraft deeper maintenance, engineering and supply chain management” for six Wedgetail aircraft over five years comes nominally to about $A26.7 million per aircraft per year.

  • In contrast, the Boeing F-111 Weapon System Business Unit contract provided services, including logistics, training, spares management, aircraft deeper maintenance, engineering and supply chain management, including numerous SME sub-contractors, plus capability enhancements, at a total average support contract cost of about $A100 million per year. For the fleet of 27 aircraft, this amounted to about $A3.7 million per aircraft per year. With, say, an average of 12 mission-ready aircraft on line, this amounted to about $A8.7 million for each aircraft per year. Even going down to the six aircraft on line during later years, the cost comes to $A16.7 million per aircraft per year.

While some may argue with the comparative scope and costs quoted, the figures give a reasonable comparison of value for money. It is up to DMO to explain transparently just why it considers the AEW&C Contract to be good value for money. One has also to factor in the reduced visibility and control of activities that impact directly force capability readiness and sustainability that come with DMO’s support contracts, as well as the dwindling skills base of the Services and our defence industry base, and Australia’s self-reliance.

Risk Consequences

The consequences of the risks inherent in this project have been at the Severe/Catastrophic level for many years.

AMPHIBIOUS DEPLOYMENT AND SUSTAINMENT
(LANDING HELICOPTER DOCKS)

Status.

The Project Summary states that the project is within budget, on schedule, and delivering the required capabilities. The project thus gives itself a maturity score of 45 against a benchmark of 45.

Major Risks.

However, the optimistic Summary above has to be read against the five major risks being encountered, covering potential regulatory changes, changes in requirement, inability of the combat system to meet performance requirements, possible damage to propulsion pods, and inadequate funding for sustainment.

Each of these should have been qualified and quantified in terms of problem or risk, and appropriate project/engineering/risk management actions planned and taken. The piecemeal remedial actions proposed will prove to be inadequate in the absence of such management actions.

Major Project Issues.

The integration complexity highlighted will almost certainly test the management of this project, but the remedial action proposed does not generate confidence that the management of engineering complexity is understood. The project, at this stage must thus be given a low probability of succeeding.

Risk Consequences

The consequences of the risks inherent in this project have the potential to reach Major, approaching Severe/Catastrophic levels.

ARMED RECONNAISSANCE HELICOPTER

General.

This project has experienced the range and type of difficulties that usually stem from incoherent project/engineering/risk management.

It is noted that $A6.5 million has been ‘harvested’ from this project, but whether this forms part of the Defence Strategic Reform Program is not revealed, nor have the impacts that will arise from the cuts.

The detail on this important project does not make good reading. It reinforces the impression that the project was scoped inadequately, and not resourced and managed properly. Where ‘project planners’ are mentioned, it appears that their functions relate more to contract administration than coherent project/engineering/risk management.

Major Challenges.

The major challenges listed reflect tasks that should have been handled in the project management sphere, the systems engineering sphere, and the support engineering field respectively.

For a project that is said to be 27 months later than originally planned, with some major elements up to 62 months late, has been re-baselined, and is still facing considerable challenges, to be assessed as “still expected to deliver the required capability within the approved budget” stretches credibility.

Risks, Major Issues and Lessons Learned.

Again, these sections give insufficient insight as to how and why these risks and issues arose, and the lessons learned do not get to the nub of the problems so as to be able to correct them in a timely way and feed changes back into management systems to ensure that they are managed properly on future projects.

Risk Consequences

The risks inherent in this project have reached the Severe/Catastrophic level.

AIR TO AIR REFUELLING CAPABILITY

General.

“Airbus Military’s ability to meet the contracted schedule milestones continues to be the greatest challenge due to an underestimation of the overall scope and complexity of work and system improvements introduced during the development”.

This situation is indicative of a project that was embarked upon when inadequately scoped, planned and resourced with the required skills and competencies, and proceeded without sound project / engineering / risk management systems and procedures in place. The usual over-reliance upon contract management and contractor support for tasks that must be managed by the customer is also evidenced.

Major Risks, Issues, and Lessons Learned.

The risks and issues listed provide better than generic statements and highlight a number of risk/problem areas that should normally have been identified in time and mitigated through standard project/engineering/risk management procedures before they emerged as real risks/ problems/issues.

There is still a confusion between risks, problems, and ‘issues’, the latter term these days being used to describe any number of unqualified and unquantified ‘things’. The word ‘issues’ should be discouraged at all levels in project management.

Again, the risks and issues listed have not been tested to identify the root cause(s), and so the ‘lessons learned’ are not of much use in the practical project management sense of ensuring that they do not cause problems for future projects.

Risk Consequences

The risk consequences inherent in this project are at the Major level, approaching Severe/Catastrophic.

HORNET F/A-18 UPGRADE – PHASE 2

General.

This project faces several ‘risks’ which seem to be managed through an amalgam of project office and contractor activities.

The key lessons learned reinforce this impression, as the majority of the activities listed for contractors and Product Teams are normally core project management functions that rightfully belong to the customer, the DMO Project Office. This arrangement also highlights the many different approaches adopted by DMO, which are more contract than project management focussed, giving rise to the problems faced by ANAO/JCPAA in trying to get objective and consistently accurate and consistent project status information across projects.

A key concern with this project, as well as Phase 3.2 of the Hornet Upgrade Program, is the cost effectiveness of the work being done as an element in Defence’s NACC planning, faulty as that planning might be.

Notable by its absence is any mention of the failed approach taken by the DMO in Air 5376 Phase 2.3, against strongly founded technical advice from within Defence and Industry, when the then Head of Electronic Systems Division of the DMO, with the support of the then Under Secretary of the DMO, ensured the selection of an extremely high-risk Radar Warning Receiver (RWR) System (the ALR-2002) over the recommended ALR-67 Version III RWR system, which is only now being installed into F/A-18 Hornet aircraft.

This decision resulted in over 5 years and some (estimated) $A 440 million dollars of project capital budget funds being wasted, plus the personnel, travel, subsistence and related costs associated with the ‘project management’ of this solution. An estimate of the overall wastage of Commonwealth resources would put the amount, conservatively, at well over half a billion dollars.

Major Risks and Issues.

Two major risks/issues are recorded for both Phase 2 and HACTS under ‘There is a Chance...’. Remedial action in all cases is contract centric. Under project management procedures, each ‘risk’ would be classified as a problem or a risk and if a risk, the classification of that risk and the manner in which it will be managed under project management procedures.

Key Lessons Learned.

These generally relate to what contractors are to do and contract management activities. Many of the activities recorded are DMO Project Office functions, as DMO carries accountability for the visibility and control of all project/engineering/risk/IV&V activities.

C-17 GLOBEMASTER HEAVY AIRLIFTER

General.

This project lost about a year because essential operational equipment was not delivered with the aircraft, as it would normally have been under pre-DRP/CSP RAAF management.

This failure indicates poor or absent project management procedures in that support requirements were either not identified in the operational requirement and/or support requirements specification, and were not included in the Project Management Support Plan.

Again, the risks stated and the lessons learned do not identify just how and why these things happened and how they might be avoided on future projects.

It should not take “a couple of years to develop up the sustainment” as DMO advised the JPCAA Hearing of 19th March 2009. Pre-DRP/CSP, RAAF showed how sustainment requirements can be integrated into Project Planning and managed such that all sustainment requirements are in place by the time an aircraft/system arrives in country.

Finally, the support contract entered into for this aircraft should be tested to see whether it provides value for money. The USAF certainly found its contract far too expensive and sought changes. Is DMO’s contract costing too much?

Risk Consequences

This project, although purchased through FMS arrangements, has still achieved a risk consequence level of Major.

GUIDED MISSILE FRIGATE UPGRADE IMPLEMENTATION

General.

This project experienced serious problems which DMO failed to identify in nature, extent, and impact, so that timely and effective action was not taken and the real lessons learned were not identified and used to good effect on this or future projects.

Delays in achieving Initial Operational Capability (IOC) are recorded as being 55 to 71 months, and Final Operational Capability (FOC) from 48 months to 65 months, and then only after reaching a ‘pragmatic agreement with Navy’, the nature and impacts of which are not identified.

Major Risks and Issues.

Although all assets are recorded as having reached their FOC during December 2009, this section reports that “There is a chance that...”

  • The Combat System Software may not meet contracted requirements.

  • Upgraded systems may not have effective software support and configuration management in place.

  • Upgraded systems may not have acceptable reliability and maintenance data which will impact capability and support.

  • The Warfare Systems Support Centre may not be ready.

  • The project may not meet current Navy Technical Regulatory requirements under the contract.

  • The required Electronic Support System performance may not be met.

  • The Torpedo Defence Systems integration and performance may not be met.

  • The Hull Mounted Sonar may not meet performance requirements.

The remedial actions given are largely contract and outsourcing orientated. They reflect no sense of any firm project management control; for example, there is no sign of those core project management functions that relate to project planning, developing and managing the design and configuration baselines, or support requirements determination (there are many others of course).

Against this background, a maturity score of 57 against a benchmark of 57 would, prima facie, seem difficult to justify.

Key Lessons Learned.

These lessons, as for the other projects reported, list the most basic competencies, without which no project should ever proceed. They are all indicative of a lack of even a basic understanding of project management and the competencies required. Peppered throughout this list are what contracts should contain and what contractors should do – hardly a word about what DMO should be doing to overcome the perceived ‘risks and issues’; nothing about project/engineering/risk/IV&V management deficiencies within DMO. This project, like many, failed because it wasn’t managed properly by people with the required management structures and competencies – it was managed by generalists using contract and business processes.

There is much to learn from this project, especially with the Air Warfare Destroyer Project coming along, as well as early planning for a Collins Submarine replacement. However, with DMO not being a true learning organisation, and being focussed on contract administration and process rather than project management and engineering, what is seen in these PDSs can be expected to continue.

Risk Consequences

This project has carried a risk consequence of Severe/Catastrophic for many years.

HORNET Phase 3.2

General.

The comments relating to the Hornet Upgrade Phase 2 Project relate also to this project.

Major Risks and Issues.

The Major Risks and Issues recorded relate to the unknown impact of fatigue damage and the lack of maintenance managed items to support the new aircraft configuration. The latter points to a failure to maintain close attention to the changes taking place in the aircraft’s configuration baseline – a project management responsibility, to calculate the variations to fly away kits spares holdings, and maintenance pipelines, and so ensure that timely notice is given of the new supply requirements - all tasks previously done by RAAF as a matter of course.

Of major longer-term concern is the fatigue state of RAAF aircraft and the soundness of future fatigue monitoring programmes resulting from DMO policy to include fatigue management in the maker’s sustainment contract. Internationally, RAAF and DSTO had an envious reputation for the fatigue management of Australia’s military aircraft, a function now dispersed and outsourced. Defence/DMO will, hopefully, learn quickly that aircraft manufacturers face a problem in monitoring the unique fatigue spectra, fatigue damage and fatigue lives of each RAAF aircraft. The results can, in the extreme, impact flying safety (airworthiness), as well as Australia’s air combat capability. Fatigue management of Australia’s military aircraft should remain in Australia and be managed centrally by Australians.

The lessons learned with the RAF Nimrod disaster are central to the inevitable effects of Defence/DMO’s change of policy on fatigue management.

Key Lessons Learned.

These miss somewhat the objective of recording lessons learned for use in refining/changing management policies, systems, and procedures.

Risk Consequences

These will emerge over time, but they will emerge.

BUSHMASTER PROTECTED MOBILITY VEHICLE

General.

The Critical Design Milestones for the Vehicle Initial Review run from 12 to 26 months late, depending upon vehicle type. System Integration now runs from 6 to 26 months late, while contractor Test and Evaluation progress is from 6 to 37 months over time.

Major Risks and Issues.

As reported, “There is a chance” that delivery of operational capabilities may be delayed by:

  • Changes due to operational feedback (now retired).

  • Delays in processing Engineering Change Proposals (ECPs).

  • Changes required by the need for sustained towing.

  • Complex requirements in the specifications for the development of the Protected Mobility Air Defence Variant due to new operational requirements and the effects of foreign sales.

The remedial actions given consist mainly of contractual activities and enlisting ‘stakeholder’ involvement. However, standard project management procedures should drive approved operational and technical requirements in a closely coordinated way, and in turn monitor contractor performance in meeting those requirements at specified milestones. Variations to project planning, including those recorded as risks and issues, are then coordinated and controlled through the project management system, which in turn drives any necessary contractor/contract activity. This project seems to suffer from inadequate or ineffective project management, allowing the contractor to control the project.

The fact that “Thales has provided an undertaking to consult the Commonwealth where any potential schedule conflict arises from other customer inquiries” reinforces this impression.

Major Project Issues.

These relate to:

  • The unavailability of wiring harnesses.

  • The late construction of facilities, which raises the question as what advantages accrue from having Defence Support Group become involved in a project in which they have no specialist competencies.

  • Delays in processing ECPs.

  • A shortage of headsets compatible with the alternative wiring harnesses procured.

Three of these four would be expected to have come under standard project management procedures. The fourth, the lateness of the facilities points to one of many unnecessary and risk-laden interfaces that exist in all Defence/DMO processes.

Key Lessons Learned.

The three key lessons learned are as recorded, as follows:

  • In the early planning phases of the project, the operational concept and functional performance requirements were not clearly defined, making it difficult to understand and undertake appropriate cost-capability trade-offs.

  • Cost Estimating – there was a lack of industry capability to provide adequate cost estimates and inability by Defence to evaluate the validity of the cost data.

  • Testing program – significant contingency planning should be conducted for compliance testing of a new capability.

These all indicate a lack of the most basic competencies required before a project can even be considered, and a total absence of any project management framework. That this project was allowed to proceed with these deficiencies is damning. It was planned to fail. In short, too much reliance upon contract administration and inadequate project/engineering/risk management, and no IV&V!

Risk Consequences

This project has reached a risk consequence of Severe/Catastrophic.

HF MODERNISATION

General.

Although this project records a delay of some 74 months for both Initial and Final Operational Capabilities, and 127 months for the Mobiles, the project still earns a DMO Project Maturity Score of 54 against a baseline of 55 – whatever that means.

Major Risks.

The two additional risks included in this report relate to the Fixed Network’s failure to meet the contracted Grade of Service and Speed of Service, as well as inadequate software design documentation. Both problem areas are to be managed by ‘working closely with, and monitoring, the contractor, both passive activities’. There is no sign of any active project management here.

Major Project Issues.

The addition items included in this report covered:

  • The Contractor’s failure to meet the schedule has been ‘fixed’ by revising the schedule.

  • Fixed Network software development had not achieved the agreed schedule.

  • Contractor delays will delay completion of Mobiles upgrades beyond current project date.

  • As a consequence of delays to final system acceptance, and the inability of Navy to provide needed operators, the project has had to get the contractor to fill the gap.

These are all indicative of a less than robust project management system.

Key Lessons Learned.

These remain unchanged from the 2007-08 MPR.

This project is symptomatic of one starting without an adequate requirements baseline, inadequate or missing project management planning, inadequate engineering skills and competencies, inadequate risk management, and no IV&V plan. Despite this, the project was allowed to proceed, relying solely upon contract administration.

Risk Consequences

This project reached a risk consequence level of Severe/Catastrophic many years ago.

ARMIDALE CLASS PATROL BOAT

General.

The project seems to have been an attempt by DMO to be a ‘smart’ buyer by acquiring a commercial design that, seemingly, had only to be modified to meet RAN operational requirements. The chosen vessel however also required Navy to change its traditional crewing policy to fit the constraints of the commercial design.

The vessel has encountered serious design defects which have led to it being virtually ‘black balled’ by RAN sailors on both operational and safety grounds. Needless to say, Defence/DMO and the RAN hierarchy have lauded the vessel.

Despite this, DMO’s PDS states that “All vessels continue to meet Navy’s operational requirements”.

Progress to FOC.

The project is now 33 months late, due to outstanding defects. FOC is now estimated to be 2011.

Major Risks and Issues.

These are recorded, as follows:

  • Navy standards are different to commercial standards resulting in a risk to customer acceptance.

  • Contractor inability to provide support vessels throughout the life of the in-service phase of the contract (performance risk).

One would expect Navy to have had such risks well in view and under tight control at least ten decades ago.

Lessons Learned.

Both of the Key Lessons Learned are simply generic, and are symptomatic of a failure to follow established project management, engineering management, risk management, and IV&V methodologies from the beginning of this project to the end. This project highlights the the capital risks accepted by DMO in embarking on a project of medium complexity, relying wholly upon contract administration for its management. The project has cost Australia much in time, money, and compromised and late capability requirements, but there is no sign in this PDS that any real lessons have been identified and digested.

Risk Consequences

This project has reached a risk consequence of Severe/Catastrophic.

COLLINS REPLACEMENT COMBAT SYSTEM.

General.

The project is recorded as now being six years behind schedule, which represents a significant part of the boat’s life of type.

The operational capability of the Collins submarine depends upon the quality and reliability of its Combat System. One would thus expect everything possible would be done to select and commission the best system available. Unfortunately, for Australia’s security, and the risk to those RAN members who will man and operate these boats, and be exposed to the hazards of war when required, the management of the Collins Combat System has to be condemned.

In response to the dilemma, DAO/DMO had reached in relation to the Combat system originally specified, an innovative South Australian SME accepted the risk and developed a combat system for the Collins submarine that was provided to Defence and has been installed on one of the boats for years.(1) This Australian designed combat system has never failed, can do more than the system selected, and costs a fraction of the very expensive American system. (2)

However, the locally designed system was ignored. Defence, DMO and Navy selected the American design because the US Navy threatened to cut off Australia’s access to intelligence information if they did not commit to the American system. The American system has cost Australia hundreds of millions of dollars in addition to the hundreds of millions of dollars wasted on the original failed system. The system selected still fails to meet requirements, even after these were downgraded to ‘help it across the bar’.

In the case of the Collins Combat System, Defence/DMO have weighed in their balance whether to accept a second rate combat system at remarkably high cost so as not to alienate our sister nation (or one of its Services), against accepting a locally-developed system that has proven to be far more capable and reliable than the US system, far less costly, and able to be developed further and supported in Australia. That they have selected the former alternative indicates clearly that the security of Australia and the safety of our Naval crews run a poor second on their scale of values.

Risk Consequences

This project reached a risk consequence level of Severe/Catastrophic many years ago.

References:

  1. http:/www.acres.com.au/

  2. http:/www.raytheon.com/business/rtnwcm/groups/public/documents/content/rtn_bus_ids_prod_anbygl_pdf.pdf

ANNEX A: ATTACHMENT 1 – PROJECT RISK CONSEQUENCES

As with other Australian and International Standards, DMO has re-invented those Standards relating to risk. For example, the Australian and International Standards define the highest level of consequence as ‘Catastrophic’, while DMO defines it, euphemistically, as ‘Severe’. At this risk level, the consequences are defined by DMO as:

  • Causing loss of life,

  • Supplies functionally unfit for their intended purpose.

  • Supplies are unsupportable.

  • Defence attracts media attention, or a commission of inquiry is launched.

  • Would cause the specified in-service date (of the IOC) to be missed by more than twelve months.

  • Would cause the in-service date for the FOC to be missed by two or more years.

  • Would cause the total actual contract costs, taking into account liabilities incurred by the Commonwealth, to exceed currently approved cost provisions by >10%.

DMO then defines the next highest level of consequence is defined by DMO as ‘Major’, the consequences being as follows:

  • Would cause serious casualties resulting in the long-term physical impairment of personnel.

  • Would cause the supplies to be only partly functionally fit for purpose (i.e. degraded ability to perform some core missions or essential tasks or unable to perform non-core missions or tasks, and there are no known workarounds).

  • Would cause the supplies to be unsupportable in low-tempo operations or for short periods of time due to a deficiency in a fundamental input to capability. There are no known workarounds.

  • Defence attracts adverse media attention or an investigation is launched.

  • Would cause the specified in-service date to be missed by 6 – 12 months.

  • Would cause the date for full operational capability to be missed by between one and two years.

  • Would cause the total actual contract costs, taking into account liabilities incurred by the Commonwealth, to exceed currently approved cost provisions by 5-10%.

The inclusion of media criticism as a risk consequence leaves the whole risk management process open to bureaucratic distortion/misinformation on the grounds that the truth may ‘damage’ the Department’s reputation. For example, the manner in which late/over-costly/ capability deficient projects might be managed.

Within both Major Projects Reports to date, no reference is made to any actual risk levels and consequences in accordance with any Risk Standard. A review of DMO MPR 2008 – 09 against DMO’s own risk standard follows.
PROJECT TYPE OF CONTRACT FORM OF CONTRACT IOC
PLANNED 		IOC
CURRENT 		FOC
PLANNED 		FOC
CURRENT 		COMMENTS
AIR WARFARE DESTROYER
- 3 AWDs + SUPPORT SYSTEMS
- PLATFORM DESIGN
- 3 AEGIS SYSTEMS
VARIABLE (Pain/Gain Share)
FIXED (With Ind Escalation)
FMS
ALLIANCE
ALLIANCE BASED
FMS 		JUN 16
DEC 18
Currently shown as on schedule, but carries risks potentially at Severe/Catastrophic level.

BRIDGING AIR COMBAT CAPABILITY FMS FMS DEC 10
DEC 12
Currently shown as on schedule, but see PDS analysis for potential operational risks
MULTI ROLE HELICOPTER VARIABLE ASDEFCON
(Strategic) 		JUL 10 (N)
APR 11 (A) 		JUL 10 (N)
OCT 11 (A) 		DEC 12 (N)
JUL 14 (A) 		DEC 12 (N)
JUL 14 (A) 		Army version currently 6 months late. Potential risks assessed at Severe/Catastrophic level.
AEW&C VARIABLE DEFPUR 101 DEC 07 DEC 11 DEC 08 DEC 12 4 Years late currently
Risk consequence now
Severe/Catastrophic
AMPHIBIOUS CAPABILITY VARIABLE ASDEFCON JUN 15
NOV 16 		JUN 15
NOV 16 		NOV 16 NOV 16 Currently shown as on schedule, but inherent risks assessed at Major, approaching Severe/Catastrophic.
ARMED RECCE HELICOPTER VARIABLE SMART 2000 JUN 07 SEP 09 JUN 09 DEC 12 IOC currently 27 and FOC 42 months late. Risk consequence now
Severe/Catastrophic.
AIR TO AIR REFUELLING VARIABLE AUSDEFCON DEC 09 Late 2010/
Early 2011 		MAR 11 LATE 2012 IOC currently 12 and FOC 18 months late.
Risk consequence now Major, approaching Severe/Catastrophic.


C-17 GLOBEMASTER FMS FMS AUG 07 SEP 07 DEC 11 JAN 11 FOC currently 11 months late. Risk consequence now Major.
HORNET UPGRADE FIRM/FIXED (5)
TIME & MATERIALS & FIRM/FIXED (1)

DEFPUR 101
FMS
ASDEFCON
MIXED 		APR 07
NOV 09 		APR 07
NOV 09 		DEC 07
AUG 11 		DEC 07
AUG 11 		Currently shown as on schedule, but see PDS analysis for risks.


GUIDED MISSILE FRIGATE VARIABLE DEFPUR 101 MAY 03
JAN 04
JUL 04
JAN 05 		APR 09
APR 09
APR 09
AUG 09 		JUL 04
DEC 04
JUN 05
DEC 05

DEC 09
DEC 09
DEC 09
DEC 09 		IOC late by 55 to 71 months and FOC late by 48 to 65 months.
This project proceeded for many years with a risk consequence of Severe/Catastrophic.
HORNET STRUCTURAL FIXED PRICE/TIME & MATERIALS (2)
TIME & MATERIALS (2)
FMS

ASDEFCON (3)
DEFPUR (Hybrid)
FMS 		N/A N/A N/A N/A See comments at PDS analysis for potential fatigue risks.
BUSHMASTER Variable DEFPUR 101 N/A
JUL 07
JUL 08
OCT 11 		DEC 04
MAR 08
NOV 08
OCT 11 		OCT 07
APR 09
APR 12 		DEC 09
APR 10
APR 12 		IOC up to 8 months late.
FOC up to 26 months late, which gives a risk consequence of Severe/Catastrophic.

HF MODERNISATION VARIABLE DEFPUR 101 v46 MAY 04 JUL 10 MAY 05
MAY 05

JUL 11
2016 		On current estimates, IOC 74 months late, and FOC 74 and 127 months late,giving a risk consequence of Severe/Catastrophic.

ARMIDALE CLASS BOAT VARIABLE SMART 2000/ ASDEFCON N/A N/A MAR 09 DEC 11 FOC currently 33 months late, giving a risk consequence of Severe/Catastrophic.
COLLINS COMBAT SYSTEM VARIABLE (4)
FIRM PRICE (5)
FIXED (6) 		AUSDEFCOM
(Strategic) (4)
(Complex) (6)
(Services) (1)
FMS (3)
Army Co-op Proj (1)

MAR 08 MAY 08 2010 2016 FOC currently 72 months late, so has been at a risk consequence of Severe/Catastrophic for many years.
The operational capability being provided will remain questionable over the life of the Collins Class boats.

ANNEX B EVOLUTION IN MANAGEMENT APPROACHES - THE PRODUCT VERSUS THE PROCESS

The Evolution of Management

Over the past 30 years or so, Western businesses have changed their focus from manufacturing to marketing, calling for a change in the thinking of their workers and the education system that provides them. The product has become less important than process and process is optimised by means of management techniques that function on a level different from those used for production.

“The contemporary office thus requires the development of workers ready for ‘Teamwork’, rooted in shared habits of flexibility rather than strong individual character. At issue in the contrast between office work and the manual trades is the idea of individual responsibility, tied to the presence or absence of objective standards.”(1) To ‘manual trades’ may be added any activities involving the operation, sustainment, or replacement of technology.

The development of modern, knowledge-based management methodologies and their conflict with objective job satisfaction and the sound management of technology has been well analysed by Matthew B. Crawford, who has experienced the problem both as a manager and a worker, in his book ‘Shop Class as Soulcraft – An Inquiry into the Value of Work’ (1). The following observations and conclusions draw upon his findings, quoting from his book.

“Those who work in an office often feel that, despite the proliferation of contrived metrics they must meet, their job lacks objective standards of the sort provided by, for example, a carpenter’s level, and that as a result there is something arbitrary in the dispensing of credit and blame. The rise of ‘teamwork’ has made it difficult to trace individual responsibility, and opened the way for new and uncanny modes of manipulation of workers by managers, who now appear in the guise of therapists or life coaches. Managers themselves inhabit a bewildering psychic landscape, and are made anxious by the vague imperatives they must answer to. The college student interviews for a job as a knowledge worker, and finds that the corporate recruiter never asks him about his grades and doesn’t care what he majored in. He senses that what is in demand of him is not knowledge but rather that he project a certain kind of personality, an affable compliance. Is all his hard work in school somehow for show – his ticket to a Potemkin meritocracy? There seems to be a mismatch between form and content, and a growing sense that the official story about work is somehow false”.

On the other hand, those who manage technology, and the technicians who work to keep it functioning efficiently in support of most human endeavours, “must reckon with the infallible judgement of reality, where one’s failures or shortcomings cannot be interpreted away, His well-founded pride is far from the gratuitous ‘self-esteem’ that educators would impart to students, as though by magic....His experience of failure also tempers any conceit of ‘mastery’”. The technologist’s work is thus based upon a real understanding rather than the ‘fantasy of mastery’ that permeates much of our thinking today.

The objectivity standards of the technologist also stand in contrast to the marketeers, those knowledge workers skilled in the art of persuasion, and demagogues who offer magical hopes. Plato pointed to the difference between technical skill and rhetoric on the basis that rhetoric “has no account of the real nature of things, and so cannot tell the cause of any of them”.

Aristotle also made a point in regard to experience, another characteristic of a skilled technologist:

“Lack of experience diminishes our power of taking a comprehensive view of the admitted facts. Hence those who dwell in intimate association with nature and its phenomena are more able to lay down principles such as to admit of a wide and coherent development, while those whom devotion to abstract discussions has rendered unobservant of facts are too ready to dogmatise on the basis of a few observations”.

Crawford sees that the vulnerability of managers today is reflected in their highly provisional manner of feeling and speaking, which is characterised by an imperative towards abstraction designed to help them cope with the physic demands of their job. Importantly, authority is seen to be embodied in those up and down the hierarchy with whom he works; indeed, his career depends upon these relationships where the criteria of evaluation are ambiguous. Managers thus spend much time managing what other people think of them, with the result that much of the job consists of “a constant interpretation and reinterpretation of events that constructs a reality in which it is difficult to pin blame on anyone, especially oneself.”

This has given rise to the art of talking in circles, presenting mutually contradictory statements persuasively and forcibly, allowing the manager to ‘stake out a position on every side of an issue, or burying what he wants done in a string of vaguely related descriptive sentences. What people say in such an organisation is generally accepted as always being provisional. Nothing is set in concrete – objectivity is the first victim. Within groups, the boss’s ‘deniability’ must be protected by using empty or abstract language to hide problems, thereby keeping the field of subsequent interpretations as wide as possible.

“When a manager’s success is predicated on the manipulation of language for the sake of avoiding responsibility, reward and blame come untethered from good faith effort. He may then come to think that those beneath him in the food chain also can’t be held responsible in any but arbitrary ways”.

Technology and its proper management is in fundamental conflict with the management approaches that have evolved in both public and business organisations, as well as the system of education that supplies these organisations with suitable workers – ‘those pliable generalists totally unhampered by any specialist skills’.

The Military/Bureaucratic Interface (2)

Mr Smith, just before taking up the position of Secretary for Defence, spoke of the military / departmental interface in the following terms:

“It is self-evident that the very different natures of military and civilian service produce different cultures, and it is important that those differences be recognised and understood if the two groups are to work together effectively. To mention just a few of these differences, civilians are, for instance, generally more readily able to tolerate, and even be comfortable with, unclear lines of command, divided authority, and open-ended guidance or ambiguous instructions. They are also tend to be willing to offer judgements and opinions on the basis of less hard data than their uniformed colleagues, and to accept that outcomes can’t always be readily predicted or easily influenced. Again, the question of ‘ownership’, so important to military commanders who very understandably want to ‘own’ or have command of the assets needed to do the tasks for which they are responsible, is much less important to civilians, who are generally more comfortable about being dependent on others to deliver results. Approaches to careers and service are also, inevitably, different and so of course are conditions of service and expectations from the service of which they are members.”

Military professionals will see immediately the incompatibility between the characteristics of the military professional, who depends upon high levels of skills, sharply-defined tasks, clear accountabilities, objective measures of performance, and sound management of the resources needed to achieve them, and the civilian approach. How can the military exist or perform professionally in an organisation that accepts vague, tolerant, unclear lines of command and divided authority, as well as open-ended guidance and ambiguous instructions? The military is about conducting its tasks professionally and on time, with least risk, learning from mistakes and accepting accountability for the results; the civilian bureaucrat is about burying problems, diffusing tasks and obscuring and avoiding accountability. Within such an organisation, transparency and public accountability soon become victims. Unfortunately, the civilian approach now permeates the Services, especially at the senior levels, as well as the activities of Defence/DMO, and must be judged a failure in any organisation that has to produce and maintain a capability.

Those familiar with the management of technology – its operation, sustainment, and replacement - will recognise that the principles underlying professional military management are much the same as used to manage technology. Both rely upon high levels of skills and competencies, robust management systems, procedures, and plans having effective feed-back loops, the continuing management of risk and the positive use of mistakes to learn, clearly stated accountabilities and objectives, and auditable reporting mechanisms. All of these are incompatible with the non-military and non-technical management methodologies that have grown up in business and public organisations over the past several decades.

The Root Cause.

Where the military and the management of the technology it operates and sustains, meets an unskilled civilian bureaucracy entirely unversed in the management of technology, a major fault line must inevitably develop which will fail repeatedly to meet the need for effective and professional management of Australia’s Military Services and National security.

It is proposed that such a fault line has been introduced into Australia’s higher defence organisation and is a root cause for the problems being encountered by Defence and DMO.

References:

  1. Matthew B. Crawford, ‘Shop Class as Soulcraft – An Inquiry into the Value of Work’, Penguin Books, 2009. Crawford is a philosopher and mechanic. He has a PhD in political philosophy and served as a post-doctoral fellow at the Institute of Advanced Studies in Culture at the University of Virginia.

  2. E.J. Bushell, ‘Rebuilding the Warrior Ethos’, Airpower Australia Analysis 2008-10, 27th December 2008.


Air Power Australia Analyses  ISSN   1832-2433


Support Air Power Development in Australia - Click for more ...




People's Liberation Army Air Power Index Page [Click for more ...]
Military Ethics, Culture, Education and Training Index Page [Click for more ...]
Russian / Soviet Weapon Systems Index Page [Click for more ...]





Artwork, graphic design, layout and text © 2004 - 2014 Carlo Kopp; Text © 2004 - 2014 Peter Goon; All rights reserved. Recommended browsers. Contact webmaster. Site navigation hints. Current hot topics.

Site Update Status: $Revision: 1.753 $ Site History: Notices and Updates / NLA Pandora Archive