|Last Updated: Mon Jan 27 11:18:09 UTC 2014|
|An Analysis of Defence Materiel
Organisation Major Projects Management and What Needs to be Fixed
A Royal Australian Navy Kaman Seasprite helicopter in 2007. The Seasprite program failure was characteristic of DMO practices which persist. The program resulted in the much publicised cancellation and withdrawal from service. None of the root causes of the Seasprite failure have been addressed (©2007 Carlo Kopp).
Each of these deficiencies has impacted adversely Australia's force structure, operational capabilities, and national security.
With each major acquisition problem, Defence has often claimed that the problem lies in the past, a 'legacy' problem, and has emphasised that things are now on track and that such problems will be avoided in the future. However, as time goes by, the same problems are seen to arise, with the same adverse impacts upon Australia's force structure, capabilities and security.
Pre-Defence Reform and Commercial Support (DRP and CSP) Programs, Australia's capability acquisition procedures were recognised and respected widely for their effectiveness, efficiency and economy. The management systems employed were based upon transparency and public accountability, and were able to provide clear and consistent information on the status of each project. RAAF, in particular, had developed extremely sound and competent policies, systems and procedures.
These skills and competencies persisted until about 1998 when they were abandoned in favour of those that gave rise to the major equipment acquisition problems that soon followed, that have persisted to this day, and that DMO has warned us to expect to continue for another decade or more.
It is thus timely to revisit the development of Defence/DMO procurement methodologies in order to identify why and how Defence abandoned the established, professional management of major capability acquisitions and adopted the problem-entrenched, executive-driven, business/commercial approaches seen today, and determine how this situation may be reversed.
“The concerns I raise to this inquiry are my observations that the existing procedures were reconstructed by executive direction using externally sourced personnel who lacked domain exposure and the knowledge of past practices and pitfalls in acquisition. Worse, established processes were by-passed with limited regard to the enormous body of knowledge and the lessons learnt that had previously been incorporated.
I was very proud to be the Acquisition Member on the inaugural Project Board for Project AIR 6000, In accordance with the Project Management Method (PMM) which DAO was committed to at the time, the Project Mandate required an open mind to the pursuit of all capabilities, airborne and other, that could contribute to meeting the New Air Combat Capability requirement. Progressive Project Boards were fully minuted and inclusive of external contributions including the integral utilisation of a DSTO combat modelling expert seconded to the Capability staff, a system that had proved very effective with the source selection of the AEW&C contenders.
This process changed with the release of the Defence White Paper 2000 which declared the intention to acquire up to 100 fighter aircraft to replace the F-111 and F/A-18 capabilities. I was never party to the development of this position but understand that it was the position agreed by Defence with the Minister of the day. It was certainly a surprise to most members of the Project Board. The Board process began to disintegrate as the Chair (person) moved to another position, having been promoted at the behest of the Minister to the AEW&C project.
The incoming Chair for project 6000 was an officer returning from overseas tour of duty and unfamiliar with the acquisition reforms of recent years. He expressed the view that the PMM process was too arduous and was not supported by the DMO executive. Both views were true as the incoming USDM (Under Secretary Defence Materiel) did not accept the decision of the former Deputy Secretary that DAO would adopt PMM following successful trials and development of the process based on PRINCE 2 over a large number of projects for an extended period. This rejection was strongly supported by a civilian Division that was subsequently assigned leadership of developing yet another project management process.
In addition, the use of Project Boards, usually at one star level, did not sit comfortably with USDM. He introduced Governance Boards of external and internal consultants to advise him directly on project performance. It took considerable effort by the Aerospace Division, using an organisational concept model I devised, to convince USDM to retain the Projects Boards which managed the progress of projects and reported to the domain Division Heads (two star level) as complementary to the higher level Governance Boards which advised USDM.
However, AIR 6000 Project Boards did not continue under the new arrangements and I was not an active participant in the subsequent developments other than to establish a core project management team and second them to the capability staff as part of an Integrated Project Management Team that I had advocated for some years. My original project manager, a PMM expert, resigned from Defence in disgust at the relegation of his position and expertise. In late November 2001, capability staff presented an Industry Brief, a role usually undertaken by acquisition staff. The brief outlined the intention to seek information from industry as part of a three tiered approach: Capability Staff sponsored a Force Mix Market Survey of potential aerospace systems; DMO sponsored a Request for Information from international aerospace industry; and DSTO sponsored a Technical Data request to populate their modelling. All requests required responses by 1 Feb 02. My incoming project manager attended the brief and reported that it was not well received by industry with many attendees expressing grave concerns. I expressed my concerns to my superior and to CAF (Chief of Air Force) in an e-mail in December 2001 prior to my departure from the DMO.
Another factor which changed the face of the Air 6000 process was the public presentation made by then Minister Reith at the Defence and Industry Conference in June 2001. He introduced the identification of defence acquisition as a monopsony. My simple interpretation of the impact of this economic term is that contract savings are not necessarily best achieved via competitive tendering in the single buyer, single supplier relationship that represents most defence specialised acquisitions. This concept had been studied by RAND (ISBN: 0-8330-3009-4 MR-1362-OSD/JSF, 2001 http://www.rand.org/publications/MR/MR1362) and was put forward by Dr Paris Genalis of the USA during the ADAC 4 meetings in Canberra and later echoed by Dr Learmonth, Head Industry Division DMO at the Partnering and Alliances Seminar in November 2001. Dr Learmonth was subsequently assigned leadership of the JSF SDD study by USDM.
The consequence of this set of public statements was that there was no apparent commitment to competitive tendering for Air 6000 within the higher echelons of Defence at the time that the three tiered requests to industry were dispatched. I recall a comment previously made by USDM to Aerospace Systems Division star rank officers to the effect that he did not believe in spending funds on F-111 and F/A-18 upgrades and wanted to direct the money to early acquisition of the JSF. He also advised that he expected his staff to present counter-positions to capability staff to allow him and the Secretary to consider a range of possibilities rather than the position of capability staff. The time-frame of these comments was mid 2001. I was very uncomfortable with this position as it was totally counter to the Capability based Management System and its inclusive Integrated Project Management teams that had been in development for many years with the specific endorsement of the Defence Committee. I had been an early contributor to this model. My point here is that there was no agreed approach to the process management of Air 6000 within the hierarchy and clearly USDM was not going to take advice from those of us working within the previously endorsed acquisition process.”
While Defence/DMO have been urged repeatedly to adopt a more (invariably ill-defined and not justified) commercial/business approach, the adoption of a contract-centric focus, as opposed to a capability and evidence-based project management system, will never provide the unrealistic benefits proffered by the seductive marketeer, or meet the unrealistic expectations of Defence/DMO.
March 2003, the Senate Foreign Affairs, Defence and Trade Reference
Committee reported on materiel acquisition and management in Defence,
finding that there was relatively poor visibility on the progress of
major projects as far as Parliament and the public are concerned, and
called for the Auditor-General to produce an annual report on the
progress of major Defence projects. The first of such reports (a
trial) was issued on 20th
November 2008, covering nine projects.
It is interesting here to note that the DMO MPR 2007-08 (Page 13) spoke of:
“These projects are often expensive, technologically advanced and managerially challenging, and require DMO to manage contracts that:
Firstly, the Auditor-General refers to “DMO to manage contracts that:”, which implies that the contract manages the three factors listed above, whereas no contract can, and should not attempt to do so.
The three factors listed fall wholly within the scope of Project Management where operational and engineering domain experts work within established policies, systems and procedures designed specifically to:
The second report, tabled by the Auditor-General on 24th November 2009, was “ ...designed to provide improved transparency and public accountability for these (15) major acquisition projects through the presentation of clear and consistent information on the status of projects”.
This report marks about a decade of DMO management following processes based upon inadequate domain expertise, and following executive governance/commercial management processes rather than capability-based project management systems.
The key question that arises is whether the changes in management that were introduced in 1998-2001 have been successful or not, following a decade of evolution and application. The answer to that question has been sought through the analysis of the fifteen major projects covered by DMO MPR 2008 – 09 that follows, and the individual project analyses included as Annex A.
In the analysis that follows, references to DMO MPR 2008-09 are identified by the document page number, for example (Page XX), or to tables in specified sections of the report.
In his Independent Review Report at Page 133 of the MPR, the Auditor-General recorded his inability to assess the information contained in both:
“DMO's systems and processes are not sufficiently mature without considerable additional examination by the Australian National Audit Office (ANAO) to provide assurance in relation to the completeness and accuracy of the above mentioned information. In view of these factors, it was not feasible to obtain an appropriate level of assurance for the purpose of this review in respect of the information presented”.
The Auditor-General's Review can only be seen as strongly qualified, due to the lack of consistent project status data; data that should flow directly from any robust project engineering data base at little, if any, cost. The Government and the Australian Parliament might also ask why such a highly qualified review by the Auditor General for Australia has been buried so deeply within the report and, more importantly, why such core deficiencies remain after a decade of DMO management.
However, Defence/DMO have been unable to provide this in either of the reports issued to date.
It should be recalled that DMO's tasks were undertaken previously by the three Service Departments without, as a general rule, encountering the serious capability and schedule problems and cost overruns that the DMO has encountered over the past decade. RAAF, in particular, pre the DRP/CSP, specified, evaluated, selected, procured and usually introduced all of its aircraft and supporting systems and facilities to capability, schedule and cost with little fuss or bother, while ensuring also that all lines and levels of maintenance, in both the Service and Industry, were in place by the time the prime equipment arrived in Australia - a benchmark far more appropriate than the failed US and UK acquisition systems, and inappropriate civil construction contracts, currently used by the DMO.
The RAAF certainly struck problems and had to review frequently its new project management policies and procedures, and modify them in line with changing circumstances and technologies. However, it was able to do this from the professional operational and engineering competencies that had been built up over some 70 years.
Dr Alan Stephens, in his centennial history of the RAAF (5), noted that the appropriateness and effectiveness of the RAAF's functional organisation were tested severely by the continuing operational demands and technological challenges that followed WWII. The F-111 saga proved to be particularly testing. The weight of the political implications, as well as the extremely complex engineering problems associated with the aircraft, were carried jointly by the Minister for Air (Malcolm Fraser) and the RAAF's Air Member for Technical Services (AVM E. Hey). As recorded by Dr Alan Stephens:
“That highly successful outcome said a great deal about the RAAF. Malcolm Fraser's strong performance in his negotiations with Secretary Laird rested essentially on the Air Force's profound technical expertise, and that expertise had not been acquired by chance. In the first instance it was attributable to the far-sighted men who in 1948 established a Technical Branch with a core of tertiary qualified engineers. And in this instance it also owed a great deal to the astute leadership of Ernie Hey, who held his nerve in hard times and ensured that the RAAF received the right answers by personally selecting the Branch's best and brightest to manage the F-111 program. Consequently, when Australia's Defence Minister argued his case with the Americans, he did so from a position of authority.”
This situation persisted until the Services were starved for funds after the Whitlam Government's reorganisation of the Defence Departments in 1972, followed by the imposition of the Defence Reform and Commercial Support Programs which led to the extreme downsizing and de-skilling of the technology-based Services, the RAAF and RAN. The Sanderson Report of 1989, which led to the disbandment of the RAAF's Technical Branch, was the final straw. The current inability of Australia to argue its case in regard to the JSF from a position of authority may be traced to these changes. Further reading on the importance of the professional competencies that once existed in the RAAF is included at references 6 and 7.
Schedule, however, forms but one of the three major elements of project management, and the solution to DMO's perceived problems with project and schedule complexity, as well as technical difficulty, lie wholly, as they always have and will, in having in place:
DMO's appeal to complexity seems to be more of a universal excuse than the result of any reasoned and objective analysis of specific project problems. Although 'complexity' is now seemingly accepted uncritically by Government and governance mechanisms, with sound project and engineering management complexity would soon come under control and cease to be the ubiquitous ogre painted by DMO.
ANAO records DMO's position as “The more complex the project, the greater the risk in delivering within budget, on schedule and to the required capability. DMO's experience supports the view that the more developmental in nature a project, the more susceptible a project is to schedule delays compared to MOTS (Military-off-the-Shelf) solutions.”
This elemental statement highlights a fundamental management misconception. There can be no comparison between inherently complex projects and MOTS solutions unless the MOTS solution meets Australia's requirements fully in capability, schedule and cost, in which case the MOTS solution should have been considered in the first place. It would be most unwise, as implied, to favour MOTS solutions to meet Service requirements simply because of DMO's inability to manage projects, especially complex ones. MOTS solutions will often result in a progressive deterioration in a country's military capabilities by forcing seemingly unimportant changes in force structure, operational concepts, support policies and the Defence Industry base. The end result is to be prepared to fight someone else's threats in someone else's threat environment and geography, not one's own. This is a classic ‘tail wagging the dog’ situation. The only remedy is to recover lost project and engineering management and technical competencies, and develop an acquisition organisation that can handle complexity as a matter of course. However, project management processes are designed specifically to pull capability, schedule and cost together and manage them in an integrated and iterative manner so that all three objectives of the project are met optimally. The Project Management Plan and its sub-plans are designed to ensure that the three elements are managed in harmony, and any attempt to give priority to one element will normally impact the other elements adversely.
In real terms, cost and schedule are transitory elements of a project, while capability, the core purpose of any project, is enduring.
Finally, DMO's averaging or summing schedule over-runs is largely meaningless, and potentially misleading. Each project over-run has its unique impact upon particular area(s) of Australia's force structure and military capabilities, and hence our national security. The sum of any number of schedule over-runs must be dimensioned not in DMO's simple, numerical terms but in their individual and combined effect on the force structure required to underpin Australia's defence capabilities and security – an exercise that Defence/DMO seems reluctant or unable to do.
Such generic factors have to be qualified and quantified on a project by project basis to be of any use, and then managed properly within a strong, capability-based, project management framework. There should be few surprises of this type in a well-managed project, and the DMO should, by now, after a decade, be well past the generic risk/problems/issues stage.
However, the proper specification and management of capability within the Project Office demands a solid basis of skills and competencies in the technology involved. Such skills and competencies then need to function within robust project and engineering management systems so as to bring together and control system capability, schedule and cost in a coordinated way. Only the adoption of sound governance of a robust management structure can provide the timely, accurate and consistent status reports, across all projects, sought by ANAO and JPCAA. Analysis of MPR 2007-08 and 2008-09 shows clearly that such systems are not in place within the DMO, but have been resisted by Defence and the DMO for more than a decade. Three areas of concern have been highlighted by ANAO:
Within DMO, this approach is missing. For example, with the risk management General Guidance contained in the DMO document titled 'Liability Risk Assessment For (Insert Details)', issued by the Office of the Special Counsel, Procurement and Contracting Branch. This defines 'risk' as “The possibility of an event occurring” , and the word 'liability' is identified as “the obligation to pay for, or to pay to rectify the consequences, of the event when it occurs”. The guidance then includes DMO's standard risk management processes, emphasising the need for the treatment of 'extreme' and 'high' levels of risk to be cost effective, and 'low' and 'medium' levels of risk to employ analysis, intuition and experience in reconciling them.
This approach cuts across accepted project and engineering management functions and accountabilities, and suggests that contract liability is the primary focus and driver of risk management within DMO. The guidelines attempt to control contract liability rather than the (usually project/technical) risks that arise throughout projects. DMO's risk management thus appears as a contract/legal-centric process, rather than a project-centric management procedure.
Such confusion and intrusion only adds to project risk, prevents the timely identification of the risk, and may explain partly the continued 'dumbing down' of internationally-accepted risk management standards and procedures within DMO – in effect, by replacing objective, outcome-focussed management with subjective, inwards-looking process.
In highlighting the need for reviewing DMO's project management and reporting frameworks and the number of different frameworks in place that contribute to (or actually impede) good governance of the acquisition process, ANAO identifies a core problem also seen in this analysis. As cited in the MPR, DMO's governance over acquisition processes is:
“guided by policies and procedures that are regularly updated to support developments in project management and DMO's own experience. Projects are assigned to project offices that have the responsibility to manage the acquisition process. A range of systems and processes provide support to the project offices and allow for centralised reporting of key project information”.
This statement raises very important points that beg analysis. There are three main reasons behind Defence /DMO's being unable to manage major projects, or indeed any military capabilities, competently. These are:
In the absence of professional project and engineering management systems founded upon good governance, all policies, processes and reviews that have developed since Defence's arbitrary change in methodology about a decade ago have only reinforced the deeply flawed processes adopted by Defence/DMO, both organisations consistently resisting all criticism and pressures to return to a professional, competencies-based management structure. There is, thus, little wonder that ANAO and JPCAA are so concerned with Defence/DMO management frameworks and the lack of assurance on the project status data that is being provided.
At the heart of this is the practice of allowing project offices to follow a range of different acquisition systems and processes. In the absence of an overarching project management discipline, this works against accurate, timely and repeatable project management information and data being available across projects. On the other hand, project and systems management methodologies have been used internationally across all manner of projects - simply tailored in scope and depth to meet the unique requirements of each project.
Until DMO's projects come under sound, capability-based, project/engineering management, supported by the operational and technical skills and competencies required, and with contract management taking its proper role and place in the acquisition process, the problems being encountered by ANAO/JPCAA will persist.
The management of risk has presented a continuing problem for Defence/DMO, as indicated previously, because of a focus on risk in terms of contract liability rather than on project/engineering/technological risks that have to be identified, assessed, analysed and mitigated within those disciplines before their likely impact upon contract liability can be estimated with any confidence, and then mitigated.
DMO's ERMF fails to recognise the basic tenets of modern day risk management, such as:
Levels of risk that are at the ‘Extreme’ or ‘High’ levels are not to be trifled with, since, if left untreated, they will assure that the activity in which such levels of risk reside will fail, and likely in a catastrophic way.
Even a casual browse through the DMO Project Data Sheets reveals anomalies and inconsistencies with project management norms and common sense, as well as standard risk management practices. Risk in relation to the 15 projects covered by MPR 2008-09 is analysed further at Annex A.
Firstly, there are two types of longitudinal analysis:
“They (projects) are very complex. I would hope that over the next five or 10 years of this report you see a transition where things do not happen anymore or when we mitigate some of these risks”.
This approach is also reflected in DMO's comments on the Alliance contracting methodology to be used on the Air Warfare Destroyer Project, when the CEO DMO stated that 'this new contracting approach will have to be used on a number of projects before all lessons will have been learned'.
Furthermore, all longitudinal analyses depend for their accuracy and usefulness upon having project management and reporting systems in place that provide objective and well-defined data items that are common and repeatable across all projects. Only in this way is it possible to accumulate repeated observations of the same data items over time. DMO cannot meet these requirements, because:
DMO's Project Longitudinal Analysis is more about 'reinventing the wheel, less a spoke or two each revolution', on a repetitive basis, 'dumbing down' its systems and processes as it goes. Such a process has no end and will never yield the right answers.
The principal value true RLA is that it focusses upon the root causes behind the situation being encountered TODAY, from which changes in the management systems and procedures being used may be identified, specified, implemented and evaluated. The nexus between problem, cause and correction is kept as short as possible so that the effects of problems are headed off on current and future projects. This is the approach used, to the extent possible, in this and the many other analyses provided over many years to Defence and the DMO, as well as to successive Governments and Parliament, but to no apparent effect.
Mr Georgiou: “My difficulty is just the notion of lessons learned. I have just skipped over three or four of them and I could just about write a template for 'lessons learned' that would cover most of the lessons learned that are outlined here.” Mr Georgiou was right – the lessons learned were so simplistic and subjective that even the most junior and inexperienced of project managers should have known and avoided them.
DMO's response to Mr Georgiou's observation was largely an appeal to the large, complex and systems integration tasks involved: “They are very complex. I would hope that over the next five or 10 years of this report you see a transition where things do not happen anymore or when we mitigate some of those risks.”
after a decade or more of these generic and ubiquitous risks, the
best DMO can offer is yet another decade or so of 'learning lessons'
from 'experience' in the expectation that somehow things will
In response, the changes to DMO's business practices (Page 51) which have taken place are seen, by DMO, to have arisen from:
A key characteristic of this approach is the further diffusion of accountability. Mortimer, like so many others who recommend that DMO adopt a more 'business-like' approach, did not specify clearly what he meant or specify what specific commercial/business changes were thought necessary to achieve what ends. The two themes of the Mortimer 'reforms' are wholly commercial, that is, contract management centric, and ignore the core, capability and evidence-based, project and engineering management systems, skills and procedures that are critical to ensuring that the contract is made as low risk and successful as possible. The problems identified by ANAO arise directly from the absence of these project related systems, and will persist so long as these systems are not in place. It is not seen how the Mortimer 'reforms' can result in any improvement in DMO's ability to provide high quality advice to Government, or increase the rigour with which projects are assessed. As a result, Defence's Capability Development and Acquisition phases will continue to be inadequate, with their inevitable adverse effects on the defence and security of Australia, as well as the public purse.
'Strengthening the accountability of the Capability Managers' will also provide no improvement as the Service Chiefs are neither organised to accept such accountability nor do they have the skills and competencies base required. This has been reflected directly in the poor requirements baselines that have plagued projects, as well as the intractable problems associated with understanding the technology involved and its integration. Hence, DMO's reforms will merely become additional processes, serving only to make the current situation worse, as well as slower and more costly.
No recognition is made in the Mortimer Review of the need for capability-based project management and expertise relevant to the technologies being managed. No mention is made of the Project Management Plans, the Systems (and Integration) Management Plans, the Configuration Management Plans, the Maintenance Analysis Plans, the Risk Management Plans, the Test and Evaluation Master Plans, or the Support Analysis Plans, and so on, that form part of and drive every project. Contractors may provide input to them, but these plans cannot be outsourced to contractors without incurring the very generic 'Risks', 'Issues', and 'Lessons Learned' that populate the DMO's PDSs.
The practice of adding additional and higher layers of process simply passes problems up the line to people who have even less competent knowledge of the problem and its solution than those below, ensuring that judgements and decisions are taken at the higher level on grounds other than what is best for the project. Woven throughout this maze is the principle of non-acceptance of mistakes and the diffusion of accountability. Such a construct has proven to be inappropriate for the management of Australia's military capabilities and national security. Unfortunately, Australia's governance mechanisms, seemingly due to limited comprehension, or lack of political will, are not up to the task of identifying problems and turning them around.
None of the changes in process listed above will redress the effects that flow inevitably from the failure of Defence/DMO to follow standard project, engineering, risk management, and IV&V systems and procedures, or add to its ability to manage projects through contracts. The management approach reflected in the 2008-09 MPR has failed and will continue to fail until major structural and management changes are imposed.
Several of the processes listed above, and included in the Defence Strategic Review, relate to accountability in Defence/DMO. Accountability was also a central theme in the Proust Review into Defence Management (8), for example (Pages 32, 33):
The question of accountability is also raised, both directly and indirectly, in this analysis. To a great extent, accountability is an output of sound management. Put simply, sound management requires:
Over recent decades, sound management has not been a feature of government bureaucracies. Accountabilities have become diffused to the point of avoidance, which, when coupled with outsourcing policies that have led to a dramatic lack of expertise in the matters for which they are responsible, have resulted in departments being unable to provide sound advice to their ministers, and hence government and parliament, or to implement government policies competently. As Proust stated in her Summary of Recommendations: “Another threat to successful change is that Defence may passively and outwardly accept the Review but be less than rigorous in its implementation.” Defence shows symptoms of an organisation without a sound management structure, which means that it has no Control Loop, and so is incapable of measuring its own performance, or reforming itself.
In addition, the Department has already identified exactly how much can be saved in eight specified areas of activity, which means that management in those areas will be required to trim their budgets (i.e. activities) to meet the financial objectives that have been set. Because of the maze of organisational, functional and financial interfaces that now exist within and between the Services and the Department that supposedly supports them, there is little probability that the cascade effects of these 'economies' on other areas will be recognised – the Services will just be told to live with them.
Pre-DRP/CSP, such economy drives were known as 'Money must be saved at any cost!' exercises. Fortunately, at that time, the Services were organised and resourced such that they were able to retain core capabilities, competencies and standards while trimming activities to meet the passing financial constraints imposed. Today, neither Defence nor the Services have the ability to do this, so the Strategic Review will carry an extreme risk of further debilitating Service capabilities and degrading further Australia's defence and security.
A similar strategic review was conducted in the UK (10) with disastrous results, when it was found that “In defence, since there is no measurement of output, a cut in one area often cascades throughout the operation with unseen negative consequences”. The UK has also learned that such consequences can lead directly to loss of life and equipment. This was highlighted in the subsequent loss of an RAF Nimrod aircraft and its entire crew in Afghanistan (11). Some of the findings from the very thorough review that followed were:
Organisational Trauma 1998-2006.
The MoD suffered a sustained period of deep organisational trauma over this period, beginning with the 1998 Strategic Defence Review. Financial pressures and cuts drove a cascade of multifarious organisational changes, which led to the dilution of the airworthiness regime and culture within the MoD, and distraction from safety and airworthiness issues as the top priority”.
While the RAAF may have an Airworthiness Board, Australia's problem lies in the loss of the three Services' Technical Services Branches which planned and implemented those engineering and maintenance policies, systems and procedures which ensured that the activities that underpin airworthiness / seaworthiness / battleworthiness were in place and working properly across the whole Service.
Returning to the Nimrod Review:
“There was a change in culture and priorities in the MoD towards 'business' and financial targets, at the expense of functional values such as safety and airworthiness. The Defence Logistics Organisation, in particular, came under huge pressure. Its primary focus became delivering 'change' and the 'change programme' and achieving the 'Strategic Goal' of a 20% reduction in output costs in five years and other financial savings. Airworthiness was a victim of the process started by the 1998 Strategic Review”.
Two senior personnel who presided over the Defence Logistics Organisation during the critical period were appropriately named in the UK review as bearing particular responsibility.
'The switch to financial and business targets at the expense of functional values' mirrors well the situation that has been allowed to develop in Australia and now drives the Defence/DMO organisations.
is no doubt that the culture of the time had switched. In the days
of Sir Colin Terry you had to be on top of your airworthiness. By
2004, you had to be on top of your budget if you wanted to get
ahead”. (Air Chief Marshal Sir Colin Terry, Chief Engineer,
RAF, 1997 – 1999). Here we see money being used for functional
control, rather than a functional resource, and then only one of many
of equal or greater value.
“The continuous organisational change during the period 2000-2006 led to a marked dilution of the safety and airworthiness regime and culture in the MoD for three reasons. First, during this period there was an inexorable change in the MoD from a safety and airworthiness culture to a 'business culture'. Second, the organisational changes in the MoD led to a safety and airworthiness regime which was organisationally complex, convoluted, confused and 'seemingly dysfunctional'. Third, meanwhile, there was also a steady dismantling of some of the important features of the safety and airworthiness regime which had previously existed: (Selected items)
Abolition of the 'Chief Engineer RAF';
Dilution of air technical support services;
Dilution of aircraft engineering skills.”
The Nimrod disaster review should have been made compulsory reading for all Defence executives and senior Service officers involved with the Defence Strategic Review.
These scores attempt to replace the milestone reports that form the backbone of sound project management, but without the auditable, objective data items that project management reports require and provide.
The soundness of DMO's Project Maturity Scores rests wholly and solely upon the Project Manager's judgement, which, by definition, can only be subjective (that is, 'based upon personal feelings, tastes, or opinions' – Oxford English Dictionary). It is impossible for any subjective assessment to become an objective measure as DMO suggests (that is, one 'not influenced by personal feelings or opinions' – Oxford English Dictionary). DMO's Maturity Scores are thus not objective scores as stated, but simply unsupported opinion, likely open to cognitive or other bias in the interests of self and/or career preservation. To then compare them with an 'ideal benchmark' is merely to invoke further magical thinking.
DMO uses US and UK Defence acquisition systems as benchmarks, but both have extremely poor records; records that have been assessed as gross failures. The US system, for example, has been the subject of trenchant criticism since the mid-1980s when Congress decided that its Department of Defense and its Defence Procurement Organisation were broken, and undertook a range of reforms under the Goldwater-Nicholls Act of 1986. Regrettably, these reforms were sabotaged by very strong vested interests, and so the problems remain even more deeply entrenched today. However, the US national security situation has now reached the point where the President's State of the Nation Address declaration “Well, I do not accept second-place for the United States of America” has a decidedly hollow ring. The final confrontation between the US Department of Defense and Congress will most probably arise in the near future as highly critical JSF Project governance reports are tabled.
That DMO should see the UK Ministry of Defence (MoD) procurement organisation as a suitable performance baseline is surprising, as that organisation has been the subject of severe criticism for repeated failures to bring capabilities in on time, capability and cost. In the sustainment area, the organisation has been similarly criticised, with the MoD being accused of causing the unnecessary deaths of many UK Service personnel.
A recent paper titled 'UK Defence Needs' (10) records the MoD acquisition organisation as having an average equipment programme over-run of five years (many going over a decade), and very high cost over-runs. Current projects are some £35bn over budget. The paper gives details and references to other substantial analyses undertaken into the UK MoD procurement organisation, to little, if any, effect.
In comparison, of the 15 DMO projects covered in the 2008-09 MPR, eight project schedules slipped by a total of 378 months: including HF Modernisation (74), Collins RCS (72), FFG Upgrade (65), Wedgetail (48), ARH Tiger (42), Armidales (33) and Bushranger (26).
If DMO is prepared to accept the US and UK procurement organisations, and some Australian civil engineering projects, as performance baselines, it is difficult to accept that the organisation could ever construct an 'ideal score'. The bar is being set far too low for the defence and security of Australia.
The development of Project Management Scores may be seen as an inevitable result of putting generalists in management positions that require professional competencies and skills in Project and Engineering management. The scores and their benchmarking give an illusion of project management, but, in reality, have the substance of vacuous smoke and mist.
The United States and Australia were found to be the lowest performing countries with regard to equipment output for every dollar spent, ranking 33 and 32 respectively. While Defence has challenged the Mc Kinsey findings, the reasons given are not persuasive that the over-all approach and relative rankings are significantly wrong.
Of particular interest is the statement that 'the effects from any major change to contracting practices would not be evident until a range of new projects has used the new practices and sufficient time had elapsed to assess the effectiveness of the changes'. Common sense management would suggest that if you do not know what you are doing, then do not do it! However, Defence seems content for DMO to use capability procurement on a trial and error basis, in an effort to identify what systemic changes to contracting processes might be needed sometime in the future.
The following corrective actions have been put forward as responding properly to the lessons learned from the DMO MPR, but they will merely add further layers of process and will not get to the core management problems that caused the problems in the first place:
Tiger ARH (©2007 Carlo Kopp).
C-17A Globemaster (©2007 Carlo Kopp).
Bushmaster PMV (©2007 Carlo Kopp).
F-35A mockup displayed at Avalon Airshow (©2007 Carlo Kopp).
In retrospect, the New Air Combat Capability / Joint Strike Fighter (NACC/JSF) Project, should have been included in ANAO's DMO MPRs, or made the subject of a separate study, as this aircraft probably represents the most risk laden project ever undertaken by Australia; one which, if it proceeds as currently planned, will entrench Australia, at best, as a second-rate, minor military power both in our region and on the world scene, a nation in which coherent and competent defence planning is clearly beyond its competencies.
As this project ran into inevitable capability, schedule and cost problems over the years, we have seen the aircraft's capabilities extolled well beyond the limits of physics, credence, or even common sense. The aircraft was designed to face threats that are now more than a decade old, and by the time the aircraft enters service it will be at least two full generations behind the threats it will have to face.
However, successive Parliaments, Governments, Defence Ministers and the DMO have chosen to ignore, even denigrate, the substantial operational and technical criticism of both the aircraft and the way in which it is being mismanaged that has been provided by local expert analysts as well a series of highly critical US governance reports.
In brief, the aircraft to which Australia has mortgaged its military future, and that of many of its allies, is: (13)
That Defence should continue its unqualified support for the JSF, and recommend the purchase of 14 aircraft from the initial production run, in the face of official US Government warnings on project cost, schedule and capability, indicates poor or absent due diligence on the part of Defence/DMO, and an equally poor or absent governance oversight.
The JSF Project thus begs an urgent and independent management audit.
To a large extent, Australia's Defence/DMO organisations follow policies, structures and processes similar to those in use in the UK and USA, and use those countries as capability procurement benchmarks. However, recent developments in both countries have further emphasised the reasons behind the failure of the defence bureaucracies in all three countries to manage capability requirements and procurement matters effectively, efficiently, timely and economically.
Following its disastrous 1998 Strategic Defence Review, the UK, under severe financial pressure, has now embarked upon another Strategic Defence and Security Review which is due to report during October 2010. This, as for all such reviews, carries both an opportunity to correct what is wrong with the management of UK defence, and the risk that the Review will only make matters worse.
Defence Secretary Liam Fox has announced that Service chiefs are to be given more control over the armed services as part of a package designed to 'decentralise' the Ministry of Defence. This would provide a much-needed strengthening of the functional backbone of the Services, if done properly, but there is a definite fear that the review will merely entrench further problems because it will be far too rushed and is seen to be 'Treasury-led'. There is thus a high risk that the review will turn out to be just another cost-cutting exercise, not one based upon clear, functional military needs, and will simply inject yet another cascade of multifarious organisational changes and processes that will work against professional military capabilities, competence and morale, and only further erode UK and NATO security.
“The modern QDR originated in 1990, conducting a bottom-up review to free the Department from the constraints of existing assumptions and refresh the intellectual capital of the political leadership in Congress as well as the Executive branch.”
“The initial Bottom-up Review was considered a success. Unfortunately, once the idea became statutory, it became part of the bureaucratic regime. The natural tendency of bureaucracy is to plan short term, operate from the top down, think within existing parameters, and affirm the correctness of existing plans and program of record.”
“That is exactly what happened to the QDR process. Instead of unconstrained, long term analysis by planners who were encouraged to challenge pre-existing thinking, the QDRs became explanations and justifications, often with marginal changes, of established decisions and plans.”
“This latest QDR continues the trend of the last 15 years...but for the reasons stated, it is not the kind of long-term planning document which the statute envisions.”
“The fourth chapter of our Report takes on the issue of acquisition reform. We commend Secretary Gates for his emphasis on reducing both the cost of new programs and the time it takes to develop them. But we are concerned that the typical direction of past reforms – increasing the process involved in making procurement decisions – may detract from the clear authority and accountability that alone can reduce cost and increase efficiency.”
“The issues raised in the body of this Report are sufficiently serious that we believe an explicit warning is appropriate. The aging of the inventories and equipment used by the services, the decline in the size of the Navy, escalating personnel entitlements, overhead and procurement costs, and the growing stress on the force means that a train wreck is coming in the areas of personnel, acquisition, and force structure.”
“The potential consequences for the United States of a 'business as usual' attitude towards the concerns in this Report are not acceptable. We are confident that the trend lines can be reversed, but it will require an ongoing, bipartisan concentration of political will in support of decisive action.”
The Independent Panel also found:
On the Core Issue: The Panel saw the core issue as follows:
“The Panel's examination of the QDR found that the Department of Defense has been in a near constant state of acquisition reform since the implementation of Goldwater-Nicholls in 1986. These efforts have generally produced more structure and more process, but have not produced notable improvement in delivering required capabilities when needed at the expected cost.”
The Panel saw the critical path to improving acquisition performance
Contract Management. The Panel highlighted:
The Independent Panel's Report on the 2010 QDR raises many matters that demand urgent and effective action if the US is to 'keep up its global leadership role', as stated by the President, and so be capable of providing for the security of its allies in accordance with its many treaties, including that with Australia. However, there is little, if any, evidence that any of those responsible, either in Government, the Defense Executive, or industry, are prepared to undertake the major structural and accountability changes required.
Report 10-789 highlights the continuing erosion of the US's tactical air force capability which will soon drop (if it hasn't already dropped) below the required levels, and warns that capability shortfalls will persist for years because of continuing financial constraints.
This situation has arisen against a background where there has been no comprehensive, long-term planning by the US DoD to identify the evolving threat environment, specify the force structure needed and the capabilities required to meet identified gaps, and enable resources to be allocated in accordance with clearly-identified priorities. All recent research in this area has been funded privately and solely by think tanks, not by DoD.
In the absence of any long-term planning input, both the QDR and the GAO Report findings and recommendations, as well as DoD's Force Structure and Procurement Plans and priorities, must carry a high level of uncertainty, and thus risk to the security of both the US and its allies.
There is only one effective and reliable form of insurance against such risks, and that is the establishment and maintenance of technologically-skilled and highly professional military services, supported by a robust and flexible defence industry. Neither now exists, while Defence/DMO policies continue to erode what little remains.
Government and governance bodies are must consider carefully the analysis of DMO MPR 2008-09 contained in this paper, as well as the wider matters raised, as the primary risk to Australia's long-term security has now moved from external threats to the cumulative effects of Defence/DMO policies and processes.
As the QDR Independent Panel's Report so correctly observes: “We are confident that the trend lines can be reversed, but it will require an ongoing, bipartisan concentration of political will in support of decisive action.”
For over a decade, the performance of Australia's Defence Department and its Capability Acquisition Organisation have been subject to criticism politically, in the Press, and by defence analysts, for poorly selected, late, capability deficient and failed major projects that has resulted in a distorted and deficient force capability and the waste of billions of dollars.
While the root cause may be traced to the widespread reorganisation, downsizing and de-skilling of the Services, particularly the disbandment of their Technical Services Branches and the loss of their Support Commands, the acquisition organisation's move from a capability and evidence-based project management methodology to commercial/business-orientated processes greatly aggravated the effects of the loss of core competencies.
After some seven years of acquisition problems, The Auditor-General was called upon to produce an annual report on major Defence projects. The second report in this series, which covered 15 major projects, was presented in November 2009, but was strongly qualified on the grounds that it was not possible to obtain an appropriate level of assurance in the information presented by DMO.
Analysis of this report indicates that there is little, if any, evidence that long-standing, critical deficiencies in the project, engineering, risk management, and Independent Verification and Validation areas have been recognised or corrected. In brief:
In addition, organisational fault lines have been identified within the Defence organisation, identical to those now identified in the US, which make the organisation resistant to change, and dictate against Defence and DMO becoming efficient and effective organisations. There is thus a need to reform DMO in harmony with reform of the higher Defence organisation.
In the end, “it all comes down to a tale of war – a war not run by generals, but by comfortable men in air-conditioned offices who are indifferent to human suffering”.(16) Today, under civilian rather than civil control, Australia's military is run, at all levels, by well-entrenched, comfortable men in their air-conditioned offices.
This analysis aims to develop an acceptance of the need for urgent reform of Defence/DMO by restructuring the Defence and Service organisations, re-valuing (re-skilling) the Services, and re-imposing sound management and governance within and over Defence/DMO functions, realigning accountability with responsibilities and resource allocation.
However, the scope and the nature of the challenges identified in this analysis will call for strong, focussed and persistent cross-factional action, as national interests will have to be put before political considerations and executive pressures if Australia is to recover the professional military capabilities and competencies needed to face current and emerging threats.
A. Analysis of Project Data Sheets
B. Evolution of Management Approaches.
of the Project Data Sheets for each of the fifteen projects covered
by DMO MPR 2008 – 09 follows.
(The data contained in the following Major Project Analyses, and many of the points made in the body of this analysis, were contained in the author’s submission to the Australian National Audit Office and the Joint Committee Parliamentary Accounts and Audit in relation to DMO MPR 2008 – 09.)
Attachment 1 to this analysis provides a summary assessment of the consequences of the risks met, or likely to be met, in regard to each project in terms of both DMO and Australian/International Standards.
AIR WARFARE DESTROYER BUILD
Summary Comment on 2008-09 MPR:
The Air Warfare Destroyer Project is in extreme danger of failing for the following reasons:
Comment on PDS Content.
Although listed as an Acquisition Category 1 project, which means that it will require extensive project and schedule management, and have very high levels of technical difficulty, there is no mention of just how these challenges will be managed, resolved, and reported in an objective, accurate, timely and consistent manner.
Defence/DMO has chosen to trial an Alliance-based contracting methodology for this project on the basis of its having been used successfully on a museum and a highway project. Just how this approach is seen to be appropriate for a high-technology naval capability, with its unique systems engineering integration challenges, is not explained. The additional organisational and functional interfaces that will have to be negotiated must only add to project risk
Alliance contracting has become a popular funding model for government infrastructure projects, aiming to share the risk of cost over-runs between government (the taxpayer) and the private sector. In practice it seems to simply financial risk back to the taxpayer. A recently-commissioned Victorian Treasury study found that so-called alliance contracting costs blew out between 48% and 55%, the highest of the three funding models studied.
This finding is not surprising, as any contract, particularly those involving high technology systems, are prone to cost over-runs if the customer does not have the required project and engineering management competencies appropriate to the technology involved. Governments, and DMO, fall into this bracket, as they focus upon the wrong risks. Not having the project and engineering competencies required, they fail to manage properly the risks associated with capability, schedule and cost. They concentrate upon the risk (liability) to the contract without understanding that risks are embedded mainly in the technical (capabilities) and schedule areas.
This new contracting approach will, as stated by DMO, ‘have to be used on a number of projects before all lessons will have been learned’, so will be in a trial mode. This approach must be assessed as only adding new and unidentified risk to those projects upon which it will be used.
Within the complex management structure devised for this project, it is difficult to identify the project and other management skills and competencies that DMO will bring to the table.
The six major challenges listed are so elementary as to be hardly worth mentioning. DMO should, at this stage of the project, and after some ten years of experience, be able to better qualify and quantify those major areas requiring project, engineering and risk management focus.
DMO states: “Progress to achievement of planned in-service dates for the three ships and their support system is as scheduled”.
DMO states: “The current status is that planned capability will be achieved”.
Both schedule and capability assessments must be considered as being prematurely optimistic, subjective, and of little value, particularly in view of the significant, if unqualified and unquantified, problems lurking under ‘Major Project Risks’.
Major Risks, Issues and Linked Projects.
The items included in this section are too broad and are seen primarily from a contract aspect. Most are merely generic risks, lacking in capability, schedule and cost focus. The project would be difficult enough to manage if it were built in an experienced, working shipyard, but to expect a new and competent shipyard becoming available from scratch to meet project milestones, is beyond optimism. The skills and competencies that must be put in place have been greatly underestimated.
Finally, any project that ‘does not have an agreed Project Certification Plan and Certification Basis’ should not proceed as it embeds certain, if indefinable, risks. The absence of such basic pre-requisites for executing a minor, let alone major capital equipment procurement contract was the root cause of the SEA 1411 Super Sea Sprite debacle, the Collins Class Submarine misadventures and the continuing challenges within the Wedgetail AEW&C Project, to name but a few.
Have these lessons been learned? Clearly not.
Key Lessons Learned.
While hardly classifiable as a ‘lesson’, this section is important in that it warns “that it takes time and effort to develop the culture necessary to achieve improved outcomes”, and advises that an invaluable external facilitator has been engaged to help make the Alliance work. This attitude seems to fit into DMO’s ‘..hope that over the next five or 10 years of this (MPR) report you see a transition where things do not happen anymore or when we mitigate some of those risks”, as given in evidence at the JPCAA hearing into the 2007-08 MPR on 19th March 2009. Australia can hardly be expected to accept a continuance of these ‘things’ which impact directly the country’s security.
Project Maturity Score and Benchmark.
See body for analysis for an assessment of the usefulness of these scores.
While DMO sees both capability and schedule being achieved as planned, potential consequences of the risks inherent in this project are assessed as approaching Severe/Catastrophic.
BRIDGING AIR COMBAT CAPABILITY
(Refer also body of this analysis and Annex B)
The background to this project re-writes some of the history of its birth, but does classify it as a ‘directed government solution’. However, it is important that this project be seen as part of Australia’s New Air Combat Capability (NACC).
The Super Hornet, an aircraft rejected by other Western nations as a fighter replacement, was marketed robustly by the maker to Defence and Government and was purchased abruptly by the Minister of the day as a ‘no brainer’, against the advice of Air Force. Both Government and Defence had been provided with substantive data and analysis that shows that the aircraft will not be able to guarantee regional air superiority, as required by government policy. However, Manufacturer’s promises were accepted without question, even where they conflicted with the basic laws of physics.
As a result, the decision was criticised severely by the Leader of the Opposition and the shadow Minister for Defence before the last election, both promising to rectify what was wrong, and many hoped that this would occur.
The new Minister for Defence, Fitzgibbon, soon initiated a two-part review of Defence/DMO New Air Combat Capabilities planning – the first covering the decisions taken in regard to the unnecessarily early retirement of the F-111 and the decision to purchase the Super Hornet – Part 2 would later review the Howard Government’s JSF decisions.
Part 1 of the review was conducted internally by Mr Neil Orme of Defence. Mr Orme was provided with substantial personal and written submissions by individuals and Air Power Australia, which analysed and detailed the areas of risk associated with the New Air Combat Capability decisions being taken by Defence, including the JSF Project. He also had the many US governance reports issued over the years to guide him. These submissions and reports were simply unacknowledged and totally ignored, despite continuing assurances that Defence welcomed open discussion of its plans and programs. The failure of Defence/DMO to take note of the extreme risks associated with the aircraft may be measured by comparing the content of the Orme Report and the presentations made to him.
Since then, Government, Defence, and DMO have been kept abreast, in detail, of the many areas of risk associated with the JSF Project, but no risks were admitted by Defence/DMO or apparently by the over fifty ‘specialists’ involved with the JSF and NACC projects. All Defence/DMO statements have merely repeated the clearly discredited statements emanating from the manufacturer and the US Project Office which have always marched in ‘lock step’. The JSF Project is analysed in more detail in the body of this analysis.
The Minister’s decision to remain ‘Hornet Country’, an aircraft ignored by other nations in search of a high capability air combat aircraft, and Defence/DMO’s blind commitment to the JSF will not only enshrine Australia’s inability to exercise air supremacy in our region, but will condemn Australia’s management of defence capabilities in the eyes of the world.
The Key Lessons Learned simply record basic knowledge that the RAAF possessed some 70 years ago.
The Super Hornet Project continues the destruction of Australia’s military capability skills base:
This is the inevitability of Defence/DMO policy to let supply and long term support contracts to foreign companies having no business reason to set up support facilities in Australia. Defence/DMO policies are based upon a perceived reduction in cost and risk, which will, in the end:
This project still carries capability risks with the potential consequences that could reach a level of Severe/Catastrophic.
This project was entered into too early and planned inadequately due to a lack of in-depth experience with helicopter technology and an absence of sound project management disciplines. This parallels problems observed in Europe with the same basic NH90 design, detailed in the February, 2010 disclosure of the highly critical BundesWehr Luftlande- und Lufttransportschule (Airborne and Air Transport School) operational suitability assessment report, which recommends “using alternative aircraft whenever possible in an operational scenario”.
This is highlighted in, for example:
This project seems to have been driven by schedule at the expense of capability and sustainability.
Maturity Scores and Benchmark.
A total score of 57 against a benchmark of 57 hardly reflects reality. The question of the value of these scores is analysed in the body of this paper.
Although mostly identified as “There is a chance...”, the major risks listed do give a better insight than many projects. However, they are still at too high a level and are all problems that should have been identified and managed (fixed) under standard project engineering procedures.
Each ‘risk’ begs the question:
‘How and why did this happen?”
The answers to that question would have led to the real ‘lessons learned’ being identified and the appropriate corrective action taken for that project and for future projects.
Finally, the remedial actions given for each of the ‘risks’ identifies a lack of understanding and application of the standard project/engineering management procedures that must be applied rigorously and consistently on every project. In the absence of responses to the question raised above, the lessons learned provide no insight as to why these ‘risks ’ and ‘issues’ occurred and how they might be corrected for the future.
This project still
carries risks with potential consequences of Severe/Catastrophic.
AIRBORNE EARLY WARNING AND CONTROL AIRCRAFT
“Integrated system performance is currently not meeting specification. … However, remediation of all radar performance shortfalls is not expected to be achieved by final delivery of the system”. The project is some four years behind schedule, so should be able to provide many useful ‘lessons learned’.
Major Project Risks.
As for so many other projects, the risks recorded are generic and have not been tested by the simple question: “How and why did this happen?” so as to get to their root cause(s) and identify the corrective action needed.
All ‘lessons learned’ fail to provide answers that can be used to modify management systems and procedures to prevent future occurrences, as they are couched in subjective, generalist terms rather than objective, management terms. The project still reflects inadequate management systems and technical expertise.
Contractor Support - Good Value for Money?
Considerable concern is felt in regard to the Minister’s recent statement regarding the Wedgetail Through Life Support Contract. The fundamental question relates to how DMO determined that the contract provided good value for money. When the RAAF managed its own operating, intermediate and deeper maintenance facilities and contractors, determining contract scope and value for money was relatively straightforward and accurate. As DMO cannot fall back on that hands-on experience, the way in which DMO determines whether Australia is getting value for money needs to be better explained and made transparent.
A first look by an ex-RAAF engineer officer with project and maintenance management expertise points out:
While some may argue with the comparative scope and costs quoted, the figures give a reasonable comparison of value for money. It is up to DMO to explain transparently just why it considers the AEW&C Contract to be good value for money. One has also to factor in the reduced visibility and control of activities that impact directly force capability readiness and sustainability that come with DMO’s support contracts, as well as the dwindling skills base of the Services and our defence industry base, and Australia’s self-reliance.
The consequences of the risks inherent in this project have been at the Severe/Catastrophic level for many years.
The Project Summary states that the project is within budget, on schedule, and delivering the required capabilities. The project thus gives itself a maturity score of 45 against a benchmark of 45.
However, the optimistic Summary above has to be read against the five major risks being encountered, covering potential regulatory changes, changes in requirement, inability of the combat system to meet performance requirements, possible damage to propulsion pods, and inadequate funding for sustainment.
Each of these should have been qualified and quantified in terms of problem or risk, and appropriate project/engineering/risk management actions planned and taken. The piecemeal remedial actions proposed will prove to be inadequate in the absence of such management actions.
Major Project Issues.
The integration complexity highlighted will almost certainly test the management of this project, but the remedial action proposed does not generate confidence that the management of engineering complexity is understood. The project, at this stage must thus be given a low probability of succeeding.
The consequences of the risks inherent in this project have the potential to reach Major, approaching Severe/Catastrophic levels.
ARMED RECONNAISSANCE HELICOPTER
This project has experienced the range and type of difficulties that usually stem from incoherent project/engineering/risk management.
It is noted that $A6.5 million has been ‘harvested’ from this project, but whether this forms part of the Defence Strategic Reform Program is not revealed, nor have the impacts that will arise from the cuts.
The detail on this important project does not make good reading. It reinforces the impression that the project was scoped inadequately, and not resourced and managed properly. Where ‘project planners’ are mentioned, it appears that their functions relate more to contract administration than coherent project/engineering/risk management.
The major challenges listed reflect tasks that should have been handled in the project management sphere, the systems engineering sphere, and the support engineering field respectively.
For a project that is said to be 27 months later than originally planned, with some major elements up to 62 months late, has been re-baselined, and is still facing considerable challenges, to be assessed as “still expected to deliver the required capability within the approved budget” stretches credibility.
Risks, Major Issues and Lessons Learned.
Again, these sections give insufficient insight as to how and why these risks and issues arose, and the lessons learned do not get to the nub of the problems so as to be able to correct them in a timely way and feed changes back into management systems to ensure that they are managed properly on future projects.
The risks inherent in this project have reached the Severe/Catastrophic level.
AIR TO AIR REFUELLING CAPABILITY
“Airbus Military’s ability to meet the contracted schedule milestones continues to be the greatest challenge due to an underestimation of the overall scope and complexity of work and system improvements introduced during the development”.
This situation is indicative of a project that was embarked upon when inadequately scoped, planned and resourced with the required skills and competencies, and proceeded without sound project / engineering / risk management systems and procedures in place. The usual over-reliance upon contract management and contractor support for tasks that must be managed by the customer is also evidenced.
Major Risks, Issues, and Lessons Learned.
The risks and issues listed provide better than generic statements and highlight a number of risk/problem areas that should normally have been identified in time and mitigated through standard project/engineering/risk management procedures before they emerged as real risks/ problems/issues.
There is still a confusion between risks, problems, and ‘issues’, the latter term these days being used to describe any number of unqualified and unquantified ‘things’. The word ‘issues’ should be discouraged at all levels in project management.
Again, the risks and issues listed have not been tested to identify the root cause(s), and so the ‘lessons learned’ are not of much use in the practical project management sense of ensuring that they do not cause problems for future projects.
The risk consequences inherent in this project are at the Major level, approaching Severe/Catastrophic.
HORNET F/A-18 UPGRADE – PHASE 2
This project faces several ‘risks’ which seem to be managed through an amalgam of project office and contractor activities.
The key lessons learned reinforce this impression, as the majority of the activities listed for contractors and Product Teams are normally core project management functions that rightfully belong to the customer, the DMO Project Office. This arrangement also highlights the many different approaches adopted by DMO, which are more contract than project management focussed, giving rise to the problems faced by ANAO/JCPAA in trying to get objective and consistently accurate and consistent project status information across projects.
A key concern with this project, as well as Phase 3.2 of the Hornet Upgrade Program, is the cost effectiveness of the work being done as an element in Defence’s NACC planning, faulty as that planning might be.
Notable by its absence is any mention of the failed approach taken by the DMO in Air 5376 Phase 2.3, against strongly founded technical advice from within Defence and Industry, when the then Head of Electronic Systems Division of the DMO, with the support of the then Under Secretary of the DMO, ensured the selection of an extremely high-risk Radar Warning Receiver (RWR) System (the ALR-2002) over the recommended ALR-67 Version III RWR system, which is only now being installed into F/A-18 Hornet aircraft.
This decision resulted in over 5 years and some (estimated) $A 440 million dollars of project capital budget funds being wasted, plus the personnel, travel, subsistence and related costs associated with the ‘project management’ of this solution. An estimate of the overall wastage of Commonwealth resources would put the amount, conservatively, at well over half a billion dollars.
Major Risks and Issues.
Two major risks/issues are recorded for both Phase 2 and HACTS under ‘There is a Chance...’. Remedial action in all cases is contract centric. Under project management procedures, each ‘risk’ would be classified as a problem or a risk and if a risk, the classification of that risk and the manner in which it will be managed under project management procedures.
Key Lessons Learned.
These generally relate to what contractors are to do and contract management activities. Many of the activities recorded are DMO Project Office functions, as DMO carries accountability for the visibility and control of all project/engineering/risk/IV&V activities.
C-17 GLOBEMASTER HEAVY AIRLIFTER
This project lost about a year because essential operational equipment was not delivered with the aircraft, as it would normally have been under pre-DRP/CSP RAAF management.
This failure indicates poor or absent project management procedures in that support requirements were either not identified in the operational requirement and/or support requirements specification, and were not included in the Project Management Support Plan.
Again, the risks stated and the lessons learned do not identify just how and why these things happened and how they might be avoided on future projects.
It should not take “a couple of years to develop up the sustainment” as DMO advised the JPCAA Hearing of 19th March 2009. Pre-DRP/CSP, RAAF showed how sustainment requirements can be integrated into Project Planning and managed such that all sustainment requirements are in place by the time an aircraft/system arrives in country.
Finally, the support contract entered into for this aircraft should be tested to see whether it provides value for money. The USAF certainly found its contract far too expensive and sought changes. Is DMO’s contract costing too much?
This project, although purchased through FMS arrangements, has still achieved a risk consequence level of Major.
GUIDED MISSILE FRIGATE UPGRADE IMPLEMENTATION
This project experienced serious problems which DMO failed to identify in nature, extent, and impact, so that timely and effective action was not taken and the real lessons learned were not identified and used to good effect on this or future projects.
Delays in achieving Initial Operational Capability (IOC) are recorded as being 55 to 71 months, and Final Operational Capability (FOC) from 48 months to 65 months, and then only after reaching a ‘pragmatic agreement with Navy’, the nature and impacts of which are not identified.
Major Risks and Issues.
Although all assets are recorded as having reached their FOC during December 2009, this section reports that “There is a chance that...”
The remedial actions given are largely contract and outsourcing orientated. They reflect no sense of any firm project management control; for example, there is no sign of those core project management functions that relate to project planning, developing and managing the design and configuration baselines, or support requirements determination (there are many others of course).
Against this background, a maturity score of 57 against a benchmark of 57 would, prima facie, seem difficult to justify.
Key Lessons Learned.
These lessons, as for the other projects reported, list the most basic competencies, without which no project should ever proceed. They are all indicative of a lack of even a basic understanding of project management and the competencies required. Peppered throughout this list are what contracts should contain and what contractors should do – hardly a word about what DMO should be doing to overcome the perceived ‘risks and issues’; nothing about project/engineering/risk/IV&V management deficiencies within DMO. This project, like many, failed because it wasn’t managed properly by people with the required management structures and competencies – it was managed by generalists using contract and business processes.
There is much to learn from this project, especially with the Air Warfare Destroyer Project coming along, as well as early planning for a Collins Submarine replacement. However, with DMO not being a true learning organisation, and being focussed on contract administration and process rather than project management and engineering, what is seen in these PDSs can be expected to continue.
This project has carried a risk consequence of Severe/Catastrophic for many years.
HORNET Phase 3.2
The comments relating to the Hornet Upgrade Phase 2 Project relate also to this project.
Major Risks and Issues.
The Major Risks and Issues recorded relate to the unknown impact of fatigue damage and the lack of maintenance managed items to support the new aircraft configuration. The latter points to a failure to maintain close attention to the changes taking place in the aircraft’s configuration baseline – a project management responsibility, to calculate the variations to fly away kits spares holdings, and maintenance pipelines, and so ensure that timely notice is given of the new supply requirements - all tasks previously done by RAAF as a matter of course.
Of major longer-term concern is the fatigue state of RAAF aircraft and the soundness of future fatigue monitoring programmes resulting from DMO policy to include fatigue management in the maker’s sustainment contract. Internationally, RAAF and DSTO had an envious reputation for the fatigue management of Australia’s military aircraft, a function now dispersed and outsourced. Defence/DMO will, hopefully, learn quickly that aircraft manufacturers face a problem in monitoring the unique fatigue spectra, fatigue damage and fatigue lives of each RAAF aircraft. The results can, in the extreme, impact flying safety (airworthiness), as well as Australia’s air combat capability. Fatigue management of Australia’s military aircraft should remain in Australia and be managed centrally by Australians.
The lessons learned with the RAF Nimrod disaster are central to the inevitable effects of Defence/DMO’s change of policy on fatigue management.
Key Lessons Learned.
These miss somewhat the objective of recording lessons learned for use in refining/changing management policies, systems, and procedures.
These will emerge over time, but they will emerge.
BUSHMASTER PROTECTED MOBILITY VEHICLE
The Critical Design Milestones for the Vehicle Initial Review run from 12 to 26 months late, depending upon vehicle type. System Integration now runs from 6 to 26 months late, while contractor Test and Evaluation progress is from 6 to 37 months over time.
Major Risks and Issues.
As reported, “There is a chance” that delivery of operational capabilities may be delayed by:
The remedial actions given consist mainly of contractual activities and enlisting ‘stakeholder’ involvement. However, standard project management procedures should drive approved operational and technical requirements in a closely coordinated way, and in turn monitor contractor performance in meeting those requirements at specified milestones. Variations to project planning, including those recorded as risks and issues, are then coordinated and controlled through the project management system, which in turn drives any necessary contractor/contract activity. This project seems to suffer from inadequate or ineffective project management, allowing the contractor to control the project.
The fact that “Thales has provided an undertaking to consult the Commonwealth where any potential schedule conflict arises from other customer inquiries” reinforces this impression.
Major Project Issues.
These relate to:
Three of these four would be expected to have come under standard project management procedures. The fourth, the lateness of the facilities points to one of many unnecessary and risk-laden interfaces that exist in all Defence/DMO processes.
Key Lessons Learned.
The three key lessons learned are as recorded, as follows:
These all indicate a lack of the most basic competencies required before a project can even be considered, and a total absence of any project management framework. That this project was allowed to proceed with these deficiencies is damning. It was planned to fail. In short, too much reliance upon contract administration and inadequate project/engineering/risk management, and no IV&V!
This project has reached a risk consequence of Severe/Catastrophic.
Although this project records a delay of some 74 months for both Initial and Final Operational Capabilities, and 127 months for the Mobiles, the project still earns a DMO Project Maturity Score of 54 against a baseline of 55 – whatever that means.
The two additional risks included in this report relate to the Fixed Network’s failure to meet the contracted Grade of Service and Speed of Service, as well as inadequate software design documentation. Both problem areas are to be managed by ‘working closely with, and monitoring, the contractor, both passive activities’. There is no sign of any active project management here.
Major Project Issues.
The addition items included in this report covered:
These are all indicative of a less than robust project management system.
Key Lessons Learned.
These remain unchanged from the 2007-08 MPR.
This project is symptomatic of one starting without an adequate requirements baseline, inadequate or missing project management planning, inadequate engineering skills and competencies, inadequate risk management, and no IV&V plan. Despite this, the project was allowed to proceed, relying solely upon contract administration.
This project reached a risk consequence level of Severe/Catastrophic many years ago.
ARMIDALE CLASS PATROL BOAT
The project seems to have been an attempt by DMO to be a ‘smart’ buyer by acquiring a commercial design that, seemingly, had only to be modified to meet RAN operational requirements. The chosen vessel however also required Navy to change its traditional crewing policy to fit the constraints of the commercial design.
The vessel has encountered serious design defects which have led to it being virtually ‘black balled’ by RAN sailors on both operational and safety grounds. Needless to say, Defence/DMO and the RAN hierarchy have lauded the vessel.
Despite this, DMO’s PDS states that “All vessels continue to meet Navy’s operational requirements”.
Progress to FOC.
The project is now 33 months late, due to outstanding defects. FOC is now estimated to be 2011.
Major Risks and Issues.
These are recorded, as follows:
One would expect Navy to have had such risks well in view and under tight control at least ten decades ago.
Both of the Key Lessons Learned are simply generic, and are symptomatic of a failure to follow established project management, engineering management, risk management, and IV&V methodologies from the beginning of this project to the end. This project highlights the the capital risks accepted by DMO in embarking on a project of medium complexity, relying wholly upon contract administration for its management. The project has cost Australia much in time, money, and compromised and late capability requirements, but there is no sign in this PDS that any real lessons have been identified and digested.
This project has reached a risk consequence of Severe/Catastrophic.
COLLINS REPLACEMENT COMBAT SYSTEM.
The project is recorded as now being six years behind schedule, which represents a significant part of the boat’s life of type.
The operational capability of the Collins submarine depends upon the quality and reliability of its Combat System. One would thus expect everything possible would be done to select and commission the best system available. Unfortunately, for Australia’s security, and the risk to those RAN members who will man and operate these boats, and be exposed to the hazards of war when required, the management of the Collins Combat System has to be condemned.
In response to the dilemma, DAO/DMO had reached in relation to the Combat system originally specified, an innovative South Australian SME accepted the risk and developed a combat system for the Collins submarine that was provided to Defence and has been installed on one of the boats for years.(1) This Australian designed combat system has never failed, can do more than the system selected, and costs a fraction of the very expensive American system. (2)
However, the locally designed system was ignored. Defence, DMO and Navy selected the American design because the US Navy threatened to cut off Australia’s access to intelligence information if they did not commit to the American system. The American system has cost Australia hundreds of millions of dollars in addition to the hundreds of millions of dollars wasted on the original failed system. The system selected still fails to meet requirements, even after these were downgraded to ‘help it across the bar’.
In the case of the Collins Combat System, Defence/DMO have weighed in their balance whether to accept a second rate combat system at remarkably high cost so as not to alienate our sister nation (or one of its Services), against accepting a locally-developed system that has proven to be far more capable and reliable than the US system, far less costly, and able to be developed further and supported in Australia. That they have selected the former alternative indicates clearly that the security of Australia and the safety of our Naval crews run a poor second on their scale of values.
This project reached a risk consequence level of Severe/Catastrophic many years ago.
As with other Australian and International Standards, DMO has re-invented those Standards relating to risk. For example, the Australian and International Standards define the highest level of consequence as ‘Catastrophic’, while DMO defines it, euphemistically, as ‘Severe’. At this risk level, the consequences are defined by DMO as:
DMO then defines the next highest level of consequence is defined by DMO as ‘Major’, the consequences being as follows:
The inclusion of media criticism as a risk consequence leaves the whole risk management process open to bureaucratic distortion/misinformation on the grounds that the truth may ‘damage’ the Department’s reputation. For example, the manner in which late/over-costly/ capability deficient projects might be managed.
Within both Major Projects Reports to date, no reference is made to any actual risk levels and consequences in accordance with any Risk Standard. A review of DMO MPR 2008 – 09 against DMO’s own risk standard follows.
The Evolution of Management
Over the past 30 years or so, Western businesses have changed their focus from manufacturing to marketing, calling for a change in the thinking of their workers and the education system that provides them. The product has become less important than process and process is optimised by means of management techniques that function on a level different from those used for production.
“The contemporary office thus requires the development of workers ready for ‘Teamwork’, rooted in shared habits of flexibility rather than strong individual character. At issue in the contrast between office work and the manual trades is the idea of individual responsibility, tied to the presence or absence of objective standards.”(1) To ‘manual trades’ may be added any activities involving the operation, sustainment, or replacement of technology.
The development of modern, knowledge-based management methodologies and their conflict with objective job satisfaction and the sound management of technology has been well analysed by Matthew B. Crawford, who has experienced the problem both as a manager and a worker, in his book ‘Shop Class as Soulcraft – An Inquiry into the Value of Work’ (1). The following observations and conclusions draw upon his findings, quoting from his book.
“Those who work in an office often feel that, despite the proliferation of contrived metrics they must meet, their job lacks objective standards of the sort provided by, for example, a carpenter’s level, and that as a result there is something arbitrary in the dispensing of credit and blame. The rise of ‘teamwork’ has made it difficult to trace individual responsibility, and opened the way for new and uncanny modes of manipulation of workers by managers, who now appear in the guise of therapists or life coaches. Managers themselves inhabit a bewildering psychic landscape, and are made anxious by the vague imperatives they must answer to. The college student interviews for a job as a knowledge worker, and finds that the corporate recruiter never asks him about his grades and doesn’t care what he majored in. He senses that what is in demand of him is not knowledge but rather that he project a certain kind of personality, an affable compliance. Is all his hard work in school somehow for show – his ticket to a Potemkin meritocracy? There seems to be a mismatch between form and content, and a growing sense that the official story about work is somehow false”.
On the other hand, those who manage technology, and the technicians who work to keep it functioning efficiently in support of most human endeavours, “must reckon with the infallible judgement of reality, where one’s failures or shortcomings cannot be interpreted away, His well-founded pride is far from the gratuitous ‘self-esteem’ that educators would impart to students, as though by magic....His experience of failure also tempers any conceit of ‘mastery’”. The technologist’s work is thus based upon a real understanding rather than the ‘fantasy of mastery’ that permeates much of our thinking today.
The objectivity standards of the technologist also stand in contrast to the marketeers, those knowledge workers skilled in the art of persuasion, and demagogues who offer magical hopes. Plato pointed to the difference between technical skill and rhetoric on the basis that rhetoric “has no account of the real nature of things, and so cannot tell the cause of any of them”.
Aristotle also made a point in regard to experience, another characteristic of a skilled technologist:
“Lack of experience diminishes our power of taking a comprehensive view of the admitted facts. Hence those who dwell in intimate association with nature and its phenomena are more able to lay down principles such as to admit of a wide and coherent development, while those whom devotion to abstract discussions has rendered unobservant of facts are too ready to dogmatise on the basis of a few observations”.
Crawford sees that the vulnerability of managers today is reflected in their highly provisional manner of feeling and speaking, which is characterised by an imperative towards abstraction designed to help them cope with the physic demands of their job. Importantly, authority is seen to be embodied in those up and down the hierarchy with whom he works; indeed, his career depends upon these relationships where the criteria of evaluation are ambiguous. Managers thus spend much time managing what other people think of them, with the result that much of the job consists of “a constant interpretation and reinterpretation of events that constructs a reality in which it is difficult to pin blame on anyone, especially oneself.”
This has given rise to the art of talking in circles, presenting mutually contradictory statements persuasively and forcibly, allowing the manager to ‘stake out a position on every side of an issue, or burying what he wants done in a string of vaguely related descriptive sentences. What people say in such an organisation is generally accepted as always being provisional. Nothing is set in concrete – objectivity is the first victim. Within groups, the boss’s ‘deniability’ must be protected by using empty or abstract language to hide problems, thereby keeping the field of subsequent interpretations as wide as possible.
“When a manager’s success is predicated on the manipulation of language for the sake of avoiding responsibility, reward and blame come untethered from good faith effort. He may then come to think that those beneath him in the food chain also can’t be held responsible in any but arbitrary ways”.
Technology and its proper management is in fundamental conflict with the management approaches that have evolved in both public and business organisations, as well as the system of education that supplies these organisations with suitable workers – ‘those pliable generalists totally unhampered by any specialist skills’.
The Military/Bureaucratic Interface (2)
Mr Smith, just before taking up the position of Secretary for Defence, spoke of the military / departmental interface in the following terms:
“It is self-evident that the very different natures of military and civilian service produce different cultures, and it is important that those differences be recognised and understood if the two groups are to work together effectively. To mention just a few of these differences, civilians are, for instance, generally more readily able to tolerate, and even be comfortable with, unclear lines of command, divided authority, and open-ended guidance or ambiguous instructions. They are also tend to be willing to offer judgements and opinions on the basis of less hard data than their uniformed colleagues, and to accept that outcomes can’t always be readily predicted or easily influenced. Again, the question of ‘ownership’, so important to military commanders who very understandably want to ‘own’ or have command of the assets needed to do the tasks for which they are responsible, is much less important to civilians, who are generally more comfortable about being dependent on others to deliver results. Approaches to careers and service are also, inevitably, different and so of course are conditions of service and expectations from the service of which they are members.”
Military professionals will see immediately the incompatibility between the characteristics of the military professional, who depends upon high levels of skills, sharply-defined tasks, clear accountabilities, objective measures of performance, and sound management of the resources needed to achieve them, and the civilian approach. How can the military exist or perform professionally in an organisation that accepts vague, tolerant, unclear lines of command and divided authority, as well as open-ended guidance and ambiguous instructions? The military is about conducting its tasks professionally and on time, with least risk, learning from mistakes and accepting accountability for the results; the civilian bureaucrat is about burying problems, diffusing tasks and obscuring and avoiding accountability. Within such an organisation, transparency and public accountability soon become victims. Unfortunately, the civilian approach now permeates the Services, especially at the senior levels, as well as the activities of Defence/DMO, and must be judged a failure in any organisation that has to produce and maintain a capability.
Those familiar with the management of technology – its operation, sustainment, and replacement - will recognise that the principles underlying professional military management are much the same as used to manage technology. Both rely upon high levels of skills and competencies, robust management systems, procedures, and plans having effective feed-back loops, the continuing management of risk and the positive use of mistakes to learn, clearly stated accountabilities and objectives, and auditable reporting mechanisms. All of these are incompatible with the non-military and non-technical management methodologies that have grown up in business and public organisations over the past several decades.
The Root Cause.
Where the military and the management of the technology it operates and sustains, meets an unskilled civilian bureaucracy entirely unversed in the management of technology, a major fault line must inevitably develop which will fail repeatedly to meet the need for effective and professional management of Australia’s Military Services and National security.
It is proposed that such a fault line has been introduced into Australia’s higher defence organisation and is a root cause for the problems being encountered by Defence and DMO.
Air Power Australia Analyses ISSN 1832-2433
|Artwork, graphic design, layout and text © 2004 - 2014 Carlo Kopp; Text © 2004 - 2014 Peter Goon; All rights reserved. Recommended browsers. Contact webmaster. Site navigation hints. Current hot topics.|