This article is an excerpt from The Art of Tendering: A Global Due Diligence Guide, which is available for purchase.
In its 2019 report entitled The Australian Criminal Intelligence Commission’s Administration of the Biometric Identification Services Project, the Australian National Audit Office (ANAO) found that the Australian Criminal Intelligence Commission (ACIC) failed to properly manage a complex information technology project. The audit dealt with the cancelled Biometric Identification Services project, which was intended to replace the existing national fingerprint identification system, while also adding new facial recognition capability to the law enforcement technology. That project was ultimately cancelled due to contract performance failures, which led to the audit review.
The ANAO found that the administration of the $34 million (AUD) contract was “deficient in almost every significant respect” and that “[n]one of the project’s milestones or deliverables were met.” In its key messages, the audit noted that major projects need to establish proper governance structures so that appropriate technical expertise is integrated at the planning stages to enable the creation of proper performance milestones for future contract administration and record-keeping:
Governance and risk management
When managing a project of this nature, it is important that sound governance arrangements are in place, that have full oversight of progress, risks and mitigation plans, contingency planning and design and delivery challenges.
An important element of governance is assurance mechanisms at each major decision making milestone — such as agreeing final business requirements for tender, or the technical deliverables in the contract — where the officer signing off tender scope or the contract has sufficient assurance that it contains all necessary business requirements, particularly those that are critical to the effective operation of the system or product. This assurance can come through adequately broad and deep consultation, assurance committees or technical advice.
Where the project is significant relative to the size of the organisation’s budget or capability, then the project risks should form part of the broader organisational risk management structures and governance arrangements given the impact on the organisation if risks were realised.
Contracts must be clear in terms of deliverables, milestones, performance measures and accountabilities, and the entity should have strong contract management capability in place with clear reporting lines.
Further, the entity should ensure that it obtains the right technical expertise such that risks, design challenges and contact deliverables are well understood and the negotiation position of the entity is evenly balanced with the successful tenderer.
Given that personnel can change and machinery of government changes can occur, records are a critical part of informing future decision making and transparency and accountability for past decision making.
Sound record management procedures should be in place not just for major projects but for all entity business transactions and decision making.
However, the ANAO found flaws in the original design-planning for the solicitation, which led to significant gaps in the development of technical requirements for the awarded contract:
Design and planning of the tender
2.18 The ANAO found evidence that two important requirements were overlooked in the requirements development phase, relating to assumed identities (AI) and witness security (known as WitSec).
2.19 Assumed identities are ‘invented’ identities given to police and intelligence officers (with official approval26) to allow them to operate ‘under cover’. Witnesses who have agreed to give evidence in the prosecution of serious crimes may be provided with police protection under the National Witness Protection Program. In both cases, it is critical to such people’s safety (and potentially their lives) that information (especially facial images) which might reveal either their true identity or their location, are closely protected and not compromised.
2.20 On 9 February 2018, the ACIC CEO advised the ACIC Board that the additional cost of the necessary work to have AI and WitSec included in the contract would be ‘approximately $10 million’.
2.21 In May 2018 (more than two years after the project commenced), various ACIC officers recognised that AI and WitSec were never adequately captured in the contract.
2.22 The ANAO sought ACIC’s explanation of the omission of AI and Witsec from the requirements gathering phase and the contract. In August 2018, ACIC confirmed that the lack of specific AI requirements in the contract was an oversight.
2.23 Other items were also overlooked. An internal email, dated 1 May 2018, said:
AI was not the only requirement overlooked by PwC’s requirements gathering exercise in the very beginning. It appears that the BIS project never asked for, nor were provided a UI [user interface] spec[ification]. Consequently there are, at last count, 15 data elements missing from the BIS tenprint browse screens that currently appear in NAFIS. Some of these are business critical. These will be highlighted, however without documented requirements, we have no mechanism to effect change. It will be interesting to see how this shakes out.
2.24 Neither of these omissions was corrected before BIS was terminated in June 2018.
The ANAO also found that the project team failed to properly manage the contract award negotiations to ensure that clear project milestones were established in the final contract documents. As the report stated, the “approach to negotiating and entering the contract did not effectively support achievement of outcomes because the contract did not explain the milestones and performance requirements in a manner that was readily understood and applied.” In fact, the subsequent stages of the procurement process left many issues still to be resolved during the contract award negotiations:
Contract negotiation and execution
CrimTrac had prepared for the contract negotiation phase and produced a BIS Negotiation Directive on 7 December 2015. The directive outlined roles and responsibilities of the negotiation team, which comprised five people, supported by subject matter experts as required.
CrimTrac’s analysis of NEC’s proposal (in Phase 2) revealed 68 issues requiring clarification with NEC. CrimTrac recognised that this was a high number that could make it difficult to negotiate a contract with NEC. By Phase 3, CrimTrac reduced the list to 32 issues.
Whilst the BIS Negotiation Directive contemplated that CrimTrac would hold negotiation sessions over a two-week period, negotiations actually concluded after two months on 17 February 2016. This was partly due to availability of personnel over Christmas/ New Year. This was also due to NEC not initially proposing specific alternative drafting to the proposed contract terms.
Major issues identified for negotiation of contract terms included the pricing schedule, solution requirements, performance requirements, solution design phase and implementation requirements.
The ANAO’s examination of the records of the negotiation process indicated that CrimTrac and NEC had not resolved every issue by the time authorised officers signed the contract. In addition, the contract did not reflect every agreed negotiation issue. Outstanding items related to acceptance certificate payment, remote access, hardware milestones, and security clearances for personnel.
The contract was finalised and signed by NEC and CrimTrac on 20 April 2016.
In turn, the overload of negotiation issues led to poorly defined project milestones and to the ripple effect of downstream implementation delays:
The Biometric Identification Services contract
The contract entered by CrimTrac and NEC comprised more than 800 pages. The contract was for NEC to design, implement, integrate, support and maintain an integrated biometric system. NEC was to complete the Solution Design by 30 June 2016 and all contract requirements by 20 October 2017. The ‘support and maintain’ phase would continue to 30 November 2022.
The contract was divided into two phases: the Solution Design Phase and the Solution Implementation Phase. Within each phase, there were milestones, each with a specific completion date. The completion dates for individual milestones were not accompanied by a description of specific documents, activities, or tests needed for each milestone. The completion dates for milestones were not accompanied by specific standards. In many instances, work was to be completed ‘to the satisfaction’ of ACIC, which introduced an element of subjectivity in assessing when the milestone was complete. Table 2.2 shows the summary of milestones to which NEC originally agreed to work.
The structure of the contract as ‘design and build’ meant that achievement of the design phase was clearly critical: without an agreed design, later phases could not be completed. It also meant that any slippage in the achievement of the first milestone, Solution Design, would inevitably have knock-on effects for achievement of all later milestones.
As the report determined, this resulted in a major failure to properly track project milestones and manage contract payments:
ACIC did not effectively manage the contract. The mandated process in the contract for assessing progress against milestones and linking their achievement to payments was not followed at any point.
ACIC established appropriate arrangements for reporting to stakeholders. However these were not fully effective because they did not result in sufficient action being taken and the external stakeholders felt that reporting dropped off over time. Numerous reports identified the project’s ‘red’ status from an early stage but little meaningful action was taken until too late.
While a detailed implementation plan was developed, ACIC did not adhere to it to ensure delivery of specified outcomes, milestones and deliverables.
Financial management of the BIS project was poor. ACIC’s corporate finance area had no responsibility for management of the financial aspects of the BIS project; neither did the project team have a dedicated financial or contract manager. ACIC was unable to definitively advise how much they had spent on the project.
In fact, the audit concluded that none of the project milestone were ever properly completed:
In summary, evidence obtained by the ANAO about milestone completion was confused and contradictory. Despite ANAO requests for clarification, at no stage was ACIC in a position to state with any confidence which Key Project Documents (and therefore corresponding milestones) were actually completed in a manner and to the extent specified in the contract. As noted at 3.26, at the time of audit, ACIC advised that in fact, no milestones were ever actually fully completed.
The report also found that record-keeping was so poor that the project team was unable to show what was received for payments made:
ACIC was unable to provide evidence to show goods and services were actually received against the contract before invoices were paid, which is a fundamental and routine financial management process. Project areas (including BIS) within ACIC merely advised the financial reporting team, through the budget team, of where costs should be allocated and to whom amounts should be paid. There was no reconciliation performed on project worksheets and the general ledger to confirm the completeness and accuracy of project worksheets.
In fact, the report found many instances where the same goods and services were paid for more than once, but found no evidence that any of the work was actually ever finished. For example, the audit found that the “stipulated contract process by which progress against milestones and deliverables was to be assessed was not followed at any stage and ACIC thus had no way of assuring itself that it got what it paid for”, and that while “ACIC agreed to more than $12 million in additional work” the reviewed documentation “showed that some of this work may have been unnecessary and other work may have already been covered under the contract.” In one instance, the audit found that “ACIC made a ‘goodwill’ payment of $2.9 million to NEC which was not linked to the achievement of any contract milestone” and that “ACIC was not able to provide details of how the quantum of this payment was calculated.”
As the project spiralled out of control, an external consulting firm was brought onboard to try to repair the deteriorating relationship between the project team and the supplier. However, those efforts failed, as evidenced by the loss of three of the supplier’s project managers in rapid succession:
PwC’s October 2014 market assessment (see paragraph 2.8) had noted that timely delivery of the project would depend on strong relationships.
Evidence showed that the relationship between ACIC and NEC deteriorated to such an extent during the project that ACIC decided to engage PwC to intervene. In December 2017, ACIC contracted PwC to facilitate ‘turnaround’ workshops between ACIC and NEC to ‘reset the relationship’ and ‘build trust’.
Subsequently in January 2018, ACIC exercised its contractual discretion to request removal and replacement by NEC of its Program Manager. The contract did not require ACIC to provide a reason for its request and ACIC was not able to provide any documented reasons for the request.
In late January 2018, a new NEC Program Manager arrived but left within two weeks. A third replacement NEC Program Manager also departed in late February 2018.
After a failed attempt to de-scope the project to remove the facial recognition functions and preserve the new fingerprint management system, the entire project was eventually terminated. The parties were left to sort out liability for the failed project. As this case study illustrates, the failure to establish appropriate performance milestones to serve as benchmarks for project performance and progress payments exposes complex projects to significant risks of downstream failure. Complex projects require proper project governance frameworks that enable the integration of technical subject-matter experts into the planning stages to enable the creation of clear milestones for contract performance. The failure to integrate these strategic procurement requirements from the outset of a project exposes that project to a high risk of failure.