This article is an excerpt from The Art of Tendering: A Global Due Diligence Guide, which is available for purchase.
In its December 2018 final report, the New Generation Rollingstock Train, the Commission of Inquiry found that the Queensland government failed to ensure that its new trains met applicable wheelchair accessibility standards. The Commission found a series of flaws in the planning and execution of the next-generation train procurement projects, including a failure to include the required accessibility experts at the outset of the procurement planning process when it would have been possible to incorporate the required accessibility standards into the train designs.
As the report summarized, the procurement process that led to the train contract was unnecessarily protracted and complicated, and was impeded by ongoing changes in overall project management responsibility and strategic direction:
The procurement process for the NGR trains was not ideal. Procurement was unnecessarily prolonged and was marred by delays, disruptions, and failures to adhere to policies, guidelines and frameworks. Decisions were made to specify design requirements that were non-compliant and there was a general acquiescence to non-compliance. There was also a flawed understanding of DSAPT requirements. The procurement process took five years from the request for expressions of interest to contract execution. This was substantially longer than would be typical of a comparable infrastructure project. While the project was subject to a ‘project pause’ for six months, procurement had been underway for more than three years at the time of the pause. Prior to the pause, the NGR project was characterised by a lack of rigour, continual slippages and missed milestones.
The project pause and subsequent changes to the procurement model and the project lead delayed completion of the procurement process and created disruption and discontinuity. The procurement model was changed to an availability PPP model. This change so late in the process created disruptions, required the recommencement of the request for proposal (RFP) phase, and resulted in difficulties for the two shortlisted proponents. Changing the project lead from QR to Projects Queensland (acting on behalf of TMR) was similarly disruptive and created animosity between the agencies.
The disruption, inherent break in continuity, and resulting animosity may have contributed to non-compliance through relevant information not being transferred and information not being shared across agencies in subsequent phases of the project.
The Commission found a major flaw in the decision to award a contract and proceed to construction based on non-compliant accessibility specifications contained in that contract:
The decision to request a non-compliant train through the procurement process and to then accept a proposal based on a non-compliant design and enter into a project deed on that basis was, in the Commission’s view, seriously flawed. It is noted that these decisions were endorsed on the basis of incomplete information as non-compliance issues were not escalated to senior decision makers during the procurement process. The lack of advice and escalation about compliance issues was a common theme throughout the different phases of the NGR project.
In fact, the Commission found that the train specifications contained design inconsistencies that were not rectified during the various stages of the project that led up to the construction of the trains:
NGR train design approval process
The NGR trains were procured based on performance specifications under a ‘design, construct, maintain’ contract. The broad parameters for the train design were defined by the technical requirements outlined in the project deed. Once the deed was executed, Qtectic was responsible for designing the train based on technical requirements, and the project team was responsible for approving the design through the various stages of the design process.
The project deed required Qtectic to design and construct NGR trains that, among other things, comply with the technical requirements (defined to mean both the performance specifications and Qtectic’s proposal). However, the technical requirements are in some respects inconsistent and uncertain. The performance specifications include contradictory requirements regarding compliance with disability legislation including specifications that are non-compliant. Additionally, Qtectic’s proposal details design elements that are inconsistent with the performance specification and that do not comply with DSAPT.
While it is not unusual for inconsistencies to arise as a result of incorporating a specification and proposal in the terms of a contract, the approach does give rise to risks of uncertainty and may have impacted on the effectiveness and outcomes of the design process.
As the technical requirements called for design elements that did not comply with the disability legislation, the design approval process for the NGR trains started from a non-compliant position. Designing the trains in accordance with the project deed would produce trains that did not meet the specifications outlined in DSAPT.
The capacity for the NGR project team to make changes during the design process was constrained by the technical requirements. A variation, with potential additional costs, would have been required to address most of the non-compliance issues identified by the Commission.
The possibility for non-compliances to be addressed through the design approval phase was also limited by the NGR project team and the QR technical experts not having a detailed understanding of the requirements under DSAPT, not recognising how equivalent access compliance could be pursued for some of the technically non-compliant elements, and not fully appreciating the possible consequences of non-compliance.
The report also found serious flaws in stakeholder management, since no specific consultations were undertaken with the disability sector in the development of project specifications. In fact, the report found that consultation was only conducted after it was too late to make significant design changes to better ensure compliance with accessibility standards:
Stakeholder management and consultation
Consultation during the procurement phase of the NGR project was, in the Commission’s view, inadequate. No specific consultation was undertaken with the disability sector about the NGR trains and the NGR project team failed to formally engage the QR Accessibility Team to strengthen its knowledge of the disability legislation and functional requirements.
If the NGR project team had undertaking genuine consultation about the NGR train design from an accessibility perspective before or early in the procurement process they would have had a greater understanding of accessibility considerations and preferences. They could have used this in formulating performance specifications, and it would have highlighted key accessibility requirements for consideration throughout the procurement and design approval processes. It would certainly have been more cost effective than rectification of the trains.
Consultation during the design approval phase was slightly improved, although it was still limited. The QR Accessibility Reference Group attended two of the train mock-up inspections, and consultation sessions regarding an assisted boarding model for the NGR trains. Feedback from the reference group suggests that its members were not given sufficient information at the mock-up inspections to make fully informed comments, and that the purpose of the consultation was not clearly explained as the were not aware that key design features were non-negotiable until late in the consultation process.
Consultation was substantially improved following the design approval stage. At this point the NGR project team sought to understand compliance issues escalated by the disability sector, and subsequently engaged with the sector regarding rectification of the trains. This consultation process included a range of effective elements that could be incorporated in future consultation process for the procurement of major public transport infrastructure. Had this transparent, respectful an consultative approach to engaging with the disability sector been adopted before the project deed was signed, or during the design process, many of the issues being examined with by the Commission could have been resolved.
Unfortunately, the cooperative approach was damaged by the decision to apply for temporary exemptions during the rectification consultation, without engaging with the disability sector on the rationale for the application. This was a serious oversight.
The Queensland Government has committed funding to rectify the NGR train design issues, but at the time of writing this report the recommended design developed in conjunction with the disability sector had not been considered by Government.
More specifically, the report identified flaws in the train design in relation to accessibility across the trains and accessibility to the onboard toilets:
The design of the NGR train restricts people with disabilities from manoeuvring within and between the accessibility carriages. Passengers with disabilities who board the train at one entrance are largely limited to the allocated spaces adjacent to that entrance. This is particularly the case in accessible car A.
The dimensions of the access paths may restrict passengers using mobility aids from relocating to another allocated space within the car, not adjacent to their entrance point, or to an allocated space within the other accessible car. Passengers with disabilities who do not board at the entrance adjacent to the toilet would find it difficult, if not impossible, to access the toilet. Relocating to another allocate space or accessing the toilet may require a passenger to disembark and reboard the train using another entrance.
As the report confirmed, this would require wheelchair passengers to seek assistance in disembarking from the train and re-entering from another door that was closer to the onboard toilet located at the front of the train cars.
The report also found flaws in the accessibility design of the actual toilet modules on the train, which could require wheelchair passengers to have to disembark and use the washrooms in the train station and then wait for the next train:
Concerns have also been raised regarding the DSAPT requirement that passengers using mobility aids must be able to enter, position their aids and exit the toilet module, as the configuration may prevent some passengers from carrying out a fully-parallel side transfer to the pan. The configuration would also not be suitable for passengers who require a right-hand transfer. Passengers unable to safely use the onboard toilet may need to disembark and use a toilet at a station (delaying their travel until the next train arrives).
The Commission found that this solution was inadequate and did not comply with accessibility requirements:
The proposed approach to provide access to the toilet for passengers with disabilities travelling in accessible car A was for the passenger to contact the guard to request to use the toilet, for the passenger to then be assisted to disembark at the next station and to reboard the train in accessible car B. Although this approach would provide access, it would not, in the Commission’s view, satisfy the criteria for equivalent access compliance, particularly regarding the need for equivalence of convenience and dignity.
Furthermore, the report found a failure to consult internally with the government’s dedicated accessibility team to ensure that the required accessibility specifications were introduced into the Request for Proposals (RFP) from the outset:
Despite QR having a dedicated Accessibility Team, there appears to have been no structured, formal engagement by the NGR project with the Accessibility Team while QR was the project lead. The Accessibility Team offered input and advice from September 2010 but, while the project team sought its input on ad hoc issues, there was no regular formal engagement. Nor was the Accessibility Team requested to engage with the disability sector on behalf of the NGR project.
The Commission also found project governance flaws in the failure to inform senior decision-makers of the non-compliance with accessibility standards, in the failure to engage an accessibility expert from the outset of the planning process, and in the decision to reassign overall project responsibility to different government entities during the project:
The various governance frameworks for the NGR project each had a project manager/program director position, which was critical to the success of the project. The position was responsible for operational management, including risk management, and for regularly advising key governance bodies of the project’s status and progress. The position was the key conduit between the project team and the project’s governing bodies. However, there appears to have been a persistent failure throughout the life of the project to inform the project governing bodies and senior executives of the issues regarding non-compliance with disability legislation and associated risks and consequences.
The NGR project did not engage an accessibility expert, during the procurement or design approval phases, to provide advice on the application of the disability legislation and functional requirements and the mechanisms for achieving compliance. The project team had general knowledge of the disability legislation, but not enough to effectively manage compliance issues.
Further, the decision to remove QR as the project lead created a degree of resentment and animosity, resulting in a competitive relationship during the delivery phase of the project.
Despite numerous documents outlining the respective roles of QR and TMR, there were ongoing tensions regarding each party’s responsibilities particularly in relation to design approval and stakeholder consultation.
In the Commission’s view, the tense relationship hindered the effective management and resolution of compliance issues. An environment where the prevailing consideration is defending roles and positions rather than working together to achieve a common goal is not conducive to positive project outcomes or the prompt and effective management of issues.
The report found that the government’s project “health check” conducted midway through the project identified a series of significant governance problems that should have been addressed by establishing appropriate project governance from the outset of the project:
4.4 Project health check
In December 2009 Ernst & Young was engaged to perform a high-level project health check following concerns raised within QR and by parties associated with the NGR project about project governance, compliance with the QRIFM and the general health of the project. The check was to examine issues regarding the robustness of the process, governance, gating points, stakeholder management, and advisors’ views of the project’s management.
The report, provided in February 2010, identified a range of issues and made a series of recommendations. The issues identified during the health check are outlined under the relevant sections of this report. Concerns significant to the procurement process included:
an RFP process substantially below what would be expected of a comparable project
the procurement agenda being driven by strong personal views of project members
the resignation of the probity advisor in December 2009 with no replacement appointed
apparent pressure to go to market with documentation not finalised
unresolved documentary and process issues that should have been finalised months prior
advisors having difficulty understanding the project’s actual timeframes
an incomplete and not appropriately customised probity plan
documents required under the QPP not having the expected rigour.
As the Commission noted, this caused the project to be put on hold for six months so that the overall project strategy could be revisited. This resulted in a major change in strategic direction late in the project cycle:
4.6 Project pause and restructure
On 8 May 2012, following the state election on 24 March 2012, the new Minister for Transport and Main Roads requested that the NGR project be put on hold pending an independent commission of audit into the state’s finances and consideration of whether the project’s ‘funding and delivery method provides the best overall transport and financial outcome’.
As part of the review of the NGR project, Queensland Treasury Corporation (QTC) was engaged to independently assess alternative procurement financing options. In considering the suitability of the financing options QTC took into account the budget impact, change risk, credit rating, off-balance sheet capability and value for money. While QTC found that an availability PPP financing model could provide additional benefits, it also noted that there were risks in introducing a significant change late in the procurement process.
These changes resulted in a revised RFP being issued to the two short-listed proponents. This decision was criticized since the other original proponents were not allowed to bid on the substantially redesigned RFP that now included a public-private partnership model, referred to below as PPP:
Disruptions and limited competition
The project pause and subsequent changes to the procurement model and project lead delayed completion of the procurement process and created disruption and discontinuity.
Changing the procurement model to an availability PPP procurement so late in the process created disruptions for the project team. It required the recommencement of the RFP phase, and caused difficulties for the two shortlisted proponents with one reportedly better able to adapt to the change as they revised their proposals.
The timing of the change also prevented other parties who were interested in bidding under the PPP model from participating in the process. Downer EDI requested re-inclusion in the process under the new model, and other parties may have been similarly interested in tendering for the contract given that it was four years since market interest was tested with the EOI. Restricting the field to the two shortlisted proponents for such a significant change to the procurement model may have been challenged by other potential suppliers. However, the Commission notes that the process contract were in place with the two shortlisted proponents, which potentially limited the State’s capacity to reopen the bidding process.
While the Commission makes no finding on the appropriateness or value for money of an availability PPP model for the procurement of the NGR trains, in the Commission’s view, changing to a PPP model so late in the procurement process was disruptive. This view was shared by a range of stakeholders interviewed by the Commission, including the probity auditor.
It is also significant to note that PQ’s End of Project report identified that two proponents were not sufficient, and that the limited competition resulted in ‘gamesmanship’ during the second RFP process. Given the substantial change in the procurement model, and the length of time since the EOI, returning to the open market when the procurement model was changed may have resulted in new bidders, increased competition and different design outcomes.
Changing the project lead from QR to PQ (acting on behalf of TMR) so late in the procurement process was also disruptive and created animosity between the agencies. Some QR staff believed the project should have remained with QR.161 The disruption, the inherent break in continuity, and resulting animosity may have contributed to noncompliance through relevant information not being transferred, noting that key members of the QR project team were retained on the project, and information not being shared across agencies in subsequent phases of the project.
In the Commission’s view, the late stage changes resulted in delay, disruption and increased risk and were not reflective of best practice procurement. The role the changes played in relation to the procurement of non-compliant trains is, however, not determinable. It is unclear whether continuing with the same procurement model and retaining QR as the project lead, particularly given QR’s ongoing involvement in the project, would have produced different compliance outcomes.
The Commission concluded that the failure to ensure that the trains met the accessibility specifications was partially caused by a failure in internal reporting to senior decision-makers:
In the Commission’s view, the decision to request a non-compliant train through the procurement process and to then accept a proposal based on a non-compliant design and enter into a project deed on that basis was seriously flawed. However, the Commission notes, as discussed below, that these decisions were endorsed on the basis of incomplete information, as non-compliance issues were not escalated to senior decision makers.
As the report noted, these flaws were exacerbated by a revolving door at the senior levels, including at the CEO, Chairman, Board, and Ministerial levels.
In fact, the report found a “tokenistic” approach to compliance with procurement guidelines and frameworks, and non-compliance with reporting requirements:
Non-compliance with procurement policies, guidelines and frameworks
A range of policies, guidelines and frameworks applied to the NGR project at various stages (see section 2.5 for information about the policies, guidelines and framework). The degree to which the project complied with these policies, guidelines and frameworks was variable.
The health check undertaken by Ernst & Young in December 2009 found that while documents required under the QPP were produced, they did not have the rigour expected for a procurement process of the scope and size of the NGR project. These documents, particularly the significant procurement plan, are intended to guide and inform procurement processes to ensure appropriate planning and preparation is undertaken to achieve the required outcomes in an accountable and transparent manner. Tokenistic compliance through the production of poor-quality documentation undermines the intent of the policy.
Another area of concern relates to disclosure requirements under the National PPP Guidelines. The guidelines, as they apply specifically to Queensland, require the responsible minister to table in parliament a summary of the project agreements, signed off by the Auditor-General as a fair reflection of the agreements. They also require the responsible minister to table the probity auditor’s final report. The former Minister for Transport and Main Roads did not table a summary of the NGR project agreements or the probity auditor’s final report.
The Commission identified only three PPP projects where the agreement summary and probity auditor/advisor final reports had been tabled as required by the National PPP Guidelines. The Commission is of the view that tabling these documents supports accountability and transparency in PPP procurement processes; however, if it is not the intention of government that the documents be tabled, this should be reflected in the National PPP Guidelines. Retaining the requirement in the National PPP Guidelines but having a practice of non-compliance creates at best confusion and at worst a culture of ambivalence or disregard for the policies, guidelines and frameworks governing procurement processes.
This led the Commission to recommend requiring further training on compliance with procurement policies, accessibility standards, and reporting guidelines:
The Commission recommends that training on procurement policies, guidelines and frameworks and the requirements under the disability legislation, be provided to directors general and relevant chief executive officers as the parties responsible for ensuring the polices, guidelines and frameworks are applied within their organisations.
The Commission recommends that templates used to seek Cabinet Budget Review Committee endorsement regarding procurements be updated to require information on the procurement’s compliance with the disability legislation.
As this Commission report illustrates, the failure to properly integrate technical specifications into RFP requirements at the outset of a project leads to significant downstream difficulties during the implementation phases of a project. Subject-matter experts need to be properly integrated into the project team during the strategic planning stage of a procurement so that their input informs the designing and drafting of the solicitation document and the creation of resulting contractual specifications.