This article is an excerpt from The Art of Tendering: A Global Due Diligence Guide, which is available for purchase.
In its July 2010 decision in Almon Equipment Ltd. v. Canada (Attorney General), the Federal Court of Appeal struck down a Canadian International Trade Tribunal determination and ordered the Tribunal to reconsider the complaint. The case dealt with the procurement for aircraft de-icing services and chemical removal services. The Tribunal found that the government evaluators failed to keep proper notes during their evaluation process.
While it determined that the irregularity undermined the fairness of the evaluation process for the chemical removal services, the Tribunal found that the failure to keep proper evaluation notes had no impact on the outcome of the evaluation for the de-icing services. The Federal Court of Appeal disagreed, finding the Tribunal’s determination to be unreasonable since the procedural irregularity compromised the evaluation process for both of the tendered services.
In so finding, the Federal Court of Appeal considered the Tribunal’s statutory framework and noted that the purpose of the Tribunal’s regulatory regime was to protect the public interest by ensuring fair, open, efficient, and transparent public procurement:
The purposes of this regulatory regime, deduced from the above provisions, are as follows:
1. Fairness to competitors in the procurement system. A fair procurement system that applies one set of transparent rules to all bidders increases confidence in the system, and encourages increased participation in competitions. This maximizes the probability that the government will get good quality goods and services that meet its needs, at minimum expense to the taxpayer. In short, fairness gives taxpayers value for the taxes they pay.
2. Ensuring competition among bidders. When bidders are placed on a level playing field and compete, it is more likely that government will get good quality goods and services that meet its needs, at minimum expense to the taxpayer. Competition also gives taxpayers value for the taxes they pay.
3. Efficiency. This speaks directly to the government getting good quality goods and services at minimum expense. This also speaks to the need for a procurement system to run in a timely, practical manner without causing unnecessary expense.
4. Integrity. A procurement process with integrity increases participants’ confidence in the procurement system and enhance their participation in it. This increases the probability that government will get good quality goods and services that meet its needs, at minimum expense to the taxpayer. A procurement process with integrity also gives taxpayers value for the taxes they pay.
5. These four purposes, and the overarching concept of value for taxpayers, are essential aspects of good governance. Important as they are, they must be at the front of the Tribunal’s mind when it finds facts, evaluates their significance, interprets its legislation, applies that legislation to the facts, and grants remedies.
The Federal Court of Appeal then determined that the Tribunal failed to consider the serious detrimental impact on the regime caused by the evaluation team’s failure to keep proper notes of its evaluation process:
This body of evidence shows that the three evaluators considering the competitors’ multi-million dollar proposals did not write their own comments on their own copies of the bids, nor did they make comments on separate sheets. Instead, they restricted their comments to a very small space on the consensus scoring sheets, except for one evaluator who scrawled additional observations on “sticky notes” attached to one of the proposals.
Before the Tribunal, the evaluators testified that “the comments on the consensus scoring sheets were not a complete list of the important factors taken into account in determining their scoring” (at paragraph 39); yet, the evaluators could not point to a single note of their own that would corroborate this.
In this significant procurement, Public Works’ contracting officer did not give the evaluators any instruction on how to use the “comments” column and did not tell them to attach separate sheets, if necessary.
Finally, the most basic characteristics of the evaluation process pursued by the evaluators remain a mystery.
For example, did the evaluators considering this large procurement ever meet as a group of three to discuss the proposals and their evaluation? The Tribunal reviewed the conflicting testimony of the evaluators and could not make any finding on this elementary fact. In a general observation about the procedures that the evaluators followed, the Tribunal found a lack of “clear recollections [by the evaluators] of certain aspects of the evaluation process” (at paragraph 41). Looking at this body of evidence with the important purposes of this regulatory regime front of mind, this is a rather mild statement; the Tribunal may not have appreciated that this body of evidence could be significant to the broader “integrity and efficiency of the competitive procurement system” under paragraph 30.15(3)(c) of the Act.
As mentioned above at paragraph 27, the Tribunal used this body of evidence to make credibility assessments about the evaluators’ testimony. But, in my view, the Tribunal was obligated to analyze and use this body of evidence for another purpose: to consider whether the evaluators’ modest record-keeping and the unknowable nature of the procedures they followed fundamentally compromised the “integrity and efficiency of the competitive procurement system” under paragraph 30.15(3)(c) of the Act.
The Federal Court of Appeal provided three reasons to support the conclusion that the Tribunal failed to properly take the significance of the evaluation team’s failure to keep records into account when making its determination. As noted below, those reasons put a high premium on the need to maintain transparency in decision-making in public procurement:
There are three reasons for this conclusion.
First, this issue was intimately related to Almon’s complaint that the evaluators used improper criteria to evaluate Almon’s proposal for the de-icing services and the glycol collection services. In order to consider this, the Tribunal had to know what criteria the evaluators actually employed in their scoring, assess whether the scores were credible and fair, and ensure that the evaluations were conducted as a result of a process with substance and integrity. The body of evidence, above, is relevant to all these matters.
Second, evaluators and their evaluations of proposals are at the heart of the procurement system. How the evaluators conduct themselves in their evaluation process determines whether the system has integrity and whether the important purposes of this regulatory regime are met. If evaluators can shield themselves from scrutiny by refraining from making adequate records and by following procedures that are later unknowable, the Tribunal cannot discharge its oversight responsibilities. Parliament’s regulatory regime is then frustrated, along with the purposes underlying it.
For all these reasons, the Tribunal erred by failing to consider whether the evaluators’ record-keeping and procedures might have affected the “integrity and efficiency of the competitive procurement system” under paragraph 30.15(3)(c) of the Act.
Given the above, the Court of Appeal directed the Tribunal to re-assess the procurement in a new determination. In its March 2011 redetermination, the Tribunal stressed the need to ensure proper evaluation team record-keeping in order to protect the integrity of the public procurement process:
In the Tribunal’s view, these inconsistencies are the likely result of imperfect recollection and do not, in and of themselves, impugn the good faith and integrity of the evaluators; they do however undermine, at least to some extent, the reliability of the evaluators’ testimony about events central to this matter.
Indeed, the evaluators’ imperfect recollection of events in this case underscores the importance of proper record keeping to the integrity and efficiency of the competitive procurement system. In this regard, and as examined in greater detail below, faulty record keeping fails both purchasers and suppliers, and poses serious challenges to the Tribunal’s procurement review mandate. In short, it fails the public procurement system as a whole.
The credibility and integrity of the competitive procurement system rest, in large part, not only on bids being properly assessed against the prescribed evaluation criteria in actual fact but also on the supplier community’s perception that bid evaluations have been conducted in a fair and transparent manner. In this regard, adequate record keeping by procurement authorities is integral to the ability of a potential supplier to assess whether its bid has been fairly evaluated against the rated criteria contained in a solicitation and to identify and assess potential grounds of complaint.
Proper record keeping allows the procurement authority to justify its decisions and the reviewing body to determine precisely what has transpired. In addition, it is conducive to fulfillment by a procurement authority of its duty of fairness to all potential suppliers. Finally, proper record keeping contributes to the Tribunal’s ability to fully assess the validity of procurement complaints brought before it. It is therefore incumbent upon procurement authorities to keep proper records.
Having reviewed the typewritten and handwritten consensus scoring sheets, the Tribunal considers that the explanatory comments that support the specific scores were at times deficient, illegible, unintelligible or even non-existent. With respect to the comments that those documents did contain and to which both the Government Institution Report (GIR) and testimony at the hearing referred, the Tribunal is of the view that they did not reflect the totality of the factors that were taken into account in determining the specific scores for Almon’s bid. The failure to fully identify the factors upon which the scoring was based renders it difficult for the Tribunal to assess whether Almon’s bid has been fairly evaluated against the rated criteria prescribed in the RFP.
In considering the appropriate remedy, the Tribunal noted that proper record-keeping is a broadly established requirement under the relevant trade treaties and that the failure to follow this requirement resulted in an otherwise avoidable dispute with the complainant. The Tribunal therefore recommended that the government review record-keeping procedures with its procurement officials. The Tribunal also awarded the prejudiced complainant one-third of its lost profits due to the government’s treaty violation.
As this case illustrates, the failure to maintain proper evaluation records can fundamentally undermine contract award decisions and result in protracted legal disputes. To guard against these risks, purchasing institutions should ensure that their evaluators receive the training and support necessary to ensure the defensibility of their evaluation processes.
