This article is an excerpt from Government Procurement, which is available for purchase.
In its February 2015 determination in 4Plan Consulting Corp. v. Shared Services Canada, the Canadian International Trade Tribunal awarded the complainant lost profit damages after finding that the government inappropriately used reference information in its evaluation process. The case dealt with an RFP for professional consulting services. The Tribunal found that the government failed to properly define the term “project” for the purposes of its past experience assessment and abdicated its responsibility to properly evaluate submissions by relying on the complainant’s references to determine whether the projects listed in the complainant’s proposal should be considered to be separate projects or a single project:
41. With respect to projects 17 and 18, and projects 19 and 20, the evaluators telephoned the listed references and asked if the listed projects were in fact a single project. While both references affirmed that the listed projects constituted a single project (17 and 18 being one project, and 19 and 20 being a second project), there is no indication that the evaluators actually specified what was understood as constituting a “project” or otherwise provided either guidance or defining criteria to those contacted references.
42. This becomes even more apparent when the responses given by the references are examined. For instance, when asked whether projects 17 and 18 were in fact one project, the reference answered in the affirmative, but further noted that he could not recall whether there were one or two contracts awarded for the work itself. So being, the evaluators seem to have accepted that, despite the fact that work may have been performed under two separate contracts, these contracts could nonetheless have been considered as a single project.
43. Whether multiple contracts would be considered a single project, and how they would be deemed separate projects, was not delineated in the RFP; it appears that the evaluators did not delineate any criteria by which to determine this.
44. The Tribunal finds that the evaluators relied on extrinsic criteria, essentially devised by the references themselves, in order to establish what actually constituted a project. They then accepted these outside determinations as a premise for their final evaluation. In doing so, the evaluators improperly abdicated their responsibility to evaluate the bids in favour of the unguided opinions of such references.
The Tribunal found that the government’s reliance on the references to define the meaning of past projects constituted an improper hidden evaluation factor:45. This is not to say that the references themselves acted inappropriately. Rather, they could not be relied upon as a basis for determinations on matters that were not defined in the RFP and for which no context was supplied for their consideration.
46. The Tribunal has repeatedly held that an evaluation will be considered to have been unreasonable where the evaluators have not applied themselves in evaluating a bidder’s proposal. In the present case, the evaluators did not apply themselves in determining what constituted a project for the purposes of the RFP, but instead deferred this decision, relying on undisclosed criteria, to the references themselves. As a result, the Tribunal finds that the evaluation of projects 17 and 18, and 19 and 20 was unreasonable and that 4Plan’s complaint is valid on this ground.
The Tribunal also found that the government failed to specify in its RFP how reference checks would be conducted or in what circumstances references would be contacted:
47. The solicitation documents do not make any mention of reference checks being conducted, nor do they set out any parameters as to whether or in what circumstances references would be contacted. In fact, the only mention of a reference being contacted is in M5 and M14, neither of which are in issue, and which simply state as follows:
The bidder must provide the following details as to how the stated experience was obtained:
• Client name (project authority) and contact information for verification purposes.48. By contrast, the rated criteria in issue only require that the bidder list the name and current e-mail and telephone number of the project authority for each listed project. This, of course, may be viewed as an implicit indication that SSC somehow intended to contact these references; however, this is not explicitly stated in the solicitation criteria or in the method of evaluation. Moreover, no information was given as to how the information would be used or how it could in fact impact the evaluation itself.
The Tribunal also found that the evaluators were inconsistent in their use of references and collected the information in an arbitrary and inconsistent manner:
49. More significantly, however, there was no clear rationale as to why evaluators contacted references in some instances and not in others. In particular, the evaluators deemed that the similarities between projects 12 and 13 were “significant enough” for them to determine that they were in fact a single project, without having to contact any references at all.
50. With respect to projects 17 and 18, and projects 19 and 20, however, SSC argued that the similarities between the projects caused the evaluators to call the project authorities for verification. Similarly, the evaluators contacted the reference provided for project 20 to determine whether it involved the development of a costing framework, but determined projects 14 and 16 did not involve the development of costing frameworks, without any apparent need for contacting the references listed.
51. Similarly, no explanation was provided, either in SSC’s submissions or in the evaluation documents, as to how the evaluators reconciled the discrepancy between the reference’s statement that project 20 “. . . did not involve the development of a costing framework”, when the project description clearly stated the opposite. As discussed above, the description of project 20 was similar to that of project 11, yet no explanation or rationale was offered as to why the evaluator chose to contact references in one instance, but not the other.
[footnotes omitted]The Tribunal concluded that the government conducted an inconsistent and arbitrary evaluation and therefore upheld the validity of the complaint:
52. In light of the foregoing, the Tribunal concludes that the evaluators not only used reference checks not listed in the relevant rated criteria but also failed to apply themselves as to their uneven recourse and instead evaluated 4Plan’s bid in an inconsistent and arbitrary manner. Therefore, the Tribunal finds this ground of complaint to be valid.
With respect to remedy, after concluding that the inappropriate use of references resulted in fundamental flaws in the evaluation process, the Tribunal determined that the complainant should be awarded its lost profits. As this case illustrates, the inappropriate definition of past experience requirements and the inappropriate use of reference information when assessing past experience can result in evaluation deficiencies that undermine the defensibility of a public procurement process. Purchasing institutions should therefore ensure that their solicitation documents clearly define their past experience requirements and transparently explain how they intend to use references to assess those past experience requirements.
