This article is an excerpt from The Art of Tendering: A Global Due Diligence Guide, which is available for purchase.
In its November 2016 decision in TPG Technology Consulting Ltd. v. Canada, the Federal Court of Appeal upheld a trial court’s dismissal of the plaintiff bidder’s $250 million lost profit claim against the Government of Canada. The claim dealt with a federal government procurement process valued at approximately $428 million for the acquisition of information technology engineering and technical support services. While the trial court found flaws in the government’s consensus scoring process, it ultimately rejected the lost profit claim since it found that the evaluation errors had no impact on the ultimate proponent rankings.
The October 2014 federal trial court’s decision was the third part in a complex trilogy of legal proceedings that included a September 2011 Federal Court summary dismissal of the bidder’s claim, followed by a July 2013 Federal Court of Appeal reversal that directed the matter to trial.
The original claim was launched by TPG, an unsuccessful bidder seeking damages against the federal government. TPG originally claimed that PPI Consulting Ltd. (referred to as PPI), the external consulting firm retained by the government to assist in the evaluation process, was biased against it and lowered its score during consensus scoring sessions.
As the September 2011 summary dismissal decision detailed, TPG claimed that the consensus scoring had been arbitrarily applied by PPI, who they claimed had a pre-existing bias against them as a “body shop”:
TPG claims that PPI, the third party facilitator “had a manifest bias against awarding the contract to TPG and disparaged TPG as a ‘body shop’” (see Powell affidavit para 15). The evaluation consisted of a consensus score model whereby the five evaluators would meet to discuss their individual scores and then arrive at a consensus score. TPG alleges that these consensus scores were arbitrarily applied to unjustifiably reduce TPG’s scores. Additionally, PPI maintained control over the evaluation record and at some point changes were made to the evaluation record that resulted in lower scores for TPG.
TPG alleged that the government had selected the consensus scoring method because its high subjectivity allowed personal biases to infiltrate the evaluation. In support of its allegations, the plaintiff pointed to disparaging public remarks attributed to the president of PPI as evidence that bias had tainted the evaluation process:
The appropriateness and effectiveness of consensus method itself, specifically chosen by PWGSC in an effort to produce the fairest result by ensuring that evaluators are using a consistent understanding of the requirements, is questioned by TPG. During the hearing, counsel for TPG advanced the argument that PWGSC intentionally selected the consensus model, the most subjective model in their view, as a way to allow personal bias and preferences to infiltrate the process. The bias was one against “body shops” – TPG maintaining that Mr. Tibbo might have had a prejudice against small companies and “body shops” as Mr. Howard Grant, president of PPI was quoted in an industry publication in 2009, as speaking disparagingly of “body shops.”
With respect to the consensus scoring sessions, the Court noted that PPI presided over group discussions to address instances where the five evaluators had arrived at different scores. While the Court recognized that some errors occurred during PPI’s tabulation of the consensus scores, it also cited affidavit evidence submitted by the individuals who were involved in the process who swore that, contrary to the plaintiff’s allegations, they had not been influenced by anyone to alter TPG’s scores or manipulate the outcome of the evaluation:
When the consensus phase was completed, the pre-set weights were applied to the scores and the results were added to reach the final score for each bidder. The consensus phase took place between September 22 and September 27, 2006. During the meetings, Mr. Tibbo was assisted by Ms. Mairi Curran, who entered individual scores into the PPI computer. The computer was connected to a projector which displayed the monitor on a screen for all of the evaluators to see. If all five evaluators entered the same score, that would be recorded as the consensus score. When the scores were different, a moderated discussion ensued. Mr. Tibbo recorded the consensus scores in a paper back-up referred to as the Master Evaluation Binder. The evaluators would then record the consensus score in their individual binders.
Although PPI would normally print out a copy of the report on site for the evaluators sign-off, Mr. Tibbo explained that they did not have access to a printer at that facility where the technical evaluation took place. Consequently, Mr. Tibbo printed off copies of the report on October 2, 2006 and provided them to Mr. Hamid Mohammad, Contracting Authority for PWGSC. On October 3, 2006 Mr. Tibbo e-mailed Mr. Mohammad a summary spreadsheet. The data had been manually entered into the spreadsheet. On October 12, 2006 the final results spreadsheet was provided – this was extracted directly from the ERGOV software onto the spreadsheet.
Mr. Tibbo admits that there was an error in the spreadsheet that he manually compiled on October 2, 2006. What he initially thought was a rounding error turned out to be a transposition error. This error, however, had no effect on the final technical result. Additionally, the final October 12 spreadsheet did not contain any human errors.
A meeting was held on October 27, 2006 to address the concerns of Mr. Mohammad. Substantiating comments were provided for some consensus scores as a result, but no scores were changed. The evaluators then signed off on each bid. Mr. Tibbo swears in his affidavit that at no time prior during or after was he approached by or influenced by anyone seeking to secure a particular outcome, nor did he witness any such activity. Similarly, Mr. Bartlett provided an affidavit testifying to the fairness and transparency of the evaluation process.
In its September 2011 summary judgment, the Federal Court rejected TPG’s allegations of impropriety in the evaluation process, finding that the plaintiff failed to prove its theory of a tainted and biased evaluation:
In my view, TPG has failed to provide any evidence that supports their theory that there was some kind of wrong-doing on the part of the PWGSC either during the evaluation process or following contract award. TPG’s theory seems to largely rest on the claim that PWGSC was biased against TPG and that the technical evaluation scores were changed at some point. TPG cannot explain where the alleged bias came from or how it was manifested, nor can TPG explain who changed the marks, or when and how they were changed. The theory that they were indeed changed is based on the fact that the “official” technical scores differed from allegedly rumoured and expected scores.
During his examination for discovery, Mr. Powell admitted that he had no concerns with the honesty and integrity of any of the five evaluators, but was concerned that there was some kind of re-evaluation after the evaluators submitted their scores, resulting in arbitrarily lowered scores for TPG. Mr. Powell surmises that Mr. Tibbo changed the scores since he controlled the scores at the time they must have been changed (see tab 3 of the Relevant Portions of the Examination of Donald Powell) and that he was likely directed to change the scores by PWGSC officials. Mr. Powell was unable to elaborate on who might have directed Mr. Tibbo to change the scores, answering question 419 on examination for discovery, “I imagine he was directed, but I don’t know who directed him, I don’t know, you know, I’m not the FBI, I can’t find out who told him to do it.” (Applicant’s Motion Record, pg 2107). The questioning continued:
Q: And how do you know, sir, what basis do you allege that, in fact, the scores were changed by anybody?
A: Because the scores are simply impossible to believe, for starters. And they don’t make sense on certain wise and the bias against our score was obvious.
Q: Is it not possible, sir, that the original five evaluators came to what you’ve described as the impossible results and that Mr. Tibbo had nothing to do with it?
A: No, it’s not possible. Jim Bezanson told us what the scores were like, and he was really in charge of the evaluation team. He had no idea what the new scores were like, not a clue. He knew what the old scores were like, the real scores.
Mr. Powell and Mr. Stanley Estabrooks, Infrastructure Manager at TPG, refer to Jim Bezanson as the ETS RFP Evaluation team leader throughout their affidavits. The Crown clarified at the hearing that Jim Bezanson was in fact, just a member of the five-person evaluation team. Mr. Bezanson left PWGCS after receiving a job offer at Canada Post in mid-November 2006, after the evaluation had been completed. The above exchange refers to a telephone conversation that Mr. Estabrooks initiated with Mr. Bezanson in March 2007, during which Mr. Bezanson allegedly expressed surprise that CGI was rumoured to have received a much higher technical score, as he was under the impression that the results of the technical evaluation were, “very close” (see Estabrooks affidavit at para 22). Mr. Estabrooks conceded on cross-examination that he did not know what Mr. Bezanson meant by “very close” (Applicant’s Motion Record, Vol 7, pg 1975). Mr. Estabrooks nonetheless relayed the contents of this conversation to Mr. Powell who concluded as a result, that the technical scores had been changed without Mr. Bezanson’s knowledge or participation and that litigation was necessary to determine exactly how the “official” technical scores had been derived, and how they had been changed without Mr. Bezanson’s knowledge.
I am not satisfied that TPG is able to provide any evidence beyond mere speculation and conjecture to suggest that a trial is warranted to further flesh out this allegation. The theory that the scores changed is based on vague, second-hand information extracted in the course of a personal phone call interpreted with the most conspiratorial gloss. I am sure that TPG was disappointed with the results, and this disappointment has largely fuelled this litigation.
The Federal Court therefore dismissed TPG’s $250 million claim against the government.
However, the Federal Court of Appeal disagreed with the Federal Court’s conclusions, finding that it had misapplied the rule for summary judgment and that there remained issues regarding liability that warranted a full trial:
The judge’s description of TPG’s claim for damages for breach of contract in relation to the bid evaluation process refers to allegations of bias and inexplicable changes to the evaluations. However, that is not a complete description. The complaint in substance is that the bids were not fairly evaluated. Although the Crown adduced considerable evidence in an attempt to establish the integrity of the evaluation process, that evidence did not squarely answer all questions about the fairness of the bid evaluations.
I will illustrate this point by one example: the documents that refer to the evaluation of the bids in respect of sections 3.3.3 and 3.3.5 of the request for proposals. Those provisions deal with performance and service level metrics. TPG alleges, and the Crown does not deny, that the scoring of those provisions was of critical importance in the final ranking of the bids. If they were unfairly evaluated, it is probable that the entire bid was unfairly evaluated.
The record discloses some evidence of confusion on the part of the evaluation team about the meaning of sections 3.3.3 and 3.3.5 and how they ought to be scored, which indicates a controversy about their correct interpretation. It is not denied that there was some confusion in that regard in the course of the evaluation process.
More importantly, however, there is evidence that could support the allegation of TPG that the final scoring of those items is not reasonably justifiable.
In determining that there remained outstanding issues that warranted a trial, the Federal Court of Appeal also clarified the Supreme Court of Canada’s rule from Double N Earthmovers v. Edmonton (City) and held that post-award conduct could be relied on as evidence of whether a winning tender was compliant or non-compliant. This point was material to whether there were issues that required a trial, since the winning bidder had allegedly planned to use TPG’s former employees in the performance of the awarded contract. As the Federal Court of Appeal observed, the availability of those employees was something that was far from certain and therefore brought the compliance of the winning tender into question:
According to TPG’s interpretation of the relevant provisions in the request for proposals, a bid is non-compliant unless, when the bid is submitted, each of the bidder’s proposed resources is either an employee of the bidder, an individual who had consented to be named as a resource, or an individual whose employer had so consented. This necessarily implies, in TPG’s submission, that a bid is non-compliant if the bidder proposes to rely on incumbent resources (that is, the resources of TPG) for whom consents do not exist and cannot be obtained. The Crown does not agree with that interpretation of the provision.
The Court was referred to a number of provisions in the request for proposals and related documents that are said to assist in defining the bidder’s obligation to establish that if awarded the contract it will have the resources to perform the services required by the contract. However, in my view the contractual documents are ambiguous on that point, and therefore the merits of TPG’s proposed interpretation cannot be determined in the absence of a full evidentiary record. That is sufficient to establish the existence of a trial issue on a fundamental aspect of TPG’s claim.
The matter then proceeded to a full trial, albeit by that point, TPG abandoned its allegations against the government regarding conflict of interest, negligence, bad faith, misconduct, bias, fraud, unconscionability, inducing breach of contract, and economic interference. TPG also abandoned its allegations that the Request for Proposals (RFP) had been drafted to favour a competing bidder and also abandoned any claims for punitive damages. At trial, TPG’s lost profit claim rested on the assertions that the government evaluation team had made significant errors in their evaluation, which had cost it the contract award. The Federal Court’s lengthy decision considered the issue of court jurisdiction, and also, considered the fairness of the evaluation process, the compliance of the winning bidder, and the issue of damages.
The Federal Court ultimately dismissed the claim by declining to take jurisdiction after concluding that the complaint should have been brought before the Canadian International Trade Tribunal. While the Federal Court acknowledged that the Tribunal does not have exclusive jurisdiction over federal government procurement but shares concurrent jurisdiction with the Federal Court, the Federal Court found that the remaining issues in TPG’s claim fell squarely within the Tribunal’s expertise. It therefore declined to exercise its jurisdiction over the matter:
In summary, I find that the Federal Court has jurisdiction to hear this claim against the Crown by virtue of section 17 of the Federal Courts Act. However, this is a case in which the Court should not exercise its jurisdiction because the CITT is a more appropriate forum that specializes in dealing with all of the issues raised by TPG in this action. Further, it is capable of recommending the remedy being requested by TPG — compensation. TPG therefore had an obligation to exhaust its potential remedies before the CITT before launching an action in the Federal Court. In the face of a refusal to exercise that avenue of redress, this Court ought not to intervene.
However, while it dismissed the case on jurisdictional grounds, the Federal Court also considered TPG’s substantive claims and found that the government evaluation committee’s consensus scoring process was flawed. The Federal Court noted that an averaging of the evaluation committee’s individual scores showed a significant divergence in TPG’s scores before and after consensus scoring. In one specific evaluation category, TPG’s average initial score was 80.6 out of 100 but its final consensus score was 49 out of 100. In another category, TPG’s average initial score was 32.8 out of 50 but its final consensus score was 22 out of 50. As the decision noted, confusion arose within the evaluation team about the method of scoring performance metrics. That confusion resulted in the evaluation committee abandoning its individual scores and re-scoring TPG’s proposal collectively. Given the government’s explanation, the Federal Court found no fairness issue arising out of this departure from the individual scores:
The evidence of the evaluators, Mr. Henderson and Mr. Tibbo establishes that there was confusion among the evaluators surrounding the evaluation of these two criteria and the meaning of the terms ‘performance metrics’, ‘service level measurements’, and ‘Service-Level contractual framework’. There was also confusion as to the methodology to employ when a bidder listed more than 100 metrics for 3.3.3, given that the maximum number of points that could be awarded was capped at 100. Furthermore, there was significant confusion among them about what constituted a “relevant and measurable performance metric/service level measurement that can reasonably be expected to be used by ITSB.”
During the progress meeting on September 14, 2006, a decision was made by the evaluators that they would only evaluate the first 100 performance metrics or service level measurements listed by a bidder in response to 3.3.3. Despite that agreement, Mr. Boudreault and Dr. Verma continued to evaluate all of the metrics at the individual evaluation stage, whereas the other three evaluators stopped after the first 100 metrics listed.
At some point during the consensus sessions, the evaluators concluded that all of the metrics proposed should be evaluated and not just the first 100 because, as Mr. Boudreault testified, “if a part of the answer is in the bid somewhere even though it’s in another heading or if it’s somewhere else, if it provides an answer to the question… if it’s in there, you have to consider it.”
Only Mr. Henderson testified that when this decision was made, the consensus meeting was adjourned and the evaluators re-evaluated the metrics of each proposal individually before continuing consensus. None of the evaluators supported that account. Their evidence, which I accept, was that the consensus meeting was not adjourned; they simply re-evaluated all of the metrics as a group while arriving at consensus. In my view, this remedies any irregularities in the individual evaluation stage in this regard. This is part of the reason there is a consensus stage — to address issues in the individual phase. I simply do not accept the submission of TPG that it was treated differently than the others because its metrics (or at least those in excess of 100) were not assessed individually by three of the evaluators but were assessed only during the consensus sessions. What matters is that all of TPG’s metrics were eventually evaluated. There is no evidence that in this respect TPG was treated differently than the others.
However, the Federal Court did find that the evaluation committee was inconsistent in its scoring process, since it failed to re-evaluate the winning bidder’s scores by consensus as it had done for TPG’s proposal:
I agree with TPG that the record of the consensus meetings and decisions made therein is quite simply incomplete. In fact, I find it woefully inadequate to answer some basic questions, such as when the evaluators became aware that they were not of one mind as to the standard to be employed in rating 3.3.3, and what was then done regarding the proposals that had already been rated in a consensus process. Further, the record is quite simply not sufficient to respond to questions by a losing bidder as to why its proposal was rated as it was. This sort of detail, Mr. Tibbo testified, was what he told the evaluators they must supply — yet Mr. Tibbo failed to maintain exactly that sort of record in the sessions that mattered most — the consensus sessions.
Based on the evidence of the evaluators, and particularly the different ink colours used by some of them in their rating sheets, it is most likely that the decision to focus on the “quality” of the metric was a decision made after the CGI bid had been assessed at consensus. If the evaluators did not go back and reassess the CGI bid in the same manner that it did TPG’s bid, then TPG was not treated in a fair and consistent manner. This is so even if, as the evaluators testified, CGI’s proposal and its metrics were attuned to the requirement that the response had to be focused on a results based framework, and TPG’s was not. It is not for the Court to make the assessment that the evaluators ought to have made — the Court’s role is to determine whether TPG was treated equally and fairly with CGI, and in this respect, I find that it was not.
TPG having proved that there was a change in the evaluation process when its bid was rated, the burden must lie on the Crown to satisfy the Court on the balance of probabilities that the same was done for the other bidders. The Crown has failed to meet that burden — in large part as a consequence of the incomplete and imprecise record of the discussions and decisions made during the consensus sessions.
The Court also finds that the evaluation of TPG with respect to the 3.3.3 and 3.3.5 requirements was not fairly made because the evaluations are not consistent.
Notwithstanding this finding, the Federal Court ultimately rejected TPG’s claim since it found that the evaluation errors in the two noted categories had no impact on the ultimate ranking, particularly since TPG failed to provide any evidence regarding unfairness in any of the other seven evaluation categories:
The Crown has proven that even if TPG were awarded full marks for 3.3.3 and 3.3.5 and the other bidders’ scores remain unchanged, the result of the RFP would be the same. In addition to having failed to lead evidence regarding the 7 other criteria in issue, it says that the 9 criteria complained of were not selected on any principled basis and there is no logical connection between them. I agree. TPG has not discharged its burden of showing that any unfairness in the evaluation would have changed the ultimate result of the evaluation.
The Federal Court also rejected TPG’s assertions that the winning bidder was non-compliant and that it should have been awarded the contract as the highest scoring, compliant proponent. The Federal Court therefore found that TPG was not entitled to any lost profits.
While the government was ultimately able to defend itself against the $250 million lost profit claim, it was only able to do so after protracted legal proceedings. Irrespective of the outcome of this controversial case, purchasing institutions should take note that public comments made by their officials and by their external consultants can be used against them in legal proceedings in an attempt to impugn the integrity of an evaluation process through allegations of bias. Furthermore, when employing consensus scoring, purchasing institutions should ensure that such processes are implemented with a high degree of diligence since: (i) scoring changes arising out of consensus scoring can be subject to a legal challenge; and (ii) tabulation errors or procedural irregularities arising out of such sessions, however innocuous or inadvertent, can be used against the institution to launch protracted legal proceedings and create significant potential legal exposure.
