This article is an excerpt from The Art of Tendering: A Global Due Diligence Guide, which is available for purchase.
The use of references in public sector bid evaluations has been subject to multiple bid protests in recent decades. The case law, which is discussed in greater detail below, reflects the following governing principles that apply to the use of references during bid evaluations:
- Remedies for Misuse: The improper use of reference information can result in re-evaluation orders, the nullification of contract award decisions, and, in some jurisdictions, lost profit damages awarded against the purchasing institution. Evaluators should therefore avoid using reference information to evaluate bids in an inconsistent, arbitrary, subjective, or non-transparent manner.
- Use of Evidence: The use of references should be limited to situations where the collection of information from references would be sufficiently reliable and relevant to enable a defensible evaluation. Purchasing institutions must establish evaluation rules that clearly define how collected reference information will impact the scoring of bidders, since that information must be used in a consistent and rational manner.
- Transparent Scoring: Prior experience and related reference checks should be scored out of an identified number of total points, with a clear description of how those total points will be allocated.
- Disclosure to Bidders: The information required from references, and the procedures around the collection and use of reference information, should be clearly identified in the solicitation document to allow bidders to select the appropriate references for the specifically required criteria and to allow evaluators to use that information based on clearly established procedures.
- Avoid Process Over Substance: Purchasing institutions should avoid the mechanical application of “content-free” reference check procedures that require the name and title of references in situations where no relevant information will actually be collected from the references.
- Transparent References: The post-bid substitution of non-responsive references should only be allowed where the solicitation rules specifically permit that practice. To avoid unnecessary disqualifications and resulting bid protests, solicitation documents should incorporate clear rectification procedures to allow bidders an opportunity to provide additional references when the originally identified references fail to respond to reference requests.
- Self-References: Purchasing institutions should establish clear rules for providing references for their own evaluation procedures to incumbent bidders or to bidders who have worked for them in the past. When providing references to their own bidders, purchasing institutions should avoid creating more lenient standards for their own references than those applied to third-party references.
- Duty to Investigate: Purchasing institutions should proceed with caution prior to denying a bidder a contract award due to a bad reference or concerns over past experience. Purchasing institutions are under a duty to look behind negative references and to assess whether the failure to complete prior projects was due to material non-performance by the bidder or due to factors outside of that contractor’s control.
- No Division of Work: Evaluation committees may not divide up the reference check process among evaluators since this creates the risk of biasing the collection of information. All evaluators should have equal access to all reference information.
- Evaluation Records: Evaluators should maintain detailed contemporaneous records of how reference information was used as part of their evaluation process.
These principles are further elaborated on below in the summary of the relevant case law.
In its April 2003 decision in Air-Tite Sheet Metal Ltd. v. Canada (Attorney General), the Newfoundland and Labrador Supreme Court found that the government improperly rejected a bidder after the government’s evaluators acted as self-references to reject one of the bidder’s prior project experience based on their flawed recollection of that bidder’s past performance. The dispute dealt with a Department of National Defence (DND) solicitation for the replacement of a steam distribution system at a Canadian Forces Base. The solicitation required bidders to provide at least two projects of a “similar size, value, and nature” that were completed in the prior five years.
The government rejected the plaintiff’s bid because the evaluators thought that one of the listed projects was never completed. The Court found that the evaluators failed to confirm the information in relation to the past project. As the Court stated, the “DND evaluators did not include this project in their evaluation of Air-Tite’s capability, believing that Air-Tite had been ‘thrown off’ the job for unsatisfactory, untimely, and incomplete performance. This was not true. This conclusion was reached, initially by Crawford, and later accepted by Brighty, apparently on the basis of five-year-old anecdotal information recalled by Crawford.” The Court found the evaluators failed to look beyond that flawed anecdotal information even though “all of the information on Air-Tite’s work on the hangar, any deficiencies as to performance and scheduling, and the state of completion of the mechanical work when Air-Tite left the site, was in the hands of DND and DefCon, those responsible for evaluating Air-Tite’s capability.”
Furthermore, the Court found that the government evaluators would have accepted the project for the past experience requirements had they conducted the proper factual investigation and confirmed that the project was 98 per cent completed. The Court found that the error in the evaluation cost the plaintiff the award of the contract and awarded the plaintiff approximately $250,000 in lost profit damages.
As this case illustrates, purchasing institutions should proceed with caution prior to rejecting a bid based on unclear past performance information, particularly where evaluators are acting as self-references for that bidder’s prior performance based on unreliable past recollections.
In its June 2003 decision in Rockwood v. Harbour Grace Community Youth Network Inc., the Newfoundland and Labrador Provincial Court found that the purchasing institution used an undisclosed reference requirement as a justification to improperly deny a contract award to a sole bidder. The dispute dealt with a tender call for the refurbishment of floors at a community centre. The plaintiff submitted a bid for $2,040. It was the only bid. He was then asked for a reference. By the time his reference contacted the purchasing institution a few days later, the contract had already been awarded to another contractor. The low bidder sued. In asserting its right to reject the low bidder, the purchasing institution relied on a privilege clause, which stated that it did not “bind itself to accept the lowest or any tender.”
The Court determined that the purchasing institution breached its fairness duties by imposing an improvised after-the-fact reference requirement, noting that “within two days of being informed that a reference was required, and before the reference was contacted, another party was informed that his firm would be awarded the contract.” The Court found that the purchasing institution had “undertaken a circuitous route to obtain other bids on the project, and to make certain that they accepted someone other than the plaintiff to undertake the project.” The Court determined that the privilege clause did not permit the use of a hidden reference requirement and awarded the prejudiced bidder lost profit damages.
As this case illustrates, references should not be relied on to arbitrarily deny a contact award to the best bidder.
However, in its May 2005 decision in Carosi Construction Ltd. v. Niagara (Regional Municipality), the Ontario Court of Appeal found that the Region could reject a low bidder based on a bad reference. While the low bidder argued that it had been improperly “blackballed” by a negative reference from another municipality, the Court of Appeal disagreed, noting that the purchaser was entitled to rely on the negative reference as a reason for the low-bid bypass and that “it is wrong to characterize that decision as a “blackballing” of Carosi.”
As this case illustrates, denying a bidder a contract award based on information provided by references is a controversial and high-risk practice that can result in defending a bid protest based on the credibility of third-party reference information. Purchasing institutions should therefore proceed with caution when relying on reference information to deny a bidder a contract award since the reliability of that evidence can be challenged in any resulting legal proceedings.
Further, in its June 2007 decision in Tailwind Associates, the New York State Comptroller upheld a complaint against the Office of Children and Family Services over the use of different evaluators to score different proponent references. The dispute involved a Request for Proposals (RFP) for IT service standby agreements. The Comptroller stated that the rules required “the same individual (or group of individuals) conduct and score all the reference checks. Otherwise, the scoring of the reference checks may be distorted based on the individual biases of the particular evaluators conducting the reference checks or because one may tend to score higher generally than another evaluator.” The Comptroller found that “this approach created an inherently unfair and unbalanced scoring methodology, since it resulted in individuals, with apparently very different scoring philosophies, scoring different references – creating the very real possibility that the outcome of this procurement was determined by which evaluator scored one or more of a particular proposer’s references.” The resulting contract awards were therefore rejected.
As this case illustrates, evaluators should avoid sub-dividing their reference checks since all evaluators who are scoring the related categories should have direct access to all of the collected reference information.
In its October 2009 determination in TELUS Communications Co. v. Canada (Department of Public Works and Government Services), the Canadian International Trade Tribunal ordered a re-evaluation after finding that a low bid was improperly rejected. The dispute involved a telecommunications services RFP that required an online reference check for the top-ranked proponent prior to contract award. When responding to the online questionnaire, one of the low bidder’s references mistakenly answered “no” to one of the questions. This resulted in the bidder’s disqualification. The Tribunal found that the disqualification was improper since the government breached its transparency duties due to ambiguities in the questionnaire relating to past experience.
The Tribunal determined that it was not reasonable to disqualify the proponent without first looking behind the “no” answer from the reference, especially since the response could be interpreted in different ways due to the complexity of the questions posted to the references. The government was ordered to either terminate the contract that was awarded to a competing bidder and award the contract to the complainant or pay the complainant its lost profits.
As this case illustrates, the failure to create clear criteria and procedures for the evaluation or reference information can compromise the defensibility of any resulting evaluation and award decisions.
Similarly, in its December 2009 decision in Turner Pest Control v. University of North Florida, the Florida Division of Administrative Hearings struck down an evaluation process due to irregularities in the scoring of references. The case dealt with an RFP for pest control services. As the Court summarized, the RFP required bidders to submit three references, but the University of North Florida initially took the position that it did not have to contact those references during its evaluation since in its opinion the RFP “permitted the committee to accept the three names and contact information at face value; and it did not presume that letters of reference from those named must be attached to the proposal, although it was acceptable to attach them.” The University also asserted that “the foregoing language of the RFP did not contemplate that the evaluation committee must contact or otherwise verify the references provided.”
As the Court noted, the University scored the references according to the flawed process in which it took its own knowledge of each bidder’s past performance into account in scoring the reference information, which resulted in scoring a bidder with no past experience with the University lower than other bidders.
However, after the second-highest scoring proponent, Terminix, challenged this evaluation and award decision, the University decided to reject the original evaluations, rescind the contract award, and override the original evaluation committee scores with new criteria and scores provided by staff who were not part of the original evaluation committee. The University then directed the original evaluation panel members to approve the new scores. The original highest scoring proponent, Turner Pest Control, challenged this re-evaluation process, maintaining that it was contrary to the RFP rules.
The Court agreed that the attempt to correct the initial evaluation errors was invalid. The Court found that “the first evaluation was flawed because the committee did not contact any references and evaluate their responses, but the first evaluation flaw could not be ‘corrected’ after an award had been posted, unless a written protest was timely filed addressing that issue together with a compliant bond.” The Court also found that the original errors “could not be corrected by requiring the evaluation committee to ‘rubber stamp’ a new procedure (scoring of responses to questions) that was not arrived-at by the committee.” More specifically, the Court noted that “the evaluation committee established by the RFP was not even given the opportunity to rank, on the newly established 1- 10 scale, the answers from the references provided.”
The Court determined that this new procedure “was contrary to law, against the University’s rules, and contrary to the terms of the RFP.” The Court therefore struck down the results of the re-evaluation and directed the University to award the contract to the complainant, who had scored highest after the original evaluation.
As this case illustrates, public institutions should be careful to develop and follow clear evaluation plans when scoring competing proposals, since mid-process improvisations can compromise the defensibility of downstream evaluation and award decisions.
In its 2010 determination in Adware Promotions Inc. v. Canada (Department of Public Works and Government Services), the Canadian International Trade Tribunal upheld the disqualification of a bidder that failed to provide the required reference information. The dispute involved an employee award program services RFP. When the evaluators emailed the complainant’s references, they received an “unknown address” response and then unsuccessfully attempted to contact the references by phone. The evaluators were therefore unable to obtain the reference information within the required timeframes.
While the complainant asserted that the evaluators should have notified the complainant regarding the difficulty in contacting the references, the Tribunal disagreed since the RFP terms did not require evaluators to directly contact proponents who failed to provide proper contact information for their references. While the rejection was upheld, to avoid bid protests over reference-related disqualifications, purchasing institutions should incorporate clear rectification procedures into their solicitation rules that permit bidders to substitute additional references when original references prove unresponsive.
In its June 2010 determination in Valcom Consulting Group Inc. v. Canada (Department of Public Works and Government Services), the Canadian International Trade Tribunal found that the government changed its evaluation rules after the submission of bids and ordered a re-evaluation. The dispute involved an informatics services RFP for the Department of National Defence. The Tribunal determined that the government breached its duty to conduct a fair evaluation by introducing a more lenient process for accepting reference information from government institutions when compared to other third-party references. As the Tribunal noted, “nothing in the RFP could have allowed a bidder to know that a distinction would be made between government and non-government addressing requirements.” The Tribunal therefore ordered a re-evaluation since the post-bid rule change constituted an undisclosed evaluation rule that breached the government’s trade treaty duties.
In its February 2015 determination in 4Plan Consulting Corp. v. Shared Services Canada, the Canadian International Trade Tribunal determined that the government used vague past experience criteria and engaged in the improper use of reference information. The dispute involved a consulting services RFP that failed to clearly define the meaning of past “project” for the purposes of evaluating past experience. The Tribunal also found that the government evaluators improperly relied on information provided by the complainant’s references to determine whether the listed projects in the proposal were separate projects or part of the same single project. The Tribunal determined that relying on the complainant’s references to retroactively define the meaning of “past projects” was a hidden evaluation rule. The Tribunal also determined that the RFP was ambiguous regarding how reference checks would be performed or under what circumstances references would be contacted.
The Tribunal also determined that the government evaluators were inconsistent and arbitrary in how they collected the reference information, which resulted in a fundamental flaw in the evaluation process. The Tribunal awarded the complainant its lost profits.
As this case illustrates, the unclear definition of past experience requirements and the improper use of reference information can undermine the defensibility of an evaluation and award process. Purchasing institutions should therefore transparently describe how they intend to use references as part of their evaluation and award decisions.
Similarly, in its July 2018 decision in Best Climate Control Corp., the Office of the New York State Comptroller overruled the rejection of a bidder after the State University of New York disqualified a bidder for using a University contact as a reference for its bid. The case dealt with an Invitation for Bids (IFB) for HVAC maintenance services, which called on bidders to provide references for their past work. The complainant included a contact at the University as a reference and was disqualified for doing so. It brought a bid protest challenging that decision since the IFB rules did not prohibit the use of University contacts as references. The Comptroller agreed, finding that the IFB rules did not prohibit the use of University references and that the University only made the decision to reject those references after the submission of bids.
The Comptroller determined that the post-bid restriction against using its own contacts as references was an improper rule change that was prejudicial to bidders who relied on University references in their bids. It found that the disqualification of the complainant was therefore invalid. The Comptroller concluded that the flawed bidding process could not stand and struck down the resulting contract award.
As this case illustrates, since mid-process improvisations can compromise the validity of any downstream evaluation and award decisions, purchasing institutions should clearly define the rules around the use of references in their bidding process so that bidders understand the ground rules ahead of time.
Finally, in its May 2019 determination in BEM Systems, Inc., the New York Office of the State Comptroller rejected a contract award due to a flawed evaluation process. The dispute dealt with a Department of Transportation (DOT) RFP for system implementation consulting services. A losing proponent challenged the evaluation process. As the determination detailed, the Department used a multi-staged best and final offer (BAFO) process, which short-listed finalists and allowed those finalists to submit revised proposals for the final evaluation. A losing bidder challenged the evaluation result after its debriefing. The Comptroller found that the two-staged BAFO RFP process that allowed short-listed finalists to submit revised final offers was an acceptable practice under applicable procurement laws. However, the Comptroller found deficiencies in the manner in which the final evaluations were conducted, which included a lack of consistency in the evaluation of competing proponents, a lack of transparency regarding how the group evaluation would be conducted, inconsistency in the stated point weightings for different evaluation categories, unclear evaluation records, and a lack of clarity regarding the scoring of references.
With respect to the use of reference information, the Comptroller found that some of the evaluators “provided scores for references when no reference checks were conducted and no process for independently scoring reference checks existed in the technical evaluation plan.” Furthermore, the decision detailed the lack of proper recordkeeping during the evaluation process, which raised significant concerns about how the individual and group rescoring was conducted after the demonstration stage and after the BAFO submissions.
The Comptroller therefore concluded that, based on its review of the evaluation records, it was “unable to confirm that DOT’s evaluation and selection decisions were in accordance with the evaluation methodology set forth in the RFP and the evaluation plan and whether the evaluation conducted resulted in a best value award.” The contract award resulting from that evaluation process was rejected due to those evaluation flaws.
As this case illustrates, while multi-staged RFP evaluation processes may be acceptable under public sector procurement rules, those multi-staged evaluations must be conducted in a manner that is consistent with the criteria and process rules established in the RFP, and those evaluations, and the use of reference information as part of those evaluations, must be well-documented in order to be defensible.
As this long line of cases illustrates, purchasing institutions need to double down on their due diligence when it comes to the proper use of reference information since the failure to meet proper legal standards can undermine the defensibility of their evaluation and award decisions.
