This article is an excerpt from The Art of Tendering: A Global Due Diligence Guide, which is available for purchase.
In its September 2008 decision in McLaughlin & Harvey Ltd v. Department of Finance & Personnel, the High Court in Northern Ireland, Queen’s Bench Division, ruled that the government’s failure to disclose its evaluation sub-criteria was a breach of applicable transparency standards. The dispute dealt with a solicitation to establish a framework agreement for the future award of a number of construction contracts over a four-year period for projects including urban regeneration and education, arts and sports developments at a cost of between £500 million and £800 million. The plaintiff, who was a losing bidder, claimed that during its debriefing it learned that the government’s Central Procurement Directorate (CPD) had scored the bids using undisclosed evaluation criteria provided to evaluators in an internal evaluation guide that was not disclosed to bidders. The plaintiff argued that this lack of disclosure was in contravention of applicable European transparency requirements and that these hidden factors impacted the final evaluation rankings:
In this case a contract notice was published, as required, in the official journal of the European Union on 15 March 2007. The primary tender documents were issued on 24 April 2007. The tender of the plaintiff was submitted on 5 October 2007. On 17 December 2007 they were informed that they had been unsuccessful. They sought a debrief meeting as they were entitled to but for various reasons this did not take place until 10 January. The plaintiff would say that it was only on this occasion that they realised and learnt that the defendant through the CPD had marked the plaintiff’s tenders alongside all the other tenders with a particular methodology that had not been disclosed in advance to the plaintiff. The heart of the case is the plaintiff’s attack on the very fact of that methodology which it alleges constitutes new and undisclosed criteria relied on by the defendant in breach of the European requirement of transparency and in a way that was unfair to the plaintiff. As the plaintiff came sixth in the competition only 1% behind the contractors placed fifth and fourth even a modest improvement in its marking by a proper approach, it contends, would materially affect the outcome.
The Court turned to leading European case law to confirm the applicable disclosure duties that apply to evaluation criteria. Citing a leading European decision, the Court confirmed that domestic law requires the general disclosure of evaluation criteria:
In Universale-Bau AG, Case C – 470/99, [2002] ECR I – 11617 the Sixth Chamber of the European Court of Justice was considering a precursor directive to the current directive. Paragraph 99 of that judgment is relevant and I quote it in full:
It is therefore clear that the interpretation according to which, where, in the context of a restricted procedure, the contracting authority has laid down prior to the publication of the contract notice the rules for the weighting of the selection criteria it intends to use, it is obliged to bring them to the prior knowledge of the candidates, is the only interpretation which complies with the objective of directive 93-37, as explained in paragraphs 88 to 92 of this judgment, since it is the only one which is apt to guarantee an appropriate level of transparency and, therefore, compliance with the principle of equal treatment in the procedures awarding contracts to which that directive applies.
This is clearly one of the decisions which informed the current Directive. It is relied on by the Plaintiff here.
Turning to a second leading European decision, the Court then confirmed that the duty to disclose evaluation criteria under the European procurement directive extends to disclosing both the main evaluation criteria as well as the sub-criteria:
Perhaps the most important judgment for the purposes of this hearing is the decision in ATI v. ACTV Venezia et alia, Case C – 331/04, [2005] ECR 1-10109. In those proceedings the defendant and the province and commune of Venice were awarding a contract for public passenger transport services on the basis of the economically most advantageous tender. The parties were given four award criteria with a maximum of 60% marks for the first criterion and maxima of a smaller order for the other criteria. Subsequently after the submission of tenders but before the envelopes were opened the jury weighted the 25% points available under criterion 3 into five sub-headings. This was analogous to the actions of the defendant here. A disappointed contractor challenged that step before the regional administrative tribunal but failed. They appealed to the Consiglio Di Stato which referred certain questions to the court for preliminary ruling. I set out the decision of ECJ in extenso:
As a preliminary point, it must be observed, as the referring court pointed out, that, by the decision at issue in the main proceedings, the jury simply decided how the 25 points allocated for the third award criterion had to be distributed among the five subheadings in the contract documents.
Accordingly, the questions referred should be understood to relate essentially to the question whether Article 36 of Directive 92/50 and Article 34 of Directive 93/3 8 must be interpreted as meaning that Community law precludes a jury from attaching specific weight to the subheadings of an award criterion which are defined in advance, by dividing among those subheadings the points awarded for that criterion by the contracting authority when the contract documents or the contract notice were prepared.
First, as the Austrian Government rightly observed, the provisions of Article 36 of Directive 92/50 and Article 34 of Directive 93/38 cannot be applied simultaneously to the same set of facts. However, the provisions cited in the questions referred for a preliminary ruling have substantially the same wording (see Case C-513/99 Concordia Bus Finland [2002] ECR P7213, paragraph 91). Therefore, the Court can give a proper answer to the question as reformulated without there being any need for it to rule as to which of the two directives is applicable in the case in the main proceedings.
Next, it must be observed that the award criteria defined by a contracting authority must be linked to the subject matter of the contract, may not confer an unrestricted freedom of choice on the authority, must be expressly mentioned in the contract documents or the tender notice, and must comply with the fundamental principles of equal treatment non-discrimination and transparency (see Concordia Bus, cited above, paragraph 64).
In the present case, it must be observed, in particular, that the duty to observe the principle of equal treatment lies at the very heart of the public procurement directives (see Concordia Bus Finland, paragraph 81) and that tenderers must be in a position of equality both when they formulate their tenders and when those tenders are being assessed (see Case C-19/00 SIAC Construction [2001] ECR 17725, paragraph 34).
It must also be observed that, in accordance with Article 36 of Directive 92/50 and Article 34 of Directive 93/3 8, all such criteria must be expressly mentioned in the contract documents or the tender notice, where possible in descending order of importance, so that operators are in a position to be aware of their existence and scope (see Concordia Bus Finland, paragraph 62).
Similarly, in order to ensure respect for the principles of equal treatment and transparency, it is important that potential tenderers are aware of all the features to be taken into account by the contracting authority in identifying the economically most advantageous offer, and, if possible, their relative importance, when they prepare their tenders (see, to that effect, Case C-87/94 Commission v Belgium [1996] ECR 1-2043, paragraph 88, and Case C470/99 Universale-Bau and Others [2002] ECR 1-11617, paragraph 98).
Finally, it is for the national court to assess, in the light of these rules and principles, whether, in the case in the main proceedings, the jury infringed Community law by applying a weighting to the various subheadings of the third criterion for the award of the contract.
In that regard, it must be determined first whether, in the light of all the relevant facts of the case in the main proceedings, the decision applying such weighting altered the criteria for the award of the contract set out in the contract documents or on the contract notice. If it did the decision would be contrary to Community law. Second, it must be determined whether the decision contains elements which, if they had been known at the time the tenders were prepared, could have affected that preparation. (My underlining). If it did the decision would be contrary to Community law. Third, it must be determined whether the jury adopted the decision to apply weighting on the basis of matters likely to give rise to discrimination against one of the tenderers. If it did the decision would be contrary to Community law.
…
I direct attention to paragraphs 28 and 29 in particular of the judgment. The court must determine whether the decision contained elements which if they had been known at the time the tenders were prepared, “could have affected that preparation”. If so the decision is contrary to Community law. Note that the court is saying “could” have affected the preparation and not “would” have affected the preparation. The court does not use the word criteria there and as indicated above it seems to me it is contemplating criteria and weighting, including, clearly in the context of paragraph 25 and of paragraph 32, the weighting of sub criteria.
Finally, the Court cited a third leading European case that confirms that, in addition to disclosing evaluation criteria and sub-criteria, to comply with the transparency standards under the European procurement directive, a purchasing institution is also required to disclose the applicable weightings:
The First Chamber of the court had occasion to return to these matters in Lianakis v. Alexandroupolis & Others, Case C 53206 (24 January 2008). These proceedings related to contracts related to the defendant municipality. It should be noted that though this case was only decided in the current year the Council Directive which was applicable at the relevant time was Directive 92-50-EEC of 18 June 1992. At paragraph 21 the European Court said:
By its question, the referring court asks in essence whether, in a tendering procedure, Article 36(2) of Directive 92-50 [the equivalent of Article 53(1) in the current directive] precludes the contracting authority from stipulating at a later date the weighting factors and sub criteria to be applied to the award criteria referred to in the contract documents or contract notice.
…
Having regarding to the foregoing, the answer to the question referred must therefore be that, read in the light of the principle of equal treatment of economic operators and the ensuing obligation of transparency, Article 36(2) of Directive 92/50 precludes the contracting authority in a tendering procedure from stipulating at a later date the weighting factors and sub criteria to be applied to the award criteria referred to in the contract documents or contract notice.
The Court then cited domestic case law rulings that applied the European precedents. As the Court noted, to facilitate competition, enable impartiality, preclude favouritism or arbitrariness, and promote a level playing field where all bidders know the criteria against which their bids will be assessed, domestic case law recognizes that the principles of equal treatment and non-discrimination require a public institution to clearly define the requirements of a tendered contract and the criteria that will govern the award of that contract:
In Regina (Law Society) v. Legal Services Commission [2008] 2WLR 803 the Court of Appeal in England had to consider the applicability of the Public Contracts Regulations 2006 to contracts for the provision of legal services to be awarded by the Legal Services Commission. In the judgment of the court Lord Phillips of Worth Matravers CJ helpfully sets out at paragraphs 42 and 43 the importance and reasons for the principle of transparency and its application in these cases:
In Commission of the European Communities v French Republic (Case-C-340/02) [2004] ECR 1-9845, para 34 the European court said that:
The principle of equal treatment of service providers, laid down in…the Directive, and the principle of transparency which flows from it…require the subject matter of each contract and the criteria governing its award to be clearly defined.
The rationale of the principle has been expressed in a number of different ways. (i) First, it enables the contracting authority to satisfy itself that the principles of equal treatment and of non-discrimination on the grounds of nationality have been complied with: Telaustria Verlags GmbH v Telekom Austria AG (Case C-324/98) [2000] ECR 1-10745, para 61; SIAC Construction Ltd v Mayo County Council (Case C-19/00) [2001] ECR 1-7725, para 41 and Commission of the European Communities v French Republic (Case C-340/02) [2004] ECR 1-9845, para 34. (ii) Second, it facilitates competition: Telaustria Veriags GmbH v Telekom Austria AG (Case C-324/98) [2000] ECR 1-10745, para 62; Parking Brixen GmbH v Gemeinde Brixen (Case C-458/03) [2005] ECR I-88, paras 50, 52 and Impresa Portuale di Cagliari Sri v Tirrenia di Navagazione SpA (Case C-174/03) (unreported) 21 April para 75, per Advocate General Jacobs. (iii) Third, it enables the impartiality of procurement procedures to be reviewed: Telaustria Veriags GmbH v Telekom Austria AG (Case C-324/98) [2000] ECR 1-10745, para 62 and Impresa Portuale di Cagliari Sri v Tirrenia di Navagazione SpA (Case C-174/03), para 75, per Advocate General Jacobs. (iv) Fourth, it precludes any risk of favouritism or arbitrariness on the part of the contracting authority: Commission of the European Communities v CAS Succhi di Frutta SpA (Case C-496/99 P) [2004] ECR 1-3801, para iii. (v) Fifth, it promotes a level playing field by enabling all tenderers to know in advance on what criteria their tenders will be judged and those criteria are assessed objectively; SIAC Construction Ltd v Mayo County Council (Case C-19/00) [2001] ECR 1-7725, para 38, per Advocate General Jacobs.
In considering whether the defendant had disclosed its evaluation sub-criteria in sufficient detail, the Court considered the meaning of “criteria” and “sub-criteria” and applied the plain dictionary meaning to those terms:
Consideration
In the absence of any definition of criteria or sub criteria in either the European Directive or the 2006 Regulations one looks to the ordinary meaning of the word. According to Chambers 20th Century dictionary a criterion is a means or standard of judging; a test; a rule, standard or canon. The root of it is the Greek word krites meaning a judge. The Shorter Oxford Dictionary defines it as a canon or standard by which anything is judged or estimated or, a characteristic attaching to a thing, by which it can be judged or estimated. When one bears that in mind it seems to be that the defendant’s reference to this as evaluation guidance is almost an admission that these or some of these are indeed criteria. They are being used to evaluate the tender bids i.e. to judge them. To call them a scoring methodology or a marking sheet is really saying the same thing e.g. at CO2 there is a topic : “realistic processes to quantify and allocate risk”. At DO1 there is : “industry best practice in innovation”. Did the bidder have this? Although obviously the language varies from topic to topic they seem at least or in some cases more like criteria than phrases like “whole life costs” or “target prices”. These were not described as criteria in the original documents and they are more like an attempt to categorise the various relevant factors or criteria.
The word criteria is not defined in Stroud but one does find it in Words and Phrases Legally Defined (4th Edition) Volume 1, p.552. The issue was whether a 12 month time limit in the regulations made under an Australian Act constituted a “criterion” within clause 31(3):
The case for a wide meaning of Section 31(3) . . . is also supported by other factors. Firstly, the ordinary meaning of the word “criteria” supports a wide construction. The Oxford English dictionary, 2nd Edition, Volume 4, page 29 defines “criterion” as meaning a “a test, principle, rule, canon or standard, by which anything is judged or estimated”. The Macquarie Dictionary, revised edition at page 437, defines “criterion” as “a standard of judgment or criticism; an established rule or principle for testing anything”. These definitions are sufficiently broad to embrace a time limit for the lodging of a visa application”. Pillay v. Minister for Immigration and Multicultural Affairs [2000] FCA 112 at para [32], per Carr, Sackville and Nicholson JJ.
The Court ultimately determined that while the defendant was responsible for disclosing all of its criteria and sub-criteria, many of the sub-criteria that were relied on by the evaluators were not disclosed to bidders and were not so obvious so as to be assumed by bidders in the preparation of their bids. The Court concluded that those sub-criteria should therefore have been disclosed to meet applicable transparency standards:
Admittedly one could argue that there was not a true breach of transparency as long as these topics were reasonably predictable by an informed bidder and were consistent with the published criteria. I find that that was true of these topics. But that argument is undermined by the fact that they are then given weightings which vary from topic to topic. It did not seem to me to be made out, or indeed even argued, that the weightings for each topic were predicted or predictable by a reasonable bidder. There may not have been anything irrational about them but they were very much subjective judgments formed by the panel collectively. I find that the sub-weightings were not predictable. It seems to me therefore that on that basis the topics as I have called them beginning with ‘data base of cost information’ under whole life costs in BO1 down to ‘liaison with employer/ICT’ in EO2 are elements which ought to have been disclosed with their weightings to the plaintiff and other bidders.
…
It seems to me that the language of ATI in particular is designed to secure, as it says, that the bidders know all the elements or sub elements which could affect their preparation of the bid. I readily accept that that is a significant onus on a contracting authority, particularly where as here they are dealing not with a single contract for the provision of bus services in Venice, (and one notes it is for Mestre rather than the ancient city), but with a Framework Agreement covering 4 years and a wide range of different construction and functional tasks. However having chosen to have a Framework Agreement for all those tasks it is clear that at least so far as the 39 items are concerned that they and their weightings did have to be furnished to the bidders in advance to comply with the decisions of the European Court. I find as a fact that this material, and the evidence before me, could have affected the preparation of the document. Indeed I consider it likely that it would have affected such preparation. A bidder would be bound to take it carefully into account in allocating their strictly limited number of characters in each section.
…
I therefore find the plaintiff’s case made out to the extent that the 39 matters to be found from BO1 to EO2 ought to have been disclosed with their respective weightings to the bidders in advance. I now proceed to deal with other subsidiary or related matters.
The Court therefore concluded that since the defendant breached the transparency standards established under European case law, it, in turn, breached its fair process duties under the Public Contracts Regulations 2006.
The parties were unable to agree to a remedy, so they brought the matter back to court for a remedy ruling. In its October 2008 decision in McLaughlin & Harvey Ltd v. Department of Finance & Personnel, the High Court in Northern Ireland, Queen’s Bench Division, determined that rather than adding the complainant to the framework roster or permitting protracted proceedings for the calculation of lost profits, the most appropriate remedy was to set aside the framework awards and require a re-tender.
The Court was not persuaded with the option of adding the complainant to the framework agreement list, noting that counsel for the complainant was unable to offer a precedent for that type or remedy. Rather, the Court concluded that setting aside the original awards would be the most appropriate remedy even if the original winners would now have to rebid:
I do not accept the submission in the defendant’s skeleton argument that the defendant would be at risk of significant litigation from the five successful economic operators (or groups of economic operators) if the court were to grant the plaintiff here the second of it’s preferred remedies. At the very most, on a setting aside, those other parties might conceivably complain at having to re-apply. If they were unsuccessful the second time round they might well be disappointed but they would be in a similar position to the plaintiff in McConnell Archive Limited v Belfast City Council 2008 NICH 3. They have not succeeded the second time but, subject to any further order of the court, the second procedure was lawful while the first procedure was conducted unlawfully. They have not lost anything to which they were lawfully entitled. If in fact they are the best economic operators under the Framework Agreement, it is likely that they will succeed on a re-run of the Framework Agreement procedure. If they do not it is because the second procedure is fairer and more transparent than the first.
In coming to this conclusion, the Court also determined that setting aside the original awards was preferable to engaging in lengthy court proceedings to determine lost profits:
The issue before me now is a different one. Which is the most appropriate remedy to grant to the plaintiff, it having succeeded in proving a breach of duty? The assessment of the loss of profits might well have to wait for some time, perhaps years, to allow the court to make a reasonable estimate of the profits which the successful economic operators will enjoy from the Framework Agreement. I consider that would be necessary here and clearly it is not ideal.
As this ruling illustrates, a lack of transparency in evaluation criteria can undermine the legal defensibility of the resulting contract awards and expose those awards to set aside orders and other potential remedies.
In its subsequent September 2011 decision in McLaughlin & Harvey Ltd v. Department of Finance & Personnel, the Court of Appeal in Northern Ireland upheld these trial decisions. While the Court of Appeal found that the failure to disclose the weightings for the 39 sub-criteria contained in the evaluation guides was not automatically a breach of the transparency rules since there was no absolute duty to disclose evaluation sub-weightings, it concluded that there was in fact a duty to disclose that information where that information could impact how a bidder would prepare its bid:
The learned trial judge concluded, however, at paragraph 20 of his judgment that the weightings of the sub-criteria were elements which if they had been known in advance at the time the tenders were prepared could have affected that preparation. He laid considerable emphasis on the fact that the respondent only had to show a possibility rather than a likelihood. It can be seen from J Varney and Sons that the mere assertion that the preparation of the tender could or would have been affected is not sufficient to engage this exception. There must be some evidential material.
Within the liability judgment there is no such material set out upon which the learned trial judge based his conclusion but in fairness to him the issue was discussed within the initial judgment at the interlocutory stage. The case made by Alan Coulter, the procurement director of the respondent, was that the answers to the questions were electronically locked so as to prevent any answer in excess of 6,000 words to each question. The respondent had exceeded this in the first draft of its answer to B01 and had to delete material in order to ensure compliance. Mr Coulter complained that over 2,000 words of his answer to the question at B01 arose from the question but was not taken into account in the evaluation. The respondent’s answer to question B01 was marked down because of a lack of detail on various aspects which he maintained would have been addressed if the criteria and weighting were known. In substance, therefore, the complaint was that some elements of the question did not have weight attached to them, a matter not known to the bidder who wasted part of his answer on them.
The learned trial judge clearly accepted some aspect of this because at paragraph 54 of his judgment he noted that the submission by the respondent covered aspects which were not taken into account in the evaluation material and it is tolerably clear that the learned trial judge was not persuaded that the respondent had been properly rewarded for those aspects. It appears to have been accepted that some allowance for that should have been made.
This complaint was supported by the report of Mr Hackett, the expert retained on behalf of the respondent. He and Mr Rowsell, the expert for the appellant, both gave evidence in the court below. Neither party submitted any transcript of the evidence nor was there any analysis of the reports or the evidence to seek to undermine the learned trial judge’s conclusion that the failure in this case to identify the sub-headings and the weighting attached to them could have affected the preparation of their tender.
We accept that the tests in ATI have to be examined in the context of the obligation of transparency but that obligation is properly stated in recital 46 of the Directive as an obligation to ensure that tenderers are reasonably informed. At paragraph 46 of his judgment the learned trial judge placed some emphasis upon the need for bidders to know all elements which could affect the preparation of the bid which is derived from paragraph 24 of ATI. It is clear, however, from paragraph 32 of the judgment that in many cases the absence of knowledge about the weighting of the sub-headings will not lead to any breach of community law and it will be for the bidder in each case to produce a case supported by adequate evidence before one of the exceptions under paragraph 32 of that judgment could be established.
The Court therefore rejected the government’s appeal, finding no reason to overturn the trial judgments. As this case illustrates, public purchasing institutions are subject to strict transparency standards that subject them to a high level of disclosure when drafting their evaluation criteria. Burying any of those details within internal documents that are not disclosed to bidders can undermine the defensibility of any resulting evaluation. Rather than exposing themselves to after-the-fact arguments over whether providing more detail would have changed how the bidders prepared their bids, when in doubt, project teams should err on the side of caution and disclose all of the criteria, sub-criteria and weightings that they intend to rely on in their evaluations.
