Audit Calls for Enhanced Group Scoring Procedures

By: Paul Emanuelli

This article is an excerpt from The Art of Tendering: A Global Due Diligence Guide.

In the December 2019 report entitled City of Palm Bay, Florida’s Auditor General found that the City needed to enhance its evaluation and award procedures, bolster its group evaluation procedures, and enhance its record-keeping when contract award decisions deviated from evaluation team recommendations.

After reviewing one of the City’s major infrastructure projects, the Auditor General found that the City needed to improve the transparency and defensibility of evaluation procedures:

The City is required to procure design-build services in accordance with State law¹⁵ and the City Procurement Manual. On January 16, 2017, the City issued a request for proposal (RFP) for design-build services for the SJHP Project. The SJHP Project RFP provided that the City would be utilizing a two-phase process whereby an evaluation team (e-team) would evaluate respondents who submitted proposals in response to the RFP.

…

Our examination of City records and discussions with City personnel regarding the selection of the design-build firm for the SJHP Project disclosed areas in which the City’s administration of the Project could have been enhanced. These areas included, for example, the use of a design criteria professional, procurement of design-build services, appointment of e-team members, RFP response scoring instructions, RFP scoring transparency, and RFP scoring methodology.

Citing public procurement standards established by NIGP: Institute for Public Procurement (NIGP), the Auditor General underscored the importance of providing evaluation team members with better instructions and training and the importance of documenting the recommendations made to evaluation teams regarding the use of transparent evaluation criteria, consistent scoring procedures, and proper record-keeping practices:

Scoring Instructions. Regarding the use of predetermined and established proposal evaluation criteria, the NIGP, in its Global Best Practices, recommends:

• Use of clearly defined criteria for procurement decisions.

• A clear understanding by evaluation committee members of how criteria and scoring should be applied.

• Use of a consistent approach when scoring against preannounced criteria.

• Transparency of the selection criteria and evaluation process.

Consistent with the NIGP’s recommended best practices, providing e-team members with written instructions that explain how criteria and scoring should be applied would provide additional assurance that members use a consistent approach to identify the most favorable proposal. However, for the SJHP Project, e-team members were not, of record, provided such instructions. Specifically:

• Although the RFP provided detailed instructions for information to be included by the responding firms in their proposals for consideration by e-team members in scoring proposals for the four Phase I criteria (project team, project team location, project management, and project approach), the RFP did not describe how the criteria and scoring would be applied to the information provided by the respondents. For example, although the RFP required respondents to provide “an organizational chart for the project team and identify any utilization of any firms that are minority business enterprises,” the RFP did not instruct e-team members on how to apply this information for scoring purposes and, thus, it was not clear as to whether a respondent should have received a higher score for utilizing minority business enterprises.

Although, in response to our inquiries, City personnel cited several ways in which e-team members were instructed regarding the proposal evaluation process, we were not provided records evidencing that e-team members were provided guidance on how the Phase 1 criteria and scoring should be applied to the information provided by the respondents.

• For Phase 2, each respondent was sent a letter listing questions to be answered during an oral presentation. We inquired with City personnel as to what instructions e-team members were provided regarding how to apply the scoring to the respondents’ answers and presentations. In response, City personnel described several documents provided to e-team members and stated that e-team members “were clear on the process when developing the questions, how the questions would be sent, how the responses would be received, and the format of the oral discussions.” However, we were not provided records evidencing that e-team members were provided instructions on how to apply scoring to the respondents’ answers provided during oral presentations.

More specifically, the Auditor General emphasized the importance of enhancing the transparency and consistency of group evaluation scoring procedures. With respect to scoring transparency, the report noted that evaluation teams needed to stay within the guardrails of the specified evaluation criteria during their group discussions and needed to record the reasons for the scores they awarded:

Scoring Transparency. Consistent with the NIGP’s recommended best practices, the City should ensure full transparency regarding the proposal evaluation process. Although the City made some efforts to be transparent as to how each e-team member scored proposals during Phase 1 and Phase 2, such efforts could be enhanced. Specifically:

• Each e-team member prepared a scoresheet showing the member’s assigned scores based on the RFP-specified Phase 1 criteria. While the scoresheets indicated the total score assigned to each criterion, the scoresheets did not indicate how each e-team member arrived at the assigned score. We listened to a recording of the e-team’s March 22, 2017, meeting, at which Phase 1 scoring of the proposals was discussed and noted that the e-team members made some comments regarding their evaluation of the criteria. However, it was not always readily apparent as to which of the RFP-specified criteria the comments pertained.

• For Phase 2, each e-team member prepared a scoresheet showing the member’s assigned score based on their evaluation of the respondents’ oral presentations. The scoresheets for two e-team members did not indicate how they arrived at the assigned scores considering the information provided by the respondents during the oral presentations. The scoresheet for the other e-team member included some comments regarding the content of the presentations; however, it was not clear to which presentation questions the comments pertained. We listened to a recording of the e-team’s June 6, 2017, meeting at which scoring of the oral presentations was discussed and noted that the e-team members made some comments regarding their evaluation of the oral presentations. However, it was not always readily apparent as to which of the presentation questions or respondents the comments pertained.

Although guidelines provided to e-team members required each member to “review in-depth and rate the written proposals as outlined in the RFP,” the members were not specifically required to make notes as to how they applied the criteria and scoring to the information provided by respondents to arrive at the assigned scores. Maintaining documented and sufficiently detailed e-team explanations on how criteria and scoring are applied would provide more transparency in the competitive selection process and additional assurance that e-team members considered all relevant RFP-specified factors.

Furthermore, the Auditor General noted that evaluators needed to stay within the parameters of the established scoring methodologies when allocating partial points within each evaluation category:

Scoring Methodology. The RFP required that responses be scored using one of six possible scores ranging from 0 (when no information was provided for a criterion) to 5 (when a proposal exceeded the minimum requirements in most aspects for a criterion). Contrary to the RFP provisions, e-team members did not always limit their scores to 0, 1, 2, 3, 4, or 5, as in several instances they used fractional scores. Specifically, one e-team member used fractions to score all four Phase 1 criteria, and all three e-team members used fractions to score the oral presentations. In response to our inquiry, City personnel indicated that fractional scores had been allowed in many past evaluations for other RFP procurements over the years.

Scoring proposals consistently in accordance with the RFP-specified methodology helps avoid the appearance of improprieties in the procurement process.

The Auditor General made a series of recommendations on how to improve evaluation practices to make those practices more legally compliant and defensible. Those recommendations included implementing proper conflict screening procedures and providing detailed written instructions to evaluators on how to consistently score proposals based on the criteria and procedures specified in the related solicitation documents. These recommendations serve as useful guidelines to evaluation teams across all jurisdictions.