UDOT Audit

8/7/2019 UDOT Audit

1/34

AUSTON G. JOHNSON, CPA

UTAH STATEAUDITOR

UTAH DEPARTMENTOFTRANSPORTATION

I-15 UTAH COUNTYCORRIDOR EXPANSION

Keeping UtahFinancially Strong

Findings and Recommendations

Report No. 11-DOT-A

8/7/2019 UDOT Audit

2/34

i

Auston G. Johnson, CPAUTAH STATE AUDITOR

STATE OF UTAH

Office of the State AuditorUTAH STATE CAPITOL COMPLEX

EAST OFFICE BUILDING, SUITE E310

P.O. BOX 142310SALT LAKE CITY, UTAH 84114-2310

(801) 538-1025

FAX (801) 538-1383

DEPUTY STATE AUDITOR:

Joe Christensen, CPA

AUDIT DIRECTORS:

Van H. Christensen, CPADeborah A. Empey, CPAStan Godfrey, CPAJon T. Johnson, CPA

MANAGEMENT LETTER NO. 11-DOT-A

April 11, 2011

Gary R. Herbert, GovernorState of Utah

The Transportation Commission, Audit Committee,and

John R. Njord, Executive DirectorUtah Department of Transportation

At the request of Governor Herbert, the Office of the Utah State Auditor has performed a review ofthe Utah Department of Transportations (UDOT) evaluation and selection process, technologyusage, and human resource management related to UDOTs I-15 Utah County Corridor Expansion(CORE) project. Below is a description of the CORE evaluation and selection process, followed bythe procedures we performed and a summary of our findings. The detailed findings andrecommendations can be found where indicated in the table of contents.

I-15 CORE PROJECT EVALUATION AND SELECTION PROCESS

UDOT elected to procure a design-builder for the CORE project to design and construct the bestvalue project possible within the $1.725 billion budget. This was accomplished through a best valuedesign-build procurement approach, which gives the design-builder a fixed price and encouragesthem to propose innovative and creative solutions for achieving the goals of the project. The contractis awarded to the proposer offering the best qualitative proposal for the established price. Thisapproach is relatively new and differs from the traditional best-price procurement approach where alow bid is the typical goal. The CORE project is the first project of this kind for UDOT and thebiggest project of this kind in the country.

In April 2009, UDOT initiated the procurement process to select a design-builder for the COREproject and in June 2009, issued a Request for Proposal (RFP) for the project. UDOT then

conducted weekly confidential one-on-one meetings with the proposers to answer questions related tothe requirements of the RFP and to discuss technical concepts. On November 19, 2009, theproposers delivered their proposals to UDOT. Proposals were received from three differentproposers: Flatiron/Skanska/Zachry Joint Venture, Provo River Constructors, and TimpanogosTransportation Constructors. Once the proposals were received, the evaluation and selection processbegan, which is the focus of our review. On December 9, 2009, the Selection Official selected ProvoRiver Constructors as the winning proposer of the CORE project.

The flowchart on the following page gives an overview of the Evaluation and Selection Process.

8/7/2019 UDOT Audit

3/34ii

I15 CORE PROJECT

EVALUATION AND SELECTION PROCESS OVERVIEW

REQUEST FOR PROPOSAL (RFP)

1. Outlined project goals and values.

2. Divided the proposal into four main categories. 1)ProjectDefinition(Scope/Pavement&Bridges); 2)MaintenanceofTraffic; 3)Schedule;4)Pass/Fail

3. Defined the specificcriteria and factors to be considered in the evaluation andselection process.

4. Determined the significance of each specific criteria and combined them into

groups (High, Medium, Low) within the first three categories according to their

importance to the overall project. These are referred to as combinedgroupsofcriteria in the findings. Pass/fail category was only rated as pass or fail.

EVALUATION AND SELECTION

MANUAL

1. Defined the AdjectivalRatingsand set numeric scoring rangeswithin each rating: Exceptional

(85100); Very Good (7084);Good (5569); Acceptable (< 54).

2. Detailed written instructions for

the evaluation and selectionprocess.

3. Defined roles of various teams,

committees, etc.

TECHNICAL EVALUATION TEAMS(Approximately 65 members)

Team members were assigned to categories based on their expertise.

1. Evaluated the technical elements of the proposal.

2. Compared each proposal to the RFP. Assessed eachproposers strengths and weaknesses in meeting thespecific criteria of the project.

3. Recommended an adjectival rating for each combinedgroup of criteria based on how well the proposer metthe qualifications for that criteria.

4. Completed forms in Appendices of Manual.

5. Made presentation to Selection RecommendationCommittee.

6. Wrote up the Proposal Rating and Ranking SummaryReports.

Rated whetherthe proposer

met the

applicable

requirements.

PROJECT

DEFINITION

MAINTENANCE

OF TRAFFICSCHEDULE PASS/FAIL

SELECTION RECOMMENDATION COMMITTEE (SRC)(7 members)

1. Reviewed information presented by the Technical Evaluation Teams and evaluatedthe proposals in their entirety.

2. Discussed each point of the proposals, obtained a consensus.

3. Determined the finaladjectival ratings for combined groups of criteria.

4. Assigned numeric scores to each of the adjectival ratings within the predeterminedranges.

5. Summarized results and presented final recommendation to the Selection Official.

SELECTION OFFICIAL

(Deputy Director of UDOT)

Made final determination of winning proposer. GOVERNMENT OFFICIALS

Observed the Selection Official

during the final stage of the

procurement process.

ADVISORS(3 people)

Provided

guidance at

all levels ofthe

evaluation

andselection

process to

helpensure the

process

was

followed.

EVALUATION

OVERSIGHTTEAM

(3 people)

Provided an

independentobservationof the

evaluation

and selection

process bycomparing it

to the

Evaluation

and SelectionManual and

the

Instructions

to Proposers

8/7/2019 UDOT Audit

4/34

iii

PROCEDURES PERFORMED AND SUMMARY OF FINDINGS

We performed the following procedures for the period related to the 2009 CORE evaluation andselection process:

1. We reviewed the procedures related to the evaluation and selection of the CORE contract foradequacy and compliance with UDOTs policies and procedures. Our testwork also included

comparing the CORE process to the bid evaluation and selection processes used for otherprojects, interviewing employees and others involved in the CORE process, and reviewing andtesting the adjectival ratings and numeric scoring summaries on a sample basis for accuracy andsupporting documentation.

We identified various weaknesses in the CORE evaluation and selection process related to:a. Blinding as reported in Finding No. 1.b. Scoring as reported in Finding Nos. 2 and 3. Because of subjectivity and lack of

documentation, we were unable to conclude regarding the actual legitimacy of eachparticular score or determine how the Selection Recommendation Committee (SRC)arrived at the scores, other than they fit into pre-determined ranges. We also identifiedfactors such as scope restriction, condensed evaluation criteria, and inadequatelydefined ratings that might have contributed to the proximity of the scoring resultsand/or influenced the results.

c. Roles, responsibilities, and reporting as reported in Finding Nos. 4 through 6.Despite the identified weaknesses with the process, all evaluators we interviewed indicatedthat they agreed with the decision to award the contract to Provo River Constructors andthat they did not experience undue influence in their decisions.

Initially we heard allegations that UDOT changed scores of at least one proposer during theevaluation and selection process, implicating improprieties in their process. However, wefound no evidence that scores were changed. Although we did find that some adjectivalratings recommended by the technical evaluation teams were changed by the SRC duringtheir part of the evaluation and selection process, the changes appeared reasonable and arein compliance with the Evaluation and Selection Manual as noted in Finding No. 2a.

UDOT followed the CORE evaluation and selection process in accordance with the COREEvaluation and Selection Manual and the Instructions to Proposers. However, although theEvaluation and Selection Manual is a documented process, it is not part of UDOTsapproved policies and procedures. We recommend that UDOT incorporate its newdocumented process for Best Value Design-Build projects into its Policies and ProceduresManual (see Finding No. 7).

2. We reviewed access to the Project Development Business System (PDBS) and the CORE Portalto determine if any unauthorized access to the information occurred in either system. The PDBSwas used to record the price allocations of each CORE proposal, and the CORE Portal is thecomputer system used to accumulate and store CORE proposal and evaluation information.

8/7/2019 UDOT Audit

5/34

iv

a) The Project Development Business System

A weakness in PDBS resulted in a breach of confidentiality and information wasimproperly disclosed by a UDOT employee to a proposer resulting in improperdisclosure of information as addressed in Finding No. 8.

b) The CORE Portal

Based on our review of two areas within the CORE Portal computer system, we found thefollowing:

Within the Procurement Portal, which allows close collaboration between internaland external project teams, access is closely controlled by limiting or granting access toindividuals for specific pages, directories, and/or documents on the site.

Within the Evaluation Portal, which is a separate security-controlled portal for theproposal evaluation and selection process set up for exclusive use by the evaluationteams, individual access was approved by project leadership and included only thosepersons with a direct role in the evaluation and selection process. These access controlsappeared reasonable to ensure that only appropriate personnel had access to theevaluation site. We were unable to determine who had actual access during the COREevaluation and selection process because security tables are only a snapshot ofcurrent user access at a point in time and past logs have been overwritten with recentlogs. Maintaining only current logs is a normal business practice and is consideredreasonable. We were, however, able to obtain a list of persons who had been givenaccess during the evaluation and selection process (based on client representations) anddetermined persons on the list to be reasonable.

3. We reviewed the adequacy of UDOTs policies and procedures over bid protests and settlementsand the applicability of the Utah State Settlement Act, and we performed other related

procedures as considered necessary. The scope of our review did not include determiningwhether it was appropriate for UDOT to agree to a settlement.

We found that the applicability of the Utah State Settlement Act (the Settlement Act) toUDOTs settling of its bid protest was unclear. UDOT settled the bid protest under theUtah Procurement Code which gives the chief procurement officer of the agency theauthority to settle and resolve a bid or award protest prior to the filing of a lawsuit. Basedon the authority granted in the Procurement Code, UDOT did not consider the SettlementAct applicable to the bid protest settlement. However, we believe that the Settlement Actmay have been applicable. No formal opinion by the Attorney Generals Office has beenrequested to make a determination since the issue is moot based on 2011 HB 34 substitutewhich was passed, resolving the issue.

We determined that UDOT did not obtain complete and adequate documentation to supportthe settlement amount as reported in Finding No. 9. We also determined that UDOT doesnot have formal policies and procedures for bid protests and any resulting settlements asreported in Finding No. 10.

8/7/2019 UDOT Audit

6/34

v

4. We reviewed UDOTs policies and procedures regarding confidentiality agreements and conflictof interest statements and tested the agreements/statements to determine if they were completedand any identified potential conflicts were adequately addressed by UDOT management. Wealso investigated specific conflict of interest/confidentiality situations as considered necessary.

UDOTs policies and procedures over conflict of interest and confidentiality issues wereinadequate. We sampled conflict of interest and confidentiality agreements for selected

employees associated with the CORE project and found all agreements sampled weresigned. However, we noted other problems associated with conflict of interest/confidentiality issues, such as incomplete and inconsistent policies and procedures,inadequate documentation for contracting decisions, and inadequate handling of anidentified conflict, as discussed in Finding Nos. 11, 12, and 13.

5. We reviewed the stipends paid to the unsuccessful CORE proposers to determine their purpose,propriety, and compliance with State law and UDOT policies and procedures.

Stipends are used to compensate unsuccessful proposers and if accepted by them, entitleUDOT to use the proposers ideas. We found that the CORE stipends were reasonable,properly paid and budgeted, and were in compliance with state law and UDOT policies andprocedures.

Our procedures were more limited than would be necessary to express an audit opinion oncompliance or on the effectiveness of UDOTs internal control or any part thereof. Accordingly, wedo not express such opinions. Alternatively, we have identified the procedures we performed and thefindings resulting from those procedures. Had we performed additional procedures or had we madean audit of the effectiveness of UDOTs internal control, other matters might have come to ourattention that would have been reported to you.

Our findings resulting from the above procedures are included in the attached findings and

recommendations section of this report.

This report is intended solely for the information and use of the Governor and UDOT and is notintended to be and should not be used by anyone other than this specified party. However, the reportis a matter of public record and its distribution is not limited.

By its nature, this report focuses on exceptions, weaknesses, and problems. This focus should not beunderstood to mean there are not also various strengths and accomplishments. We appreciate thecourtesy and assistance extended to us by the personnel of UDOT during the course of theengagement, and we look forward to a continuing professional relationship. If you have anyquestions, please call Debbie Empey, Audit Director, at (801) 538-1342.

Sincerely,

Auston G. Johnson, CPAUtah State Auditor

cc: Carlos Braceras, UDOT Deputy DirectorDerek Miller, Chief of Staff, Governors Office

8/7/2019 UDOT Audit

7/34

UTAH DEPARTMENT OF TRANSPORTATION

I-15 UTAH COUNTY CORRIDOR EXPANSION

TABLE OF CONTENTS

PageMANAGEMENT LETTER:

I-15 CORE PROJECT EVALUATION AND SELECTION PROCESS i

FLOWCHART SHOWING OVERVIEW OF SELECTION PROCESS ii

PROCEDURES PERFORMED AND SUMMARY OF FINDINGS iii

FINDINGS AND RECOMMENDATIONS:

EVALUATION AND SELECTION PROCESS:

1. EVALUATION PROCESS NOT BLINDED 12. ISSUES WITH SCORING BY THE SELECTION RECOMMENDATION COMMITTEE (SRC):

a. Lack of Proper Documentation for the SRCs Conclusions and Recommendations 2

b. Risk of Inconsistent/Misleading Information Given to Proposers 3

c. Concerns Over Extended Warranty 5

d. Sensitivity Analysis Not Part of Policies and Procedures 7

3. POSSIBLE CONTRIBUTORS TO CLOSE SCORES 84. EVALUATION OVERSIGHT TEAM ROLES AND RESPONSIBILITIES

NOT ADEQUATELY DEFINED 12

5. GOVERNMENT OFFICIALS ROLES AND RESPONSIBILITIES NOTADEQUATELY DEFINED 136. UNNECESSARY PRELIMINARY MEETING 157. WEAKNESSES IN THE NEW BEST VALUE DESIGN-BUILD PROCESS 158. INADEQUATE SECURITY AND PROTECTION OVER BID INFORMATION 17SETTLEMENT:

9. INCOMPLETE AND INADEQUATE DOCUMENTATION TO SUPPORT SETTLEMENT AMOUNT 1810.LACK OF POLICIES AND PROCEDURES OVER BID PROTESTS 21CONFLICTS OF INTEREST:

11.INADEQUATE CONFLICT OF INTEREST/CONFIDENTIALITY POLICIES AND PROCEDURES 2212.LACK OF CONTRACT DECISION DOCUMENTATION AND POTENTIAL

CONFLICT OF INTEREST 24

13.INAPPROPRIATE HANDLING OF CONFLICT OF INTEREST 25AUDIT RESPONSE LETTER Attachment 1

8/7/2019 UDOT Audit

8/34



FINDINGS AND RECOMMENDATIONS

1

EVALUATION AND SELECTION PROCESS:

1. EVALUATION PROCESS NOT BLINDEDThe Utah Department of Transportation (UDOT) did not blind (conceal the identity of)the contractors submitting the proposals and, as a result, there is the possibility that theproposals might not have been evaluated objectively. It was necessary for the TechnicalEvaluators to know the identity of the proposers since exchange of information wasnecessary between those parties during the technical portions of the process. However,once the Technical Evaluators formulated their recommendations, blinding should haveoccurred throughout the rest of the evaluation process. It was also necessary for both theProject Director and the Deputy Project Director to know the identity of the contractors

because they were heavily involved in the entire procurement process. Therefore, theyshould not have been members of the Selection Recommendation Committee (SRC) so thatthe proposals could have been blinded from the SRC.

Blinding helps to ensure that proposals are reviewed objectively and that the possibility ofbias, whether real or perceived, is avoided. UDOT uses blinding on other projects asevidenced by other UDOT projects we reviewed. Using blinding on the CORE projectwould have prevented the appearance of bias in the awarding of the contract and might haveprevented the protest and eventual settlement.

Recommendation:

We recommend that UDOT include blinding in evaluation and selection procedures,

wherever possible.

UDOTS Response to Finding 1:

Regarding the audit finding that Blinding . . . might have prevented the protest andeventual settlement, UDOT does not believe that this is supported nor that blinding islikely to have prevented the protest or to have impacted the settlement, which wasreached for pragmatic reasons regarding moving the project forward and preventinggreater costs to the public. UDOT agrees that blinding could have prevented theappearance of bias in awarding the contract; however, UDOT disagrees with theemphasis the audit has placed on that single element of the process.

Blinding of proposals is typically beneficial, and historically UDOT has used it toisolate cost considerations from technical evaluations on more typical best-valueselections. On the I-15 CORE project, the cost was fixed and only the technicalelements evaluated, so blinding at the SRC level was not comparable to other projects.

8/7/2019 UDOT Audit

9/34




2

UDOTS Response to Recommendation 1: UDOT agrees with the recommendation as a

general principle but does not believe blinding should necessarily override all otherconsiderations. However, since the I-15 CORE procurement process, UDOT developed andimplemented a Best Value Design-Build Selection Manual of Instruction that requiresblinding on projects. In the event there is a situation in which blinding would not beadvisable, UDOT senior leadership will carefully consider the options before making adecision.

2. ISSUES WITH SCORING BY THE SELECTION RECOMMENDATIONCOMMITTEE (SRC)

Refer to the flowchart on page ii for a summary of the scoring process.

a. Lack of Proper Documentation for the SRC's Conclusions and Recommendations

Per the Evaluation and Selection Manual, the SRC was responsible for making the finaldetermination of the adjectival ratings and assigning corresponding numeric scores toeach of the combined groups of criteria; however, they did not provide adequatedocumentation to sufficiently support the adjectival ratings and scoring. While therewere some factors like highway miles paved, number of structures replaced, etc. whichcould be measured quantitatively, other factors like phasing or design strategies weremore inherently subjective and could not be measured quantitatively. Because of the

inherent subjectivity, some conclusions might have been based on personal preference,which is a part of any evaluation process. For instance, one evaluator might valueproject completion in entirety over project efficiency or one design or traffic flowpattern over another. Regardless of the nature of the subjectivity, we felt that theconclusions or recommendations of the SRC werent always documented completelyand thoroughly in the Final Recommendation Report. Although they did specifyscoring considerations, they did not document the details as to how they arrived at thescore, but rather included only a broad overall summary for each proposer. Therefore,it was not obvious what criteria or strength(s) they valued over another. Another factorcontributing to this weakness was that only general guidelines were provided in theEvaluation and Selection Manual regarding how scores were to be awarded (as reported

in Finding No. 3.c.), which allowed for additional subjectivity in the scoring process.As a result of subjectivity and inadequate documentation, we were unable to evaluatethe specific scores given, but we were able to determine that the scores given werereasonable and applied within numeric ranges assigned to the adjectival ratings.

In addition, there were allegations made that scores were changed in order to award thecontract to Provo River Constructors. For 3 of the 27 combined groups of criteria, theSRC changed the adjectival ratings to be one level higher than that of the Technical

8/7/2019 UDOT Audit

10/34




3

Evaluators. The SRC documented that they disagreed with some of the adjectival

ratings of the Technical Evaluation Teams but did not document the reasons and/or thesignificant strengths that supported their change in ratings. However, since the SRCwas responsible for determining the final adjectival rating and the Technical Evaluatorswere only responsible for recommending adjectival ratings to the SRC, the changes tothe adjectival ratings were made within the purview of the SRC and were found to bereasonable.

Because subjectivity is inherent in any evaluation and selection process of this nature, itis critical that the SRC adequately document their conclusions and recommendations.

Recommendation:

We recommend that UDOT provide adequate documentation to support

adjectival ratings and scoring.

UDOTS Response to Finding 2a:

Regarding the audit finding that the SRC did not provide adequatedocumentation to sufficiently support the adjectival ratings and scoring, UDOTresponds that the SRC relied on the work of the ETs (Technical EvaluationTeams), which provided detailed information and analysis of the proposals. TheETs reports, developed in communication with the SRC, comprise the

foundational documentation for the SRC recommendation. There are advantages and disadvantages to providing the type of detail

recommended in the audit regarding scoring and documentation. UDOT notes thegeneral guidelines and ratings process employed on the I-15 CORE project areconsistent with national best practices, that they served their function inidentifying the best-value proposal, and that the audit finding concluded that thescoring was both reasonable and applied as specified.

UDOTs Response to Recommendation 2a: UDOT believes the documentation wasadequate, it was consistent with existing best practices, and it met its intended purpose.However, UDOT will add language to the Best Value Design-Build Selection Manual

of Instruction to emphasize the importance of adequate documentation with respect toadjectival ratings and numeric scoring.

b. Risk of Inconsistent/Misleading Information Given to Proposers

UDOT might have given instruction in one-on-one meetings that was inconsistent withscoring by the SRC. Provo River Constructors (Provo River) were given scoring valuefor lane-extension miles beyond (south of) Spanish Fork; however, in the one-on-one

8/7/2019 UDOT Audit

11/34




4

meeting minutes kept by UDOT, Flatiron/Skanska/Zachry Joint Venture (Flatiron Joint

Venture) were instructed to stay within the limits identified in the Instructions toProposers, which were lengthening the Project to extend as far as Spanish Fork MainStreet. This instruction seems to imply that the proposers should not include designsextending beyond Spanish Fork, yet the scoring evidences that the SRC positivelyvalued the extension proposed by Provo River. Therefore, Flatiron Joint Venturesstatement in their protest that that they were misled when advised by UDOT that laneextension south of Spanish Fork had no value seems plausible given theinconsistency in the UDOT meeting minutes and the final scoring.

This inconsistency or misunderstanding between UDOT and the proposers could haveoccurred because both UDOT and the proposers kept their own sets of minutes for one-

on-one meetings and no attempt was made to ensure that both parties agreed with whatwas documented in both minutes. Only one set of minutes should have been kept, andthe minutes should have been approved at the next meeting as is customary. Inaddition, the approved minutes should have been reviewed to ensure that instructionsgiven to each proposer were consistent and properly followed in the scoring process.One approved set of minutes would help to prevent disagreements andmisunderstandings.

Recommendation:

We recommend that UDOT provide instruction and/or answers given in one-on-

one meetings that are consistent with the Instructions to Proposers and theEvaluation and Selection Manual and do not provide misleading or inconsistent

information to proposers. We also recommend that UDOT keep only one set of

minutes for one-on-one meetings and ensure they are approved by both parties,

are consistent with instruction given to other proposers, and are properly followed

in the scoring process.

UDOTS Response to Finding 2.b:

The audit states that Provo River Constructors (PRC) was given scoring value forlane extension miles beyond (south of) Spanish Fork but omits why. PRC was given

credit for the lane miles as a transition into the CORE reconstruction project, not forthe reconstruction itself. The extension was assessed as a minor strength as atransition that improved traffic flow into the project.

The I-15 CORE team made a concerted effort to provide consistent information to allproposers regarding the RFP. UDOT does not believe Flatiron/Skanska/Zachry JointVenture (FSZ) was misled in discussions of the southern terminus of the project orthat FSZ provided sufficient evidence to support such an assertion. Certainly there was

8/7/2019 UDOT Audit

12/34




5

no intent to misdirect any of the proposers. Meeting minutes taken by UDOT reflect a

different interpretation of the discussions.

UDOT accepts that FSZ may have misinterpreted some portion of the RFP or haddifferent perceptions of the discussions that took place in the one-on-one meetings. Atthe most, different perceptions result from an ambiguity in the RFP documents, whichis a common issue with performance-based design-build requirements. UDOT hasimplemented numerous processes to prevent this issue from occurring. UDOT acceptsthat more specificity should have been provided regarding the limits of the fullreconstruction project versus transitions into those project limits.

UDOT agrees that ideally only one set of meeting minutes for each proposer should be

kept and agreed upon, which was initially implemented in the one-on-one discussionsto ensure consistency; however, they were abused by at least one proposer to try toelevate them to contractual documents and to re-characterize the communications. Thisled to extended disputes during the meetings over exact wording of the minutes ratherthan substantive discussions regarding what information was exchanged. Thissignificantly impaired the usefulness of the discussion process. The next-best optionwas deemed to be separate meeting minutes, which was then adopted in order toprevent such abuse.

UDOTs Response to Recommendation 2b: UDOT agrees with the recommendationand asserts that it did provide instruction consistent with the ITP. As to the meeting

minutes, UDOT agrees in theory but notes that other considerations may affect thepractice. UDOT will include standardized protocol for the one-on-one meetings in theBest Value Design-Build Selection Manual of Instruction.

c. Concerns Over Extended Warranty

The SRC might have been improperly influenced by a $49.3 million extended warrantyincluded in the Provo River proposal. The warranty was to provide extendedmaintenance on the new structures, road surfaces, etc. built during the CORE project.For purposes other than award selection, price allocations (or bids) were submitted aspart of the proposal. However, these amounts were not to be known or considered in

the evaluation and selection process since this was a best value design-build projectwith a fixed price. But Provo River noted the amount of the warranty in their proposal;therefore, the SRC knew the amount and, during deliberations to determine ratings andscoring of the proposals, the SRC discussed the possibility of UDOT not exercising thewarranty, which would ultimately save UDOT $49.3 million. This, in essence, wouldmake the project no longer fixed price, but rather low bid.

We question the propriety of discussing the possible cost savings associated with notexercising the warranty because the project was supposed to be evaluated based on the

8/7/2019 UDOT Audit

13/34




6

goals and values listed in the Instructions to Proposers. Members of the SRC

represented that the warranty was evaluated and valued based only on its merits to theprojects goals and values and that its potential cost savings did not influence the awarddecision. Also, no decision was made as to whether to exercise the warranty or notduring the evaluation and selection process. However, it appears that UDOT decided tonot exercise this warranty early in the contract process because a change-order toremove the warranty from the contract was made almost immediately after UDOTissued a notice to Provo River to proceed with the contract, thus saving UDOT $49.3million.

Therefore, associated cost savings might have improperly influenced the SRC in theevaluation and selection process, and consequently, might have had some influence on

the decision to settle the bid protest with Flatiron Joint Venture since UDOT had theseunallocated resources.

Recommendation:

We recommend that the SRC limit its conversations to the projects goals and

values as identified in the Instructions to Proposers while conducting deliberations

of adjectival ratings and scoring.

UDOTS Response to Finding 2.c:

Regarding the extended warranty, UDOT notes the selection documentation showsthat the SRC compared the extended warranty against the Project Goals andvalues and assessed it as a good value in a low-significance category (whichUDOT upholds was the appropriate consideration).

UDOT notes the extended warranty was unanticipated and that for a proposal toidentify a price associated with a value-added item was unusual. UDOT also notesthat despite the SRCs access to this unanticipated information, it did not affect theSRCs recommendation. The audit states the cost saving might have influenced theSRC. UDOT understands this, but states that it did not influence the decision-making process.

The audit is correct that the I-15 CORE project team moved soon after contractexecution to develop a change order to remove the extended warranty and recoverthe associated costs. Prior experience indicates that it is difficult to capture thefull value of extended warranties. The timing of the change order was driven bybonding costs, which are a direct pass-through to UDOT. Bonds are requiredprior to starting work and are determined as a percentage of the contract amount.By reducing the contract amount immediately, UDOT avoided having to reimbursePRC almost $400,000 in unnecessary bonding costs.

8/7/2019 UDOT Audit

14/34




7

UDOTs Response to Recommendation 2c: UDOT agrees in principle with the

recommendation and believes the CORE evaluation process was consistent with thatprinciple. It will be expressly stated in the Best Value Design-Build Selection Manualof Instruction; however, UDOT notes the conversation to which the audit refers wasregarding an unanticipated value-added element initiated by the proposer, and anyprocess established needs to allow for exploring the value and implications of allproposal elements.

d. Sensitivity Analysis Not Part of Policies and Procedures

The SRC performed a sensitivity analysis which was not included in the Evaluation andSelection Manual. After adjectival ratings were determined by the SRC, the SRC gave

numeric scores based on those ratings. Those scores were then weighted based on pre-determined significance and the total weighted scores were used to determine who wasawarded the contract. The result was a one point difference between the top twoproposers. The sensitivity analysis was a process used to help the SRC determine theeffect and magnitude of scoring adjustments on the outcome of the award bydetermining if it would take major or minor adjustments in the scores to get a differentresult. Based on representations from members of the SRC, the scoring analysis wasnot done to adjust any scores but instead to help them feel comfortable with the onepoint difference in the scoring results. We found no documentation of the sensitivityanalysis and no evidence that scores were adjusted based on that analysis. However, ifthis sensitivity analysis is performed, it should be included in the policies and

procedures to document the what, why, and how of the analysis to help ensure proprietyand compliance.

Recommendation:

We recommend that UDOT include the sensitivity analysis in its policies and

procedures if its use is continued.

UDOTs Response to Finding 2.d:

The sensitivity analysis exercise was not intended to, nor did it, result in modifications

to the scoring.

UDOTs Response to Recommendation 2d: UDOT will not use sensitivity analysisduring the evaluation of Design-Build proposals. Therefore, sensitivity analysis will notbe included in UDOTs current policies, procedures, or manuals.

8/7/2019 UDOT Audit

15/34




8

3. POSSIBLE CONTRIBUTORS TO CLOSE SCORESThree possible factors that might have contributed to the proximity of the scoring resultsare: a) The scope was too restricted; b) The evaluation criteria were too condensed; and c)The adjectival ratings were not clearly defined to differentiate between the various ratings.

a. Scope Too RestrictedThe scope (length of the project) appeared to be too confined in that the UltimateInfrastructure Configuration (the expected section of freeway to be completed) beganat the American Fork Main Street interchange and ended at the Provo Center Streetinterchange. If the proposer was able to go beyond Provo Center Street, the

Instructions to Proposers appeared to provide a southern limit by using the words asfar as Spanish Fork Main Street. As indicated in Finding No. 2.b. above, UDOTinstructed the parties to stay within the limits of the Instructions to Proposers. Thus,it appears there was a southern boundary. UDOT indicated to us that they weresurprised that all three proposers were able to meet the Ultimate InfrastructureConfiguration and that two were able to go beyond that to Spanish Fork. UDOTexplained that the two proposers were able to go beyond their expectations because ofthe unexpected decline in the economy that occurred after the Instructions to Proposerswere finalized. Various individuals suggested, in hindsight, that the scope should havebeen larger. The fact that two proposers were able to go beyond Provo Center Street toSpanish Fork is an indication that the scope was too restricted. The idea of a best value

design-build project is to get the most value for a fixed price. In theory, thecompetition between the proposers should motivate them to try to give more in order towin the award. It is possible that the defined scope decreased the ability to get the bestvalue. Other indicators that the scope was too restricted are that one proposer was ableto propose HOV lanes, impressive interchanges, and an off-highway roadimprovement, while another proposer was able to offer an extended maintenancewarranty of $49.3 million. One person interviewed suspected that the proposer addedthe warranty because after calculating their price for meeting the UltimateInfrastructure Configuration they still had money left. If the scope had been expanded,the evaluation process might have ended in clearer results and been less subject tocriticism because differentiations between proposals might have been clearer.

Additionally, an expanded scope might have allowed the proposers to offer moreproject length, allowing UDOT the opportunity to evaluate the value of a lengthenedproject over other values such as HOV lanes, impressive interchanges, etc.

8/7/2019 UDOT Audit

16/34




9

b. Evaluation Criteria Too CondensedOrganizing the evaluation criteria into only three main categories (Project Definition,Maintenance of Traffic, and Schedule), and then under each of those categoriescombining together criteria based on relative significance or importance (High,Medium, Low), may not have been broad enough to accurately differentiate thestrengths of the proposers. We reviewed the evaluation and selection process of twoother projects and found that the criteria for those projects were organized into twelvedifferent categories, such as technical solutions; lighting, traffic signals, signing, etc.;pavement and geotechnical; roadway geometrics; etc. These expanded categoriesmight allow for a more detailed evaluation process where ratings and points can begiven for more specific criteria than that possible when all criteria is condensed and

combined. The expanded criteria categories would more easily identify differencesbetween proposers and allow a more obvious determination of the values the SRCplaced on those criteria, possibly resulting in more definite scoring differentiations.

c. Adjectival Ratings Not Clearly DefinedThe adjectival ratings assigned to the specific criteria groups were not clearly definedto easily differentiate between different ratings. UDOT defined the ratings in terms ofstrengths and weaknesses. For example, an exceptional rating was primarily definedas having significant strengths that significantly outweighed weaknesses, and a verygood rating was defined as having strengths that outweighed weaknesses. Significant

strengths were defined as having a considerable positive influence on the Proposersability to advance the Project goals and values, and exceed requirements. Strengthswere defined as benefits to the Project which are expected to increase the Proposersability to advance the Project goals and values, and exceed requirements. As can beseen, the difference between the two ratings is not clear and is highly subjective. Tofurther complicate the issue, both definitions of these ratings included Mostrequirements are met or exceeded. The differences between the two categoriesseem to be minor differentiations in weaknesses. These definitions did not seem toallow the SRC to adequately differentiate between proposals as is evident in that allthree proposals were given exceptional ratings in the Project Definition Highgroup. Documentation indicated that all three proposals met or exceeded project goals.

This issue is further complicated by the limitations in scope as addressed above. Itseems that an easy way to differentiate between the two ratings would have been todefine exceptional as exceeding requirements versus very good as meetingrequirements. If this differentiation was made, all three teams would not have receivedexceptional ratings because only two teams exceeded the requirements.

UDOT should consider whether changes to these areas would be beneficial for futureprojects to broaden scoring results and help alleviate protests caused by close scoring.

8/7/2019 UDOT Audit

17/34




10

Recommendation:

We recommend that UDOT consider the effects of limited scope, condensed criteria,

and unclear adjectival rating definitions on the evaluation and selection process and

take necessary measures to help allow for more definite scoring results.

UDOTs Response to Finding 3:

a. Scope too restricted.

UDOT does not believe the scope was too restricted. Those assertions follow astatement by the audit that links best-value on a design-build project to only one

evaluation element: scope. However, UDOT procured and contracted for threeelements (evaluation criteria): amount of scope, speed of delivery, and minimalimpacts to the public during construction. Scope constituted 60 percent of theequation, with 40 percent allotted to the other two criteria.

o Each of the three categories requires considerable resources to deliver. Thecriteria were established so the successful proposer would apply itsresources optimally to provide the greatest balance of the most scope,shortest schedule, and least impacts to the public.

o The three criteria and relative weights were consistent with the ProjectGoals and were established based on prior UDOT experience, public input,and the specific characteristics of the proposed road.

o Scope was not evaluated based solely on how many lane miles wereproposed, but upon the level of service to be provided at the year 2030, orthe usefulness of the scope proposed. Minimal service level required wasD, with higher service levels recognized as adding value. It is important torecognize that the most scope offered is not necessarily equal to the mostvalued scope.

UDOT does not believe that the scope was too restricted. Although two of theproposers were able to deliver almost all of what was desired in scope, UDOTbelieves that it ultimately estimated the scope just right. What was valued was usefulscope that would improve the movement of traffic throughout the corridor, whichincluded considerations such as increased regional mobility, the functionality and

types of interchanges on and adjacent to the corridor, and the future capacity ofcross-streets and arterials.

To the south, the RFP stated that full functionality was valued as far as SpanishFork Main Street but did not specify the ending point for the transition. UDOTstated it valued a smooth transition to a logical terminus, which was to be defined bythe proposer (this is the source of FSZs mistaken assumption regarding thesouthern limit).

8/7/2019 UDOT Audit

18/34




11

b. Evaluation criteria too condensed.

UDOTs experience indicates that while close proposals make the evaluationprocess more difficult, (and more open to challenges), when bids are close theproject is typically well-defined in the ITP or plan set, the risk allocation is clear,and other expectations are well communicated. A well-defined project combinedwith stiff competition from capable proposers yields bids that are likely to be closeand at the same time provide the greatest value.

Focusing on critical criteria amplifies the substantive differences rather thancompressing them. The evaluation and scoring for the I-15 CORE project focused onthe project elements that had the greatest value and the highest likelihood ofshowing meaningful and significant differences.

The narrow evaluation categories were derived from past lessons learned, andUDOT intends to implement it similarly on future design-build projects (it has beenadopted by the industry and is being considered as a best practice nationwide).

The audit suggestion of increasing the categories, especially by minorconsiderations, would generally compress the numbers and could skew them awayfrom material considerations for the mundane ones. Extraneous and lesser criteriawere intentionally left out in order to focus on the true differentiators between theproposals. This change was a direct response to prior UDOT best-value selectionsin which values assigned to items such as lighting, signals, and signing flattened andcompressed scores between proposers and provided limited opportunity forproposing teams to differentiate themselves from the competition. Such requirements

typically cause the proposal teams to simply restate the project requirements andthen commit to meet those requirements.

c. Adjectival ratings not clearly defined.

UDOT concurs that some additional language in the Evaluation and SelectionManual would be helpful in describing the assignment of adjectival ratings. UDOTnotes, however, the existing language was carefully considered and is consistentwith national best practices, which address issues such as balancing the advantagesand disadvantages of specificity, flexibility, and defensibility. Enough flexibilityneeds to be built into the process to properly evaluate the innovative solutions of

proposers. Language that is too restrictive may limit the evaluation process andprevent award to the team with the best solutions and value to the public.

UDOTs Response to Recommendation 3: UDOTs Best-Value Design-Build SelectionManual of Instruction addressed the process to develop appropriate scope, criteria, andevaluation definitions to design a selection process to best achieve the Project Goals.

8/7/2019 UDOT Audit

19/34

8/7/2019 UDOT Audit

20/34




13


Regarding the audit statement it could be misleading for UDOT to indicate that theevaluation and selection process was observed by three individuals, when really onlyparts were observed by three individuals, no effort or intent was made to misleadothers to believe the oversight was all-inclusive. UDOT never stated that oversightofficials attended every hour of every meeting that took place in the selection process.

Instead of constant monitoring, UDOT believed a sampling method was sufficient toensure the process was being administered in a fair manner. Throughout the evaluationprocess, the independent observers attended meetings without prior notice and forundeclared durations.

Nationally, it is more typical to have only a single observer and process expert.Historically, UDOT has found this to be sufficient and cost-effective. However, due tothe size and nature of the project and in an effort to raise the standard, UDOTincreased that number on the I-15 CORE project to three observers and three processexperts.

UDOT concurs that additional guidance and instruction could be provided for futureoversight team members.

UDOTs Response to Recommendation 4: UDOT agrees with this recommendation. TheBest Value Design-Build Selection Manual of Instruction provides for a process witness atboth the evaluation and selection. The roles and responsibilities of the process witness aremore fully defined.

5. GOVERNMENT OFFICIALS ROLES AND RESPONSIBILITIES NOTADEQUATELY DEFINED

The Evaluation and Selection Manual was very general and vague in describing theresponsibilities of the Government Officials, which consisted of a legislative fiscal analyst,a legislative auditor, and two transportation commissioners. The Manual states simply TheGovernmental Officials will observe and advise the Selection Official during the final stageof the procurement process. (italics added). However, all four of the Government Officialsindicated that their responsibilities were to evaluate the process, i.e. determine if UDOT

followed the process, determine if there were any problems with the process, evaluate thefairness of the process, observe any biases, evaluate the way the proposals had beenreviewed, and determine if UDOTs review of the proposals would justify the final decision.In other words, they were essentially asked to validate the process. Also, the SelectionOfficial indicated that the Government Officials were to give a broad, high-level review ofthe process to determine that the process made sense. The Manual also states that once anacceptable recommendation has been delivered, the SRC will give a presentation to theSelection Official and Government Officials. Asking the Government Officials their

8/7/2019 UDOT Audit

21/34




14

opinion of the process and if they agreed with the SRCs conclusion after the SRC had

already determined their recommendation of the winning proposer seems to be inherentlyflawed since there could be a high risk of bias in the presentation because the presenterscould skew their presentation in favor of their selection.

Since the Manual is too vague for anyone to know what the role of the GovernmentOfficials is intended to be and nothing was issued in writing from the Government Officials,there is a high risk that the Government Officials role could be easily misunderstood andgive a false sense of comfort by giving the appearance that the Government Officials wereaccomplishing more or something completely different than what they did; therefore, forfuture projects the Manual should state the responsibilities of the Government Officialsclearly and precisely.

Recommendation:

We recommend that UDOT include adequate instruction regarding the responsibilities

of Government Officials in their manuals and policies.


UDOT concurs with the audit that the roles of the government officials were notadequately defined in the Evaluation Manual and notes the process intended was notwell described in the manual.

UDOT appreciated the attendance of the governmental officials and their feedback asthe process neared its conclusion. We agree that with limited exposure to the selectionand evaluation process, limited instructions, and a largely undefined role, it wouldhave been unfair to ask them to provide a complete endorsement of the final selection.Nonetheless, the process was beneficial as it allowed the governmental officials to seeand understand the complex and comprehensive procurement process and the intenseeffort that was undertaken to ensure the public received the best value for this immensepublic infrastructure project.

UDOTs Response to Recommendation 5: UDOT agrees with this recommendation.Because of the uniqueness of the I-15 CORE project, government officials were included in

the selection process to provide an additional level of input into the process. In the future,UDOT does not intend to use government officials in the selection process for projects.Consequently, the Best Value Design-Build Selection Manual of Instruction does not referto the term Government Official. Instead, this manual defines the membership of theselection committee and the selection officer. If the Department undertakes another projectwith the characteristics of the I-15 project and decides that including government officialsin the process would add value, the Department will provide better role definition.

8/7/2019 UDOT Audit

22/34




15

6. UNNECESSARY PRELIMINARY MEETINGThe SRC presented their results in both a preliminary meeting and the meeting with theGovernment Officials; however, the preliminary meeting excluded the GovernmentOfficials. As a result, it appears that the preliminary meeting may have been conducted inorder for the Selection Official to pre-approve the information to be presented to theGovernment Officials, including their recommendation of the winning proposer. Althoughthe preliminary meeting was not as detailed as the meeting that included the GovernmentOfficials, the meeting with the Government Officials does not seem to accomplish anythingexcept to give the appearance that they were an independent group overseeing the process,again causing the Government Officials role to be misunderstood. Also, the meeting toreview the preliminary findings was included in a narrative portion of the Manual, but it was

not included in the flowchart of the selection process in the Manual. UDOT shouldeliminate the unnecessary preliminary meeting and have only one meeting to present theresults to the Selection Official and the Government Officials for future projects.

Recommendation:

We recommend that UDOT eliminate the unnecessary preliminary meeting and have

only one meeting to present the results to the Selection Official and the Government

Officials for future projects.


The preliminary meeting was to ensure proper planning and preparation for the subsequent,complete presentation to the SO and public officials. If used in the future, it will bedescribed in the Evaluation Manual.

UDOTs Response to Recommendation 6: UDOT agrees with this recommendation.

7. WEAKNESSES IN THE NEW BEST VALUE DESIGN-BUILD PROCESSSubsequent to the CORE project, UDOT completed an instruction manual for Best Value

Design-Build projects to give direction on how selection processes occur on theseprojects. This new manual should be approved and incorporated into UDOTs formalpolicies and procedures to help ensure standard procedures are followed on all projects. Inreviewing UDOTs new manual, we noted the following problems:

a. The roles and responsibilities of the Selection Official are not included in the newmanual.

8/7/2019 UDOT Audit

23/34




16

b. The roles and responsibilities of the Process Witnesses (called the Evaluation OversightTeam in the CORE project) are not adequate as included in the new manual. Theprocesses, meetings, and/or deliberations that the Process Witnesses should observe arenot included. The diagram shows the process in general for which each ProcessWitness is responsible; however, the written instructions should specify their individualresponsibilities as well as their combined responsibilities. The reports or other writtendocumentation to be completed by the Process Witnesses are not specified. Theobservations of the Process Witnesses need to be adequately documented to evidencethis control in the evaluation and selection process.

c. The terms used in the new manual are not clearly defined or consistently used. Themanual uses the rating terms of technical, categorical, and adjectival; however, it is

unclear whether these represent different ratings or are interchangeable. For example,the manual says that the Selection Committee should understand the EvaluationCommittees technical rating, but then says they should approve the categorical rating.The terms should be consistently used in the manual or adequately defined to avoidconfusion or misunderstanding.

d. The new manual does not include instructions and procedures for handling andresolving differences of opinions between committees, or for documenting theresolutions. As in the example above, the Selection Committee must understand andapprove the ratings of the Evaluation Committee. This could mean the two committeesnegotiate until they agree or that one committee trumps the opinion of another.

Another example is that the Selection Committee takes their determination to theSelection Official for concurrence. The procedure to be followed whendisagreements occur should be included in the new manual to help ensure thatdisagreements are properly handled, resolved, and adequately documented, asconsidered necessary.

e. Standard definitions and requirements that should not vary between projects are notincluded in the new manual. For instance, common definitions for adjectival ratingsand applicable numeric scoring ranges within those ratings could be defined so thatthey are used consistently between projects.

Incorporating the process for best value design-build projects, as well as other types ofconstruction projects, into UDOTs Policies and Procedures Manual will help to ensure thatthe procurement of contracts is proper and consistent.

Recommendation:

We recommend that UDOT incorporate the Best Value Design-Build Manual into its

Policies and Procedures Manual and revise it to include:

8/7/2019 UDOT Audit

24/34




17

a. The roles and responsibilities of the Selection Official.b. Clearly defined roles and responsibilities of the Process Witnesses.c. Clearly defined and consistently used terms.d. The resolution processes to be followed when differences of opinions occur

between committees and the documentation of such resolutions.

e. Standard definitions and requirements between projects.We also recommend that UDOT establish set procedures for the procurement of other

types of construction projects that are not best value design-build projects and

incorporate them into their policies and procedures.


The procurement of the I-15 CORE project, with only one other similar example tolearn from in the entire country, pioneered a new standard in the industry. UDOTselected the fixed-price best-value approach to leverage ideal bidding conditions andstretch the limits of the size and quality the project could reach. UDOT believes theachieved value of twice as much project completed two years sooner with fewer trafficdelays than every estimate had shown, validates the process. UDOT, as is its practice,learns from every project procurement and has already begun to implement changesbased both on the process and on input received from the audit.

Consistent with UDOT practices, information derived from the I-15 CORE projectprocurement was added to existing lessons learned and will be incorporated into thecurrent draft procedures manual for application in future procurements. The auditcomments will also be addressed.

UDOTs Response to Recommendations for Finding 7: UDOT concurs with therecommendations and will incorporate the recommendations into the Best Value Design-Build Selection Manual of Instruction.

8. INADEQUATE SECURITY AND PROTECTION OVER BID INFORMATIONUDOT did not adequately secure and protect the bid information on their ProjectDevelopment Business System (PDBS) for the CORE project. The proposers were requiredto submit bid amounts (in the form of price allocations) which were entered into PDBS forpurposes other than award selection. Although bid amounts were received, they were not tobe considered in the evaluation and selection process and/or the eventual award of the

8/7/2019 UDOT Audit

25/34




18

contract for the CORE project because it was not supposed to be awarded based on low bid

(See Finding No. 2.c. which reported a concern affecting low bid.)

Based on representations from UDOT employees, CORE project bid amounts weresupposed to be blocked or blinded so as not to be accessible to anyone through PDBS;however, there was a programming error and the bid amounts were not blocked andproperly secured. An employee indicated that the CORE project was the first project to usethis blocked feature, and UDOT was evidently unaware that the programming did not workas intended. Consequently, employees with access to PDBS were also able to access andview the bid results. Based on information from informants, we determined that the CivilRights Manager accessed the bid amounts and improperly disclosed the bid rankings to anemployee at Wadsworth Brothers Construction, a contractor who is part of Provo River

Constructors, the proposer who won the award for the CORE project. Although the bidinformation was misinterpreted and improper conclusions were reached, this breach ofconfidentiality compromised the integrity of the evaluation and selection process.

Recommendation:

We recommend that UDOT secure and protect bid information by ensuring that bid

information is blocked when necessary. We also recommend that UDOT take

appropriate disciplinary action against the Civil Rights Manager for improperly

disclosing confidential information.

UDOTs Response to Recommendations for Finding 8: UDOT concurs with therecommendations. UDOT has already instituted changes to the security of the bid systemand took appropriate disciplinary action with the Civil Rights Manager.

SETTLEMENT:

9. INCOMPLETE AND INADEQUATE DOCUMENTATION TO SUPPORTSETTLEMENT AMOUNT

UDOT did not obtain supporting documentation for the entire $13 million of costs incurredby Flatiron/Skanska/Zachry Joint Venture (Flatiron Joint Venture), which was the basis fordetermining the bid protest settlement amount. UDOT received a bid protest from FlatironJoint Venture as a result of awarding the contract for the CORE project to Provo RiverConstructors. While neither Flatiron Joint Venture nor UDOT admitted liability orwrongdoing, they agreed to settle for $13 million to avoid the costs of litigation. Thesettlement amount was based on the representation that Flatiron Joint Venture incurred costsin excess of $13 million in Flatiron Joint Ventures pursuit of the CORE project proposal.Although UDOT obtained a written certification of costs exceeding $13 million from

8/7/2019 UDOT Audit

26/34




19

Flatiron Joint Venture, and UDOT represented to us that they had documentation from

Flatiron Joint Venture in their possession supporting those costs, we found that thedocumentation received from Flatiron Joint Venture was incomplete since it supported only$8.6 million of the $13 million of represented costs. The documentation did not include thecosts of Skanska and Zachry nor did it include source documentation, such as invoices,employee time records, etc. Instead, the documentation consisted merely of FlatironConstructors portion of internal bid costs and spreadsheets representing third party bidcosts incurred by the FSZ Joint Venture to pay the design firms and other outsideprofessionals and vendors. UDOT should have verified that the documentation receivedfrom Flatiron Joint Venture completely and adequately supported the representation of $13million in costs prior to the payout of the settlement. Failure to adequately reviewsupporting documentation could result in incomplete and inadequate supporting

documentation and improper settlements.

Recommendation:

We recommend that UDOT obtain and review adequate supporting documentation

prior to paying settlements.


UDOT, while believing it would ultimately win any legal challenge, was concerned aboutthe additional costs that would accrue due to any delay to the project. UDOT explored three

options:

1. Fight the protest.2. Re-procure the project.3. Negotiate a settlement.

Because finite resources were a primary concern and the need for the project wasimmediate, UDOT used expected costs and delays to the project as the two determiningfactors for selecting the next course of action. UDOT also considered the past history oflegal challenges to large projects, particularly the Legacy Parkway. After an injunction tohalt that project was issued by the courts, eventual delay costs were in the hundreds of

millions of dollars a prospect that UDOT felt was unacceptable for the I-15 COREproject.

The first and second options fighting the protest or re-procuring the project wouldincur substantial additional costs. Reviews of protest procedures showed delay periods wereunpredictable; with the first stage alone taking up to three months and the entire processtaking 18 months or longer to be fully resolved. A protest delay would have accumulated thefollowing estimated costs:

8/7/2019 UDOT Audit

27/34




20

1. Escalation costs: The signed contract with PRC permitted them to collectconstruction escalation costs that keep the bid intact in the event of a delay. Thosecosts accumulate at a rate of 3 percent per year after the first 6 months, leading to a$16.5 million cost increase for a one-year delay and a $33 million cost increase foran 18-month delay.

2. PRC suspension costs (overhead and carrying): PRC could claim furtherreimbursement of delay costs that could have reached $6 million per month.

3. Carrying costs: Project costs incurred by UDOT to keep certain elements of theproject moving forward (such as right-of-way acquisition, utility agreements, officespace rental, etc.) during a delay would have been between $.5 million and $1million per month.

4. Litigation costs: A protracted legal battle would have incurred additional costsupwards of $1 million.

Re-procuring the project would have added additional costs and delayed the project bymore than two years. A risk analysis was conducted, which indicated that costs to re-procure would include the following:

1. Paying PRC damages for proposal costs, design costs, overhead, anddemobilization.

2. Additional design costs to prepare for a new bidding process.3. Possible damages to FSZ and litigation costs.4. Re-procurement costs of $20 million to $30 million.5. Construction inflation costs.

In light of those high risks for substantial additional costs and long delays, UDOTleadership selected the last option of an immediate settlement as a sound business decisionand most cost-effective way to move the project forward.

UDOT entered into negotiations with FSZ and proposed to pay FSZs certified costs toprepare their bid proposal; a portion of which was covered by the original stipend that wasto be paid to each unsuccessful bidder.

Because of the certification of the costs, UDOT could reasonably rely upon the certification

because the certification is a guarantee that those costs are accurate, even if anindependent third party audited the records.

Certifying costs, instead of providing source documentation, is common constructionindustry practice. This certification still allows UDOT the ability to audit all sourcedocumentation, and the costs are not certified unless the costs can be independentlyaudited. If the documented costs are less than what was certified, UDOT can and would bereimbursed.

8/7/2019 UDOT Audit

28/34




21

Subsequent to the signed settlement agreement, UDOT obtained documentation from

Skanska and Zachry for the estimated costs for FSZs proposal for the I-15 CORE project.The documented bid costs for Skanska and Zachry were over $5.9 million. FSZ, includingeach of the joint venture partners, certified a cost in excess of $13 million. Thedocumentation provided later by the joint venture partners as a result of the auditdocumented bid costs in excess of $14.5 million. Therefore, UDOTs reliance upon certifiedcosts was justified and reasonable for the settlement.

UDOTs Response to Recommendation 9: UDOT agrees that adequate documentationshould be in-hand prior to paying settlements. Documentation provided as a result of theaudit shows that the actual costs to prepare the FSZ proposal was greater than the amountagreed to as part of the settlement.

10. LACK OF POLICIES AND PROCEDURES OVER BID PROTESTSUDOT does not have written policies and procedures for handling and resolving bidprotests. The written policies and procedures should include such things as procedures andresponsibilities of employees, the requirements of the Procurement Code, and applicableauthorizations as required by the State Settlement Act as amended by HB 34 substituteduring the 2011 State Legislature. While certain procedures related to bid protests aredocumented in specific projects Instructions to Proposers, they are not adequate and are notformalized written policies and procedures. Written policies and procedures are necessaryto help ensure that protests are consistently handled and resolved as allowed by Utah Code.

The lack of written policies and procedures could cause bid protests to be handledinconsistently or in violation of State law.

Recommendation:

We recommend that UDOT implement written policies and procedures over bid

protests.


Part 8 of the Utah Procurement Code provides the time deadlines for filing the protests, the

individual who decides the protest, the time period for rendering a decision, relief that isgranted when a protest is sustained and notification of the right to judicial or administrativereview.

UDOT has followed Part 8 of the Utah Procurement Code and applicable statutes inresolving all the protests. Only the Executive Director has decided the protests as requiredby the Utah Procurement Code. In this case, UDOT followed existing law.

UDOTs Response to Recommendation 10: UDOT agrees with the recommendation.

8/7/2019 UDOT Audit

29/34




22

CONFLICTS OF INTEREST:

11. INADEQUATE CONFLICT OF INTEREST/CONFIDENTIALITY POLICIES ANDPROCEDURES

UDOT does not have adequate policies and procedures related to employee conflict ofinterest and confidentiality. UDOT has not effectively defined potential conflicts of interestand has not adequately emphasized the importance of avoiding conflicts of interest andmaintaining confidentiality. We noted the following problems with conflict of interest andconfidentiality policies and procedures:

a. UDOTs current conflict of interest and confidentiality policies are not comprehensive.The current conflict of interest policy identifies possible conflicts of interest as outsideemployment and exchanges, such as gifts and meals over $50. However, it does notidentify other possible conflicts, such as family or close personal relationships withparties associated with UDOT, ownership interests, etc. In addition, the currentconfidentiality policy is too brief. We noted one project-specific conflict ofinterest/confidentiality form that did not include as much information as in the policy,but another project-specific form (used on the CORE project) that included much moredetail than the written policy. UDOT should expand the wording of the writtenconfidentiality policy so that the policy is complete.

b. UDOT does not require employees to complete a conflict of interest/confidentialitystatement on a periodic basis. UDOT appears to require these statements to becompleted on a project basis. Therefore, employees not assigned to a specific projectare overlooked and do not complete the required statements. We noted the followingsituations:

1) The former Director of Research and Bridge Operations, whose job position waschanged to Right-of-Way Engineer in 2010, worked for 19 years withoutcompleting a conflict of interest/confidentiality statement. As a result, she had notdisclosed her private engineering and consulting company or her personalrelationship with a contractor who regularly conducts business with UDOT andsubmitted a proposal for CORE. In August 2010, she was required to sign astatement in response to a legislative audit recommendation; however, she still didnot disclose her conflicts. In addition, she did not maintain confidentiality in thatshe disclosed confidential contractor information to a former UDOT employeeworking in private industry.

2) The Civil Rights Manager did not complete a conflict of interest/confidentialitystatement, and she improperly disclosed CORE bid information to a contractorwho had submitted a proposal for the CORE project, as reported in Finding No. 8.

8/7/2019 UDOT Audit

30/34




23

UDOT should evaluate which employees should complete a conflict of interest/

confidentiality statement and how often. At a minimum, a conflict ofinterest/confidentiality statement should be completed annually or when changes inemployee circumstances or job assignments occur. In addition, UDOT should considertaking disciplinary action against employees who do not properly disclose all conflicts orwho fail to maintain confidentiality.

c. The conflict of interest/confidentiality forms reviewed were inadequate. We notedthree different forms used by UDOT (and there could be more), and all three haddifferent verbiage. The forms we reviewed included a Conflict of InterestDeclaration form that was used as a general form and did not cover confidentiality, aproject-specific Confidentiality and Disclosure Agreement form that was used on the

CORE project, and a Conflict of Interest & Confidentiality Statement form that wasused for an employee associated with a bridge pool. The following components areinconsistent or lacking in the forms and should be included (however, this list is not all-inclusive):

1) Disclosure of conflicts of interest and agreement to maintain confidentiality on thesame form.

2) Disclosure of all outside employment.3) Disclosure of substantial ownership interests.4) Disclosure of family or other close personal relationships with parties associated

with UDOT.

5) Disclosure of all potential conflicts of interest. This section should require listingpotential conflicts or stating that no conflicts exist.

6) Managements assessment of the potential conflicts of interest and their resolutionof any identified conflicts.

To ensure adequate information is included in conflict of interest/confidentiality forms,UDOT should consider standardizing the forms.

The lack of adequate conflict of interest/confidentiality policies and procedures could resultin conflicts of interest or breaches of confidentiality occurring due to inadequatecommunication, confusion, and/or misunderstanding by employees, or due to a perceivedlack of emphasis on the importance of avoiding conflicts and maintaining confidentiality.

Recommendation:

We recommend that UDOT:

8/7/2019 UDOT Audit

31/34




24

a. Implement comprehensive policies and procedures regarding conflict of interest/confidentiality.

b. Require employees to complete a conflict of interest/confidentiality statement atleast annually or more often if changes in employee circumstances or job

assignments occur.

c. Standardize the conflict of interest/confidentiality statements to include adequateinformation.

UDOTs Response to Recommendations for Finding 11: UDOT concurs and will revise theRules of Conduct and Code of Ethics Policy and the Conflict of Interest Declaration toaddress the issues in the finding. UDOT is in the process of developing a training course

dealing with conduct and ethics for employees. Employees will be required to complete aconflict of interest statement on an annual basis. UDOT will address the confidentialityissue when it revises the Rules of Conduct and Code of Ethics Policy and the Conflict ofInterest Declaration. The project specific confidentiality and conflict of interest declarationhas recently been updated to address the issues raised in the finding.

12. LACK OF CONTRACT DECISION DOCUMENTATION AND POTENTIALCONFLICT OF INTEREST

Policies and procedures related to the Bridge Operation Division do not require adequate

documentation for final contract decisions related to emergency bridge repair. Because ofconcerns that a potential conflict of interest existed between the Director of Research andBridge Operations and a particular contractor with whom she was personally involved, wereviewed a sample of emergency bridge repair projects. For one project, UDOT employeesindicated that the contract decision was made by the Director to award the contract to thisparticular contractor without going through the established process. However, due to a lackof documentation of the contract decision on this contract, we were not able to determinewith certainty whether the process was followed in awarding the contract to this particularcontractor. Other similar contracts also lacked documentation of the contract decision; thus,we could not distinguish whether this contract was handled differently than other similaremergency contracts and whether a possible conflict of interest occurred.

The current emergency bridge repair policy requires two employees of the Bridge OperationDivision to evaluate the pool of bridge repair contractors, reach a consensus on whichcontractor will be contacted first, and then obtain approval from the Director. However, thepolicy does not require employees to document their contracting decisions. The policyshould require that the considerations and justification for final contract decisions bedocumented in order to avoid even the appearance of a conflict of interest.

8/7/2019 UDOT Audit

32/34




25

Recommendation:

We recommend that UDOT strengthen their emergency bridge repair policy to require

adequate documentation of considerations and justification for contract decisions.

UDOTs Response to Recommendation 12: UDOT is currently revising the emergencycontracting policy. The revision will result in a selection process that includes anabbreviated competitive process for all emergency contracts. The emergency contract willonly include work to stabilize the particular emergency. After the emergency, the existingprocurement rules will be followed.

13. INAPPROPRIATE HANDLING OF CONFLICT OF INTERESTUDOT allowed an employee with a conflict of interest to be involved in the evaluation andselection process as a member of one of the Technical Evaluation Teams. The spouse of theTechnical Evaluation Team member was employed by one of the contractors submitting aproposal on the CORE project. The team member disclosed the conflict on hisConfidentiality and Disclosure Agreement, and the project director dismissed the conflict asnot being a problem because representations were made that the spouse was notparticipating in the proposal process for the contractor. However, we consider this conflictto be a serious situation that could have affected the outcome of the evaluation and selectionprocess in favor of the company for which the UDOT employees spouse worked.

Therefore, UDOT should have taken a more conservative approach and not allowed theemployee to be a member of a Technical Evaluation Team. Although this conflict ofinterest did not cause a problem because the contractor was not awarded the contract, theintegrity of the entire evaluation and selection process could have been compromised hadthe contractor won the award.

Recommendation:

We recommend that UDOT take conservative measures to prevent either actual or

perceived conflicts of interest to help ensure the integrity of the evaluation and

selection process.


The Consultant Services Manual of Instruction will be amended to include a conflict ofinterest review process. UDOT will also review the UDOT Procurement Division processesfor any necessary revisions.

UDOTs Response to Recommendation 13:UDOT agrees with the recommendation and hasimplemented measures to prevent and/or reduce actual and perceived conflicts of interest.

8/7/2019 UDOT Audit

33/34

Attachment 1

8/7/2019 UDOT Audit

34/34

UDOT Audit

Documents