Audit Report on the Department of Transportation’s Performance Indicators as Reported in the Mayor’s Management Report

This audit determined whether the Department of Transportation (DOT) maintained adequate controls to ensure that the performance indicator statistics it reports in the Mayor’s Management Report (MMR) are accurate and reliable. This audit focused on the following three critical indicators: (1) average time to respond to traffic signal defect and make safe (hours) (traffic signal indicator); (2) average time to repair street lights (days) (street light indicator); and (3) average time to close a pothole work order where repair was done (days) (pothole indicator).

The MMR serves as a public report card on City services affecting the lives of New Yorkers and mainly covers the operations of City agencies reporting directly to the Mayor. DOT is responsible for bridge and roadway conditions, parking and traffic operations, sidewalks, and other matters that affect the safety of drivers, cyclists, and pedestrians throughout the City. As reported in the MMR, DOT’s key public service areas include: ensuring the safety of the traveling public; improving mobility throughout the City; rehabilitating and maintaining the City’s bridges, streets, sidewalks, and highways; and expanding walking and cycling options and ferry service. To report on DOT’s progress in achieving its critical objectives, the MMR for Fiscal Years 2009 and 2010 included 51 performance indicators, 23 of which were identified as critical indicators.

Audit Findings and Conclusions

Our review of the information technology (general and application) controls for the databases associated with the three tested critical performance indicators provided assurance that if all procedures and controls as explained to us by DOT officials are consistently applied and followed, the data reflected therein are sufficiently reliable and accurate. However, because of control weaknesses disclosed in DOT’s manual processes for calculating the indicator values, there is only limited assurance that the traffic signal, street light, and pothole indicators published in the MMR are accurate and reliable.

The audit also determined that DOT’s automated processes provided assurance that the data that it used to calculate the values of the subject indicators was complete. DOT’s corresponding calculation formulas were consistent with the subject indicator definitions published in the MMR. Further, the indicator values that DOT recorded in the Mayor’s Office of Operations performance data collection and reporting system, the Performance Management Application (PMA), corresponded to those that appeared in the preliminary and final MMR versions for Fiscal Years 2009 and 2010.

However, any assurance these results provided was reduced because DOT: (1) made errors in calculating the tested indicators, and (2) lacked adequate checks (i.e., independent verification) of indicator values prior to them being entered into the Mayor’s Office of Operations PMA system. These weaknesses limited our assurance about the reliability and accuracy of the tested performance indicators that appear in the PMA system and ultimately the published MMR.

Audit Recommendations

To address the above weaknesses, the audit recommends that DOT should:

• Develop procedures to verify reported performance measure statistics. Such procedures should require that the performance statistics be independently verified by either a second person within each division or another party designated by DOT prior to being recorded in the PMA.
• Consider retaining a snapshot (copy) of data that are used to calculate the reported indicator values as a supplement to the retained hard-copy reports
• Disclose information in the MMR about the underlying factors and relevant calculations from which the “Average time to respond to traffic signal defect and make safe (hours)” and other similar “Average” value indicators are based to help users of the MMR better understand the agency’s performance in these areas.

DOT Response

We received a written response from DOT officials on January 20, 2012. In their response, DOT officials agreed with the audit’s findings and recommendations.