Audit Report on the Development of AutoTime by the Human Resources Administration

Total

In 1993, the Human Resources Administration (HRA) developed requirements for an automated timekeeping system that was contained in a document entitled HRA AutoTime—A Paperless Timekeeping System. In August 1994, HRA issued a Request for Proposals (RFP), based on this document, for the development of the Auto Time System (AutoTime). AutoTime captures and processes employee timekeeping, attendance, and leave information from HRA’s 112 work sites. By developing AutoTime, HRA intended to reduce the cost of timekeeping, eliminate paperwork, reduce errors in employee time records, and enable transmittal of Electronic Time Records (ETR) and data adjustments to the City’s Payroll Management System (PMS).

From among the received proposals, HRA selected Davisco Inc. (Davisco) to create AutoTime. On June 25, 1995, HRA and Davisco entered into a five-year $9.6 million contract under which Davisco was to provide for: the delivery and installation of system components; all modifications necessary to assure that the system would comply with the functional requirements in the RFP; programming according to specifications in the RFP; training for HRA employees; establishing connections between AutoTime equipment and HRA’s data communication networks; and on-site warranty and maintenance services. The contract was amended on October 22, 1998, December 1, 1998, May 18, 1999, and October 10, 2000, bringing the total contract price to $12,897,563. In addition, on August 2, 2000, HRA awarded Davisco an $11.9 million renewal contract. The scope of work of these contracts and amendments is shown in Table I, following.

The objectives of this audit were to evaluate whether: (1) AutoTime met the initial business and system requirements, as specified in the 1994 Request for Proposal (RFP); (2) the overall AutoTime system design allowed for future enhancements and upgrades; and, (3) AutoTime, as a finished product, meets user needs. Audit fieldwork was conducted from April 2001 through December 2001.

Since the City does not have a formal Systems Development Methodology, we used the following as criteria in this audit: New York City Procurement Policy Board (PPB) Rules; New York City Comptroller’s Internal Control and Accountability Directive #18, "Guidelines for the Management, Protection and Control of Agency Information and Information Processing Systems"; and the National Institute of Standards and Technology (NIST) Special Publication #500-233, A Framework for the Development and Assurance of High Integrity Software.

This audit was conducted in accordance with Generally Accepted Government Auditing Standards (GAGAS) and included tests of the records and other auditing procedures considered necessary. This audit was performed in accordance with the City Comptroller’s audit responsibilities as set forth in Chapter 5, § 93, of the New York City Charter.

"Before addressing specific findings and recommendations, however, it is important to note that the authors ofthe report fail to disclose the fact that the City Comptroller’s Directive 18 statement, related to retaining an independent Quality Assurance (QA) consultant, was not in force at the time the initial procurement, vendor selection and system design and development phases of the AutoTime project took place. Directive 18 did not address independent QA consultants for major system development projects until 1998, several years after the AutoTime project was begun by HRA. The version of Directive 18 that was in force when HRA began the AutoTime project was released in 1981 did not address QA consultants, and was not updated for another 17 years, despite the tremendous changes that occurred with respect to information technology during that period. Therefore, it is inappropriate for the report to cite HRA for failing to comply with Directive 18 with respect to retaining a QA consultant for AutoTime, since AutoTime pre-dates this aspect of Directive 18. This factshould be disclosed throughout the report whenever Directive 18 is cited in connection with retaining a QA consultant for AutoTime procurement, design and/or development."

HRA is correct in stating that the Quality Assurance provisions of Directive #18 did not go into effect until after the initial AutoTime contract was let. However, it had long been industry practice, prior to 1998, for projects to be overseen by quality assurance groups. In fact, KPMG, the City’s external auditors, in its 1989 Management Letter recommended that the City establish a quality assurance group to oversee system development projects. Moreover, after the current Directive was issued the contract was amended or renewed five times to provide for additional software and programming, which increased the initial contract from approximately $9.6 million to more than $24.8 million. At the time of those subsequent amendments or renewals, HRA should have hired a quality assurance consultant to help expedite system development and minimize additional costs. A quality assurance consultant would have learned, in detail, the user and technical requirements of the system and would have used this knowledge to ensure that all requirements and design specifications were accurate and complete.

Davisco installed the basic timekeeping and attendance functions of AutoTime at all HRA sites in 1998 and generally met the initial business and system requirements specified in the RFP. More than 15,000 employees at all of HRA’s 112 work sites use swipe cards or their desktop computers to "clock in" and "clock out" on the system. All absence requests and approvals are processed through the system. AutoTime’s overall modular design allows Davisco to develop future enhancements and periodic upgrades. MIS personnel have created formal procedures for program change control and software testing. Finally, users are generally satisfied with AutoTime’s basic functions.

AutoTime, however, is incomplete although it has been in development since 1993, and its costs have significantly exceeded the original contract amount. HRA did not retain an independent QA consultant who could have identified the needs of the users and evaluated the proposals HRA received in response to the RFP. AutoTime could have benefited from a QA consultant at any time during the seven-year course of this contract. HRA also did not develop a design document that included system specifications. Moreover, HRA did not assign a sufficient number of business analysts to develop system design specifications that would have enabled the vendor to program, test, and implement the system expeditiously. Some AutoTime users told us that several functions do not work properly and that Davisco employees have improper access to personal information on the AutoTime database.

Moreover, the renewal contract included services that could be performed by HRA employees. Finally, as part of its August 1993 "Pre-Solicitation Review Report," HRA estimated that by implementing the AutoTime system the City would save $15.7 million. However, based on the actual costs of developing and maintaining the system, we concluded that the City would not realize any savings from the implementation of the system. Additionally, HRA does not monitor the activities of the 12 Davisco employees who have "super user" access to all AutoTime’s source code and database records. A super user has the ability to create, modify, and delete database records and programs.

To address these issues, HRA should:

• Assemble a project team headed by a full-time project manager who will ensure that all necessary AutoTime functions are identified and implemented. The team should include a representative from HRA’s audit group to serve as a quality assurance consultant.
• Immediately survey system users to determine areas of AutoTime that require changes to meet their needs.
• Monitor the activities of the "super-users" to ensure that only authorized work is performed on the system.
• Amend Davisco’s renewal contract to include training for HRA’s technical staff so that they can take over the programming, maintenance, reporting, and database administration of AutoTime when the renewal contract ends.
• Use the experience of developing AutoTime as a guide in the development of future system projects.
• Retain an independent quality assurance consultant to help identify the needs of the users and evaluate the proposals received in response to future RFPs.

The matters covered in this report were discussed with officials from HRA during and at the conclusion of this audit. A preliminary draft report was sent to HRA officials and discussed at an exit conference held on August 13, 2002. On September 4, 2002, we submitted a draft report to HRA officials with a request for comments. We received a written response from HRA on September 19, 2002, which stated that "HRA will consider the findings and recommendations of this report in evaluating the overall success of the AutoTime project and in planning, developing and implementing future information systems." HRA’s response also indicates: that it does not believe that it should be held to the requirements of Directive #18 regarding the hiring an independent quality assurance consultant, as the Quality Assurance provisions of Directive #18 did not go into effect until after the initial AutoTime contract was let; that certain of the report’s findings are incorrect or flawed; and, that the implementation of AutoTime constitutes a positive change in the way it does business.