Audit Report on the Department of Education’s NYC21C Project

AUDIT REPORT IN BRIEF

Chart I DOE Chart from page 23 of the 2009 Plan

The Department of Education (DOE) provides primary and secondary education to over one million pre-kindergarten to grade 12 students in 32 school districts in over 1,500 schools, and employs approximately 75,000 teachers. DOE prepares students to meet grade-level standards in reading, writing, and math, and prepares high school students to meet graduation requirements. The School Construction Authority coordinates the development of DOE’s Five-Year Capital Plan, selects and acquires sites for new schools, leases buildings for schools, and supervises conversion of administrative space for classroom use.

DOE launched the NYC21C initiative in May 2009. The initiative was created as a research and development project aimed at innovating school practices to better prepare students for college and careers in the 21st century. On an individual basis, it was intended to “leverage state of the art technology to personalize instruction in core academic skills.” School model development was intended to “invent, codify, and replicate the most effective school wide practices that prepare students for the 21st century.” There were 10 NYC21C schools enrolled in the project, including six high schools, one transfer high school¹, and three middle schools. Eight of these 10 schools were first started in School Year (SY) 2009-2010.

In May 2009, DOE created a Five-Year Information Technology Strategic Plan (the Plan) that contains its plans for incorporating technology in all schools, including the 10 NYC21C schools. The Plan includes the IT Strategic Planning Process and the NYC21C initiative. In June 2010, DOE updated the Plan. The Plan indicates that over the course of its five-year horizon, new technologies will emerge that will be incorporated into the Division of Instructional and Information Technology’s (DIIT) plans. The NYC21C initiative has evolved into various other pilot programs.

Audit Findings and Conclusions

DOE provided the 10 NYC21C schools with technology, human resources, curriculum, and instruction as indicated in its original Plan, dated May 2009. We found that the schools have wireless upgrade, Smartboards, laptops, access to online classes, and college credit courses. In addition, the schools have educational software, which includes Rosetta Stone, Revit Architecture, Powerspeak, and Brain Pop. The schools also have their own technician to handle technical issues.

DOE did not, however, develop measurable outcome criteria to assess the success of the project (e.g., whether students were better prepared for college in the 21st century). Furthermore, the project was not in existence long enough for DOE to determine whether the goals of the Plan were successful for the 10 NYC21C schools. DOE changed its approach to the NYC21C project and the program has evolved; therefore, the original purpose of this program can no longer be evaluated using the 2009 plan. The 10 schools that were included in the NYC21C initiative have been transferred into different initiatives with no clear specific measurable criteria to use in assessing the effectiveness of the NYC21C initiative.

DOE did not communicate with the NYC21C schools on a regular basis. DOE also did not have all the required documentation for the NYC21C initiative. In addition, DOE provided us with a list of inventory and expenditures for technology that was obtained by City schools, but did not keep track of the inventory each school received from DOE nor the amount associated with it. Consequently, we were unable to determine whether the NYC21C schools met their budgetary allocations.

Audit Recommendations

DOE should:

• Establish and specify firm measurable goals, objectives, and guidelines for all future DOE projects;

• Monitor and communicate with schools to ensure New York City schools continue to provide students with innovations for the purpose of 21st century learning;

• Keep track of all New York City schools’ technology inventory and expenditures for each school year for program cost effectiveness purposes;

• Ensure required documentation related to the NYC21C initiative and future pilot programs are submitted and maintained; and

• Ensure that schools in the Career and Technical Education (CTE) program submit all documentation needed for approval into the program.

Agency Response

In their response, DOE officials generally agreed with all the recommendations, but disagreed with the report, stating: “As discussed with the audit team throughout the audit and at the exit conference, the NYC21C Project that is the subject of this audit was a small educational initiative that began in the Department’s Office of New Schools within the Division of Portfolio Planning in Spring 2009, and that received support and assistance from the Department’s Office of Career and Technical Education (“CTE”) and Division of Instructional and Information Technology (“DIIT”). . . . While the Comptroller, of course, is free to opine that this small educational initiative to bring together school leaders with similar goals around the use of new technologies in the classroom should have been designed, executed and controlled by the district, with each of the ten participating schools actively monitored and measured against specific NYC21C benchmarks, that simply is not what the NYC21C Project was. The project emphasized developing principal leadership to manage and integrate new technologies for learning, and teacher training around the use of new technologies in instruction, with an additional goal of fostering collaboration among principals and teachers across schools on those topics.”

Auditor Comment: DOE’s current statements and comments that NYC21C was a “small educational initiative” which, by implication, need not “have been designed, executed and controlled by the district, with each of the ten participating schools actively monitored and measured against specific NYC21C benchmarks,” are at odds with how DOE described the NYC21C when it launched the initiative in 2009.

• In May 2009, DOE released version 1.0 of its 5-Year Information Technology Strategic Plan (the Plan), which describes NYC21C as a “3 year R&D effort to roll out new and existing schools that will scale system wide by 2013” as reflected in the following diagram from the Plan (emphasis added). (See Chart I)

• On May 12, 2009, DOE issued a press release announcing the NYC21C program as a “research and development project” and stating that the initiative would “develop new schools and introduce innovations that leverage technology to provide for more personalized instruction and better prepare students for college and careers” (emphasis added). The announcement of the NYC21C initiative was made at the NYC iSchool, which the press release described as “a new small selective high school in SoHo whose success at incorporating technology into everyday learning will serve as a model for the development of other schools in the NYC21C initiative.” (See Appendix I)

• Emphasizing the importance of the initiative, in June 2009, the then Chancellor authored a column that called the NYC21C program “an initiative I think is the most exciting work we are now embarking on here in New York City’s public schools” (emphasis added). (See Appendix II)

Now in 2012, in its response, DOE indicates that NYC21C was a “small educational initiative” and an “educational plan” whose goal was to develop principal leadership and teacher training around leveraging technology. These statements are inconsistent with the publicly released written Plan and published statements made by DOE when it launched this initiative in 2009 (See Chart I, Appendix I, and Appendix II). As DOE has provided no support for its current explanation and written evidence is at variance with it, we reject DOE’s explanation.

DOE also disputes that there was a need for close and specific NYC21C monitoring or evaluation of its effectiveness against established benchmarks. We respectfully disagree and note that NYC21C was described by the then Chancellor as “the most exciting work we are now embarking on here in New York City’s public schools” and DOE’s 2009 Plan called for a system wide roll-out by 2013. We would opine that a pilot of such importance that it was to be the basis of a system wide roll-out should have been closely monitored and evaluated by DOE centrally. Creating new initiatives, plans, or programs without also establishing specific criteria to measure their success does not do justice to those programs or help DOE to determine which actually work.