Audit of the Operating Practices of the City University of New York’s College Discovery Program

Letter from the Comptroller

To the Residents of the City of New York:

My office has audited the City University of New York (CUNY) College Discovery Program, which provides comprehensive academic, financial, and social support to assist capable community college students experiencing various educational and financial challenges. The objective of this audit was to determine the effectiveness of CUNY’s College Discovery Program in meeting the intended outcomes for program participants.

The audit found that the College Discovery Program was generally effective in meeting the intended outcomes for program participants during the audit’s scope period. First-year College Discovery Program students in the Fall 2019 cohort remained in the program and graduated at generally higher rates than those in the general student population at the six participating community colleges. Additionally, results of a student survey conducted by the audit showed that the majority of respondents had a favorable opinion of the program.  Program outcomes, however, were not uniform across the six community colleges. This might have been due in part to the de-centralized nature of the program and to the absence of established Academic Plans at all six colleges. Lastly, the audit found that students with low cumulative grade point averages (GPAs) or extended part-time statuses would benefit from more counseling and tutoring assistance.

To strengthen the effectiveness of the College Discovery Program, the audit makes six recommendations, including that CUNY: (1) develop best practices to be used in all six community colleges, which could include aligned Academic Plans, common academic goals (such as retention and graduation goals), and improved processes for measuring program success; (2) consider conducting its own surveys of program students on a periodic basis; (3) develop mechanisms to more closely monitor  students whose GPAs fall below 2.0 and develop plans for these students to obtain the support services they need; and (4) more closely monitor the part-time enrollment status of program students so that required waivers are obtained and students receive the counseling and tutoring services they need to graduate on time.

The results of the audit have been discussed with CUNY officials, and their comments have been considered in preparing this report. CUNY’s complete written response is attached to this report.

If you have any questions concerning this report, please email my Audit Bureau at audit@comptroller.nyc.gov.

Sincerely,

Brad Lander
New York City Comptroller

Audit Impact

Summary of Findings

The audit found that the College Discovery Program was generally effective in meeting the intended outcomes for program participants during the audit’s scope period (September 2019 to August 2022). First-year College Discovery Program students in the Fall 2019 cohort were retained (as of the Fall 2020 term) and graduated (by August 2022) at generally higher rates than those in the general student population. Additionally, results of a student-focused survey showed that the majority of respondents had a favorable opinion of the program.

However, the audit also identified areas where improvements could be made to strengthen the program. In particular, CUNY should develop best practices for the program by analyzing the approaches adopted by community colleges which achieved significantly higher retention rates than the general student population.^[1] Bronx Community College stood out as a clear outlier in improving graduation rates—by more than 12 percentage points over the general student population.

Intended Benefits

The audit identified areas where improvements could be made to strengthen the effectiveness of CUNY’s College Discovery Program.

Introduction

Background

CUNY’s College Discovery Program was established in February 1964 as a higher education opportunity program that provides comprehensive academic, financial, and social support to assist capable students who otherwise might not be able to attend community college due to their educational and financial circumstances. The overall mission of the program is to help academically and economically disadvantaged students receive the support they need to remain in college and graduate and be given the opportunity to thrive in an increasingly competitive society. The program is currently offered at six of CUNY’s two-year community colleges (Borough of Manhattan, Bronx, Hostos, Kingsborough, LaGuardia, and Queensborough).^[2]

To be eligible to participate in the College Discovery Program, an individual must:

• Be a legal resident of New York City for at least one year prior to entering the college;
• Be a high school graduate or recipient of a New York State-approved equivalency diploma;
• Have a college admissions average of less than 80;
• Be a first-year college student; and
• Have a gross family income that meets State guidelines.

Once an individual is determined eligible, as a final condition of their admittance, the individual must attend and successfully complete a mandatory summer program. Based on each student’s summer academic performance, College Discovery Program staff work with each student to develop their college curriculum.

The College Discovery Program awards supplemental financial assistance to its students, in addition to the financial aid that is available to regularly admitted students. The assistance includes stipends and three types of student fees (student activity fee, student senate fee, and consolidated service fee) that are paid directly to the community college on behalf of each student.

A College Discovery Program student must earn a minimum of 60 credits within 7 terms and must have a cumulative grade point average (GPA) of at least 2.0 to graduate with an associate degree from their community college. The 60 credits must meet the requirements of both the college’s core curriculum and the student’s declared major. Students are required to attend classes on a full-time basis (12 credits or more); however, there are certain exceptions which permit a student to attend classes on a part-time basis (at least 6, but no more than 11 credits). For instance, although a student who is placed on academic probation is encouraged by CUNY to attend classes on a full-time basis, they may be allowed to attend classes on a part-time basis to lessen their course load and improve their cumulative GPA.

Upon admission to the College Discovery Program, a student is assigned to a counselor who is responsible for guiding the student throughout their time in the program. In addition, the student is required to attend tutoring throughout their first year and later if they are on academic probation. Tutoring is also available to all program students who continue past their first year and would like additional help to improve their potential for academic achievement.

The CUNYfirst system (CUNYfirst) is CUNY’s system of record that houses student data. Each student record in the system contains various information, including academic information (e.g., the community college the student is enrolled in; whether the student is enrolled on a part-time or full-time basis; the term when the student began as a freshman at the community college; courses taken by the student; credits attempted and earned each term by the student; the student’s cumulative GPA for each term enrolled; the date when the student became active and inactive in a particular program; the student’s graduation date, if applicable; and the type of degree the student was awarded).

A total of 3,857 students were enrolled in the College Discovery Program at some point during the audit’s primary scope period of September 2019 (Fall 2019) through June 2022 (Spring 2022). Of this total, 2,024 were Hispanic, 1,093 were Black, 490 were Asian, 229 were White, and 21 were American Indian/Native Alaskan. There were 1,458 male students and 2,399 female students. For the 2021–2022 Academic Year, the budget for the College Discovery Program totaled $3.9 million for all six community colleges.

Objective

The objective of this audit was to determine the effectiveness of CUNY’s College Discovery Program in meeting the intended outcomes for program participants.

Discussion of Audit Results with CUNY

The matters covered in this report were discussed with CUNY officials during and at the conclusion of this audit. An Exit Conference Summary was sent to CUNY and discussed with CUNY officials at an exit conference held on May 18, 2023. On June 16, 2023, we submitted a Draft Report to CUNY with a request for written comments.  We received a written response from CUNY on July 3, 2023.

In its response, CUNY stated that it “appreciates the valuable feedback from the OCC [Office of the City Comptroller] audit.” CUNY also stated that it was “pleased to note that OCC’s recommendations are aligned with practices already in place or implemented during AY [Academic Year] 2022-2023.”  Based on this statement, the audit concludes that CUNY accepts the audit’s six recommendations.

CUNY’s written response has been fully considered and, where relevant, changes and comments have been added to the report. In a number of instances, CUNY’s response references preliminary findings that were discussed during the audit but were not presented in the Draft Report. Consequently, few changes were made to the Final Report.

The full text of CUNY’s response is included as an addendum to this report.

Detailed Findings

The audit found that the College Discovery Program resulted in better retention rates at all six participating community colleges, and better graduation rates at four of the colleges. A survey of students in the program also showed generally high satisfaction rates.

Program outcomes, however, were not uniform across the six participating colleges. This may be due in part to its de-centralized nature and to the absence of established Academic Plans at all six colleges. The audit also found that students with low cumulative GPAs or in extended part-time status would benefit from more assistance and recommends that the colleges work together to develop best practices to enhance performances across the board.

The following sections of this report discuss these findings in further detail.

College Discovery Program Generally Improves Retention and Graduation Rates

The College Discovery Program’s mission is to give capable students who face additional challenges an opportunity to obtain an associate degree with additional academic support, financial assistance, tutoring and counseling services. CUNY measures the program’s success by comparing College Discovery Program and CUNY-wide retention and graduation rates for full-time first-year students entering associate degree programs.

CUNY’s Office of Applied Research, Evaluation, and Data Analytics (OAREDA) publicly reports retention and graduation data for two of its four opportunity programs (College Discovery Program and SEEK) on its main website.^[3] However, CUNY does not report this data separately by each individual opportunity program or by individual college; rather, CUNY reports combined outcomes for both programs and all colleges.

Based on data provided by CUNY, the auditors calculated the retention and graduation rates of each community college for first-year College Discovery Program students in the Fall 2019 cohort and compared the rates to those of the general student population.^[4] Results of this analysis are summarized below and in Chart I.

Retention Rates

For the Fall 2019 cohort, the retention rates were higher for the College Discovery Program than for the general student population at all six community colleges. The most significant percentage point differences in retention rates existed at three community colleges: Bronx, Hostos, and Kingsborough. The retention rates for College Discovery Program students were 15.5 percentage points higher than they were for the general student population at Bronx and Hostos community colleges, and was 12.2 percentage points higher at Kingsborough Community College.

Graduation Rates

For the Fall 2019 cohort, the graduation rates were higher for the College Discovery Program than for the general student population at four of the six community colleges.^[5]  The most significant percentage point differences in graduation rates between the College Discovery Program and the general student population existed at Bronx Community College, where the graduation rate for College Discovery Program students was 12.5 percentage points higher than it was for the general student population.

In two of the six community colleges, the graduation rates were slightly lower for the College Discovery Program students than for the general student population—nine-tenths of a percentage point lower at LaGuardia and three-tenths of a percentage point lower at Queensborough.

Chart I

In its response, CUNY stated that the differences in graduation and retention rates for College Discovery Program students could be attributable not just to program differences among the six community colleges but also to “differences in populations based on their selection into the CD program [the College Discovery Program] and/or campuses.” The auditors present this comparison to illustrate the performance of the College Discovery Program students within their respective community colleges, notwithstanding any non-programmatic distinctions that may exist among the colleges. As recommended in this audit and agreed to by CUNY officials, CUNY should identify the best programmatic practices being used by the community colleges to further improve outcomes and to better measure the success of the College Discovery Program.

Survey Respondents Generally Viewed the College Discovery Program Favorably

Surveys can serve as a vital resource that colleges can use to enhance their programs and operations. By regularly collecting and analyzing student feedback, a college can make more informed decisions to help improve student performance and achieve institutional goals. Students have a valuable perspective on the quality of their education. They can provide important insights to help management address challenges and improve program operations.

The audit successfully sent a survey questionnaire to 3,708 College Discovery Program students who were enrolled at some point during the audit scope period from September 2019 to June 2022 and received 181 (5%) responses.

The number of respondents to the survey represents a small percentage of the students who participated in the program, so the degree to which their responses reflect the general consensus of College Discovery Program students as a whole is unknown. Nonetheless, the results are suggestive and worth reporting.

The survey questionnaire consisted of 17 questions intended to capture students’ feedback, insights, and opinions about the program.  For example, one question asked whether the students thought that the required Summer Bridge Program was helpful in preparing them for community college.^[6] Of the 179 students who responded to this question, 141 (79%) stated that they found the program to be very helpful and another 29 (16%) stated that it was somewhat helpful. Chart II below summarizes the results of a few of the key questions asked, related to the program’s counseling and tutoring student support services, and to the program’s overall effectiveness (the full survey results for questions in which respondents were asked to rate their levels of satisfaction are presented in the report Appendix). The results were positive overall, as follows:

• Of the 166 students who responded, 157 (95%) found counseling services to be effective in preparing them to graduate.
• Of the 139 students who responded, 129 (93%) found the tutoring services to be effective in helping the students successfully complete their classes.
• Of the 181 students who responded, 164 (91%) found the College Discovery Program overall to be effective in helping them to graduate.

Chart II

The survey also asked students three open-ended questions. One question asked the students to provide any suggestions they might have for improving the College Discovery program.  Some students provided suggestions on how they believed the program could be improved, such as:[7]

• The program should do a better job informing the students about the variety of majors that are available. One student specified that the College Discovery Program should inform the students about all majors, including arts majors.
• There should be more in-person meetings with other College Discovery Program students, as was done prior to the pandemic.
• There should be an increased effort to recognize students’ achievements. One student, for example, stated that “it would be nice to have an honors certificate/award” for achieving certain goals.
• Tutoring services should be expanded. One student stated that the program should “put more hours on tutoring sessions and get more tutors for each subject.” Others stated that there should be tutoring for all majors, that the College Discovery Program should offer tutoring in more accessible areas for students with disabilities, and that “it would be better if they had higher level tutors for the more advanced classes.’’

The auditors encourage CUNY to undertake its own surveys of College Discovery Program students. Surveys with a greater response rate would help CUNY gather feedback from students who participate in the program and help the program identify areas where further improvements could be made.

Improvements Needed in Assisting Students with Cumulative GPAs Below 2.0 or in Part-time Status

College Discovery Program Students with Cumulative GPAs Below 2.0 Not Provided Adequate Assistance

If a College Discovery Program student’s cumulative GPA falls below 2.0, the student may be  placed on academic probation until the cumulative GPA rises to 2.0 or above. In some circumstances a student can have a cumulative GPA of less than 2.0 and, based on the number of attempted credits, still be considered in good academic standing.^[8]

If a student is placed on probation, they are required to attend tutoring, and College Discovery Program counselors are responsible for assisting students to ensure that they remain on track to graduate with an associate degree. A student who does not exit academic probation after two consecutive terms can be removed from the program unless granted an exemption by the director of the College Discovery Program.

College Discovery Program directors at 4 of the 6 community colleges stated that students should maintain a cumulative GPA of at least a 2.0 per term to maintain their spot in the program. The directors also stated that if a student’s cumulative GPA falls below 2.0, then they are placed on academic probation. The directors at the remaining two community colleges (Borough of Manhattan and Queensborough Community Colleges) stated that they determine whether a student should be on academic probation based on the cumulative GPA and the number of credits the student has attempted to earn since enrolling in the community college. Nonetheless, these two officials also emphasized the importance of a student maintaining a cumulative GPA of 2.0 in order to graduate.

The auditors selected a sample of 78 College Discovery Program students from the student dataset CUNY provided on October 14, 2022.[9] The auditors reviewed information recorded in CUNYfirst and found that 13 of the 78 sampled students had cumulative GPAs below 2.0 for three or more consecutive terms, with the maximum being six consecutive terms for one student.[10]    The audit found that of the 13 sampled students with cumulative GPAs below 2.0 for three or more consecutive terms, two were dismissed from the program for academic reasons.

A review of documentation provided by CUNY showed that each of the remaining 11 students had been placed on academic probation at some point during their time in the program.  For 10 of these, CUNY provided no evidence that it had prepared any plans to help the students exit academic probation. CUNY provided evidence of such a plan for only 1 of the 11 students (from Kingsborough Community College)—a College Discovery Program Academic Probation/High Risk Contract signed by the counselor and the student explicitly requiring the student to meet with their “academic advisor on a weekly basis during the semester”  and stating that the student could be at risk of academic dismissal if their GPA remained below 2.0. Requiring only some of the students placed on academic probation to sign such a contract is evidence of inconsistent treatment and is potentially unfair.

For students with cumulative GPAs below 2.0 for three or more consecutive terms, the audit determined whether the community college: (1) held at least one counseling session in which the student’s academic status was discussed during each term that the student’s cumulative GPA fell below 2.0; and (2) provided the student with tutoring during each term that the student’s cumulative GPA fell below 2.0. The auditors’ review of documentation provided by CUNY for the 11 students showed that none of them received both types of support in each of the consecutive terms that the students’ cumulative GPAs fell below 2.0. Specifically, the auditors found that:

• Five of the 11 students received counseling and tutoring during some of the terms their cumulative GPAs fell below 2.0, but not during each term.
• Two of the 11 students did not receive any tutoring during the terms that their cumulative GPAs fell below 2.0, and only received counseling during some of the terms their cumulative GPAs fell below 2.0.
• Two of the 11 students only received tutoring during some of the terms their cumulative GPAs fell below 2.0 and did not receive any counseling in which the student’s academic status was discussed during the terms that their cumulative GPAs fell below 2.0.
• Two of the 11 students did not receive any tutoring or counseling in which the student’s academic status was discussed.

The auditors question whether the program is providing the support needed to ensure students most at risk remain in the program and graduate on time. Three of the 11 students started in the program prior to the Fall 2019 term and therefore should have graduated by June 2022 (the Spring 2022 term). However, none of these students had graduated by then. Of the two students who started in the Fall 2019 term, one graduated with an associate degree.

In its response, CUNY argues at length that auditors misinterpreted the requirements for placing students on academic probation (referred to by CUNY as Satisfactory Academic Progress). The auditors make no assertion regarding whether students should have been placed on academic probation. Rather, the auditors assessed whether assistance was provided to students whose cumulative GPAs fell below 2.0 for consecutive terms, which would have placed them at risk of not graduating on time.

Part-Time Program Students Not Consistently Granted Waivers or Provided with Assistance

A College Discovery Program student is allowed to study part-time for only one term (other than their graduating term). The student may be allowed to be part-time for additional terms, but this can only be granted by a waiver for each term that the student remains in part-time status. Since a student cannot remain in the College Discovery Program beyond seven terms, limiting the number of part-time terms a student can have is intended to help the student graduate on time.

Of the 3,857 students who were enrolled in the College Discovery Program during the scope period of Fall 2019 through Spring 2022, the auditors identified a total of 329 (9%) students who had part-time status for two or more terms, other than their graduating term.[11] Of the 329 students, 32 had part-time status for 4 to 6 terms.

For purposes of this analysis, the audit focused on the 32 students who had part-time status for 4 or more terms (well beyond the allowed number of terms under CUNY’s policies). Based on information provided by CUNY, it appears that 29 of the 32 students were not granted waivers by the University Dean or the College Discovery Program directors for a total of 116 part-time terms.[12]

CUNY provided several reasons students might remain in part-time status without official waivers being granted;^[13] however, even where waivers were officially provided, plans for assisting students to graduate on time (within seven terms) were not developed. CUNY indicated that counseling sessions were held with many of these 29 students, but there is no evidence that the students’ part-time status or their plans to graduate on time were discussed during these counseling sessions.

Not adequately monitoring the part-time enrollment status of College Discovery Program students increases the risk that these students will not be provided the assistance they need to graduate on time. The approval of a waiver allowing a student to attend classes part-time for an additional term beyond the number of terms automatically allowed is an opportunity for the community college to develop a formal plan of assistance to help the student graduate on time. Of the 29 students in extended part-time status, 13 started in the College Discovery Program prior to the Fall 2019 term and therefore should have graduated by June 2022 (the Spring 2022 term). However, none of these students had graduated by then.

As noted above, applying the required policy some, but not all the time, also potentially leads to unfairness and an inability by CUNY to assess the impact of a consistent application of policy to all students.

Academic Plans for College Discovery Program Were Not Prepared

The June 1994 Guidelines for the Structure and Operation of the College Discovery Program of the City University of New York, Section 3 (College Plans) requires that community colleges submit on an annual basis both an Academic Plan and a Fiscal Accountability Plan. Specifically, Section 4 (The Academic Plan) states that the plan should: (1) include the specific academic goals and the means to be used to achieve the goals; (2) describe the educational content of the program; (3) identify indicators to be used to assess whether the goals are being met; and (4) describe the tutoring and counseling services that are provided to College Discovery Program students from the time of their admission to graduation. In addition, Section 5 (The Fiscal Accountability Plan) states that the plan should describe the projected pattern of expenditures (e.g., the administrative, Summer Program, counseling, tutoring, and instructional costs) for the College Discovery Program and the means for tracking such expenditures.

Academic Plans

The audit found that none of the community colleges prepared the required College Academic Plans. According to CUNY, the last time a formal three-year College Academic Plan was submitted by any of the community colleges was in 2008 (15 years ago). CUNY officials provided no explanation as to why follow-up was not conducted with community colleges in prior years.

Although plans were not submitted for some time, each of the six directors stated that they met monthly with the CUNY University Dean of the Office of Special Programs (OSP) to discuss aspects of the College Discovery Program at their respective community colleges, including the strengths and weaknesses of the program, enrollment numbers, financial aid, and areas for improvement. OSP officials also argued that the community colleges submitted annual academic reports that presented “performance and programming” information. However, CUNY did not provide any details as to the types of “performance and programming” information that were included in these reports or share any examples of such reports with the auditors.

The absence of prospective plans and academic goals for its College Discovery Program limits CUNY’s ability to ensure appropriate benchmarks and approaches are established across the board. CUNY should consider introducing additional measures of the program’s effectiveness, such as measuring student satisfaction and job placements following graduation.

CUNY also appears to have missed opportunities to establish best practices for the community colleges, based for example on an assessment of Bronx Community College’s particularly high rates of both retention and graduation, and conversely, LaGuardia Community College’s and Queensborough Community College’s lack of improvement in graduation rates. There is an opportunity to ensure all Academic Plans are aligned at participating colleges.

Fiscal Accountability Plans

OSP officials stated that prior to Fiscal Year 2020, they required community colleges to submit annual budget projection and expenditure reports which contained the same information required by the Fiscal Accountability Plan. During Fiscal Year 2020, OSP introduced a new budget template and began meeting with officials at each community college to explain the revised budget template and budgeting process. In Spring 2020, the community colleges submitted their annual budgets using the new template. CUNY officials provided the auditors with the template as well as the Fiscal Year 2022 projected annual budget submitted by Bronx Community College.

Interestingly, the average cost per student in the College Discovery Program was approximately $2,205 in the 2021–2022 Academic Year.[14] The two community colleges with the highest average cost per student—LaGuardia ($3,223) and Queensborough ($2,442)—were also the community colleges for which the graduation rates of students in the program were slightly lower than they were for the students in the general population.  It should be noted that these two colleges also had the lowest percentage point differences in retention rates (+2.3 and +1.9 percentage points, respectively) among all six colleges.

Recent Developments

OSP officials informed the auditors that they recently developed internal policies regarding the College Academic Plan and the Fiscal Accountability Plan and compiled them into the Fiscal Year 2024 (July 1, 2023 through June 30, 2024) General Plan. Community colleges offering College Discovery Programs were required to complete and submit their General Plans by March 31, 2023.

CUNY is requiring the Fiscal Year 2024 General Plan to include: (1) retention goals—the percentage of “all program students [that] will persist from one academic year to the next academic year” and (2) graduation goals—the percentage of “new program students served each year [that] will graduate within 3-years.” The General Plan will also include the following goals that were not tracked previously:

• Percentage of incoming first-year program students who will enroll in and complete the four-week minimum summer program and matriculate in the fall semester; and
• Percentage of all enrolled program students who will meet the performance level required to stay in good academic standing.

The audit concludes that CUNY has begun to take the necessary steps to improve its academic planning efforts for the College Discovery Program.

Inaccuracies in Recording When Students Leave the College Discovery Program

Of the sample of 78 College Discovery Program students from the October 14, 2022, student dataset provided by CUNY (noted above), the audit identified 11 (14%) for which CUNYfirst maintained inaccurate information relating to when the students left the College Discovery Program. For 10 of the 11 students (at 5 of the 6 community colleges), the audit found that the change in status from active (e.g., when students started in the program)  to inactive (e.g., when students withdrew, transferred, or graduated from the program)  was not recorded until long after the students left the program.

For example, one student who attended LaGuardia Community College left the program as of the Spring 2019 term. However, this was not recorded until a year later, on June 16, 2020 (Spring 2020 term). In another example, for a student who attended the Borough of Manhattan Community College, CUNYfirst showed two dates of departure from the program, both of which were inaccurate. The audit also found that CUNY has no mechanism (e.g., system flags or periodic reconciliations) for monitoring the accuracy of the inactive dates in CUNYfirst.

CUNY acknowledged the inaccuracies in CUNYfirst for each of the 11 students  and further stated that due to “competing obligations” the change in status from active to inactive for students who leave the program may not always be done in a timely manner.   However, CUNY stated that it strives to enter this information as soon as possible.

Because program enrollment numbers are capped, the accurate and timely recording of when students leave the program directly impacts the number of students the program can enroll in the College Discovery Program each academic year. If students are not properly recorded as inactive, other students may be prevented from participating in the program, even if there are one or more vacancies.

Recommendations

To address the abovementioned findings, the auditors propose that CUNY:

1. Develop best practices to be applied in all six CUNY community colleges, including aligned General Plans for each fiscal year, common academic goals (including retention and graduation goals) and practices, to improve outcomes and the ability to measure success.

CUNY Response: CUNY agreed with this recommendation, stating that a General Plan was submitted to CUNY’s Office of Special Programs at the end of March 2023 that “will aid in developing common academic goals and practices.”

2. Consider conducting its own surveys of College Discovery Program students on a periodic basis to help determine the effectiveness of the program and identify areas for improvement.

CUNY Response: CUNY agreed with this recommendation in that it stated that the audit’s recommendations “are aligned with practices already in place or implemented during AY [Academic Year] 2022-2023.”

Auditor Comment:  CUNY stated in its response that it agrees that “surveys can serve as a vital resource that colleges can use to enhance their programs.” It also stated that it conducts CUNY-wide surveys of its student population. However, CUNY did not specifically state that it would conduct surveys of College Discovery Program students on the effectiveness of the program. The audit urges CUNY to do so to help it identify areas for improvement.

3. Develop mechanisms to more closely monitor the cumulative GPAs of College Discovery Program students that fall below 2.0 and develop plans for these students to obtain the support services they need to improve their GPAs and graduate on time.

CUNY Response: CUNY agreed with this recommendation in that it stated that the audit’s recommendations “are aligned with practices already in place or implemented during AY [Academic Year] 2022-2023.”

Auditor Comment:  Although CUNY stated in its response that support services plans are created for students on academic probation, it did not provide the auditors with any such plans for 10 of the 11 students cited in this report. Furthermore, the documentation provided by CUNY for the 11 students showed that none of them received both counseling and tutoring services during each of the consecutive terms that the students’ cumulative GPAs fell below 2.0. The audit urges CUNY to consistently prepare and implement support services plans for those students at risk of not graduating on time.

4. Consider requiring the use of a formal agreement between a counselor and an academic probationary student, such as the College Discovery Program Academic Probation/High Risk Contract, so that students are aware of the program’s expectations for them in the upcoming term and of the support services that are available.

CUNY Response: CUNY agreed with this recommendation, stating that it had already included this requirement in the General Plan and that “[t]he General Plan includes contracts for students on Academic Probation.”

Auditor Comment: The most recent template for the General Plan that CUNY provided the auditors on March 3, 2023, included no such requirement. CUNY should ensure that its General Plans contain this requirement.

1. More closely monitor the part-time enrollment status of the College Discovery Program students so that: (1) required waivers are obtained; (2) the waivers include plans to help students stay on track to graduate (i.e., within seven terms); and (3) the students receive the counseling and tutoring needed to help them graduate on time (i.e., within seven terms).

CUNY Response: CUNY agreed with this recommendation, stating that it “worked alongside CUNY’s OAREDA team to develop a centralized database that tracks student enrollment status by term and academic year. The database will go live for the CDP Directors and Office of Special Programs staff in FY24.”

Auditor Comment: The auditors are pleased that CUNY identified the need to track the part-time status for all students and developed a centralized database to do so by term and academic year. For the new database to be successful, it should include features that help CUNY ensure that required waivers are obtained; that the waivers include plans to help the students stay on track to graduate; and that the students receive the counseling and tutoring needed to help them graduate on time.

6. Ensure that the date a student becomes inactive in the College Discovery Program is accurately recorded in CUNYfirst in a timely manner.

CUNY Response: CUNY agreed with this recommendation, stating that the new centralized database that will track “student enrollment status by term and academic year” will also “include accurate student coding across the programs.”

Recommendations Follow-up

Follow-up will be conducted periodically to determine the implementation status of each recommendation contained in this report. Agency reported status updates are included in the Audit Recommendations Tracker available here: https://comptroller.nyc.gov/services/for-the-public/audit/audit-recommendations-tracker/ https://comptroller.nyc.gov/services/for-the-public/audit/audit-recommendations-tracker/https://comptroller.nyc.gov/services/for-the-public/audit/audit-recommendations-tracker/

Scope and Methodology

We conducted this performance audit in accordance with Generally Accepted Government Auditing Standards (GAGAS). GAGAS requires that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions within the context of our audit objective(s). This audit was conducted in accordance with the audit responsibilities of the City Comptroller as set forth in Chapter 5, §93, of the New York City Charter.

The primary scope of this audit covered the period of September 2019 (Fall 2019) through June 2022 (Spring 2022) at each of the six community colleges (Borough of Manhattan, Bronx, Hostos, Kingsborough, LaGuardia, and Queensborough). In terms of calculating the graduation rates for College Discovery Program students, the audit scope period extended through August 2022 (Summer 2022).

To obtain an understanding of the responsibilities and regulations governing CUNY’s College Discovery Program, the auditors reviewed and used the following documents as their audit criteria:

• The June 1994 Guidelines for the Structure and Operation of the College Discovery Program of the City University of New York;
• CUNY’s Student Aid Manual, Chapters 2 (Student Eligibility Requirements), 9 (Packaging), 10 (Special Programs), and 13 (New York State Award Programs); and
• CUNY’s internal memos and policies regarding the College Discovery Program, including Policy 1.12 College Discovery; College Discovery and SEEK Waiver of Leave of Absence/Readmission Policy dated February 4, 2020; Financial Aid Allocation Process (for stipends and student fees) dated August 17, 2020; and Part-Time Student Financial Aid (for stipends and student fees) dated September 28, 2020.

The auditors also reviewed various documents obtained from the CUNY website, including the College Discovery (CD) Program ; the Eligibility & Model of College Discovery & SEEK; and the History & Purpose of College Discovery & SEEK. In addition, the auditors reviewed CUNY’s CD Financial Aid Funds Overview which contained, per community college, the total number of students who were provided College Discovery Program financial aid (and the associated dollar amounts) for the Fall 2019, Spring 2020, Fall 2020, and Spring 2021 terms.^[15]

Comptroller’s Directive #1, Principles of Internal Control, §5.4, Control Over Computer Information Processing, was considered to assess CUNY’s controls over the recording of student information in CUNYfirst and the accuracy of the information generated from the system.

The auditors reviewed the organizational chart for CUNY’s central office and identified key officials to be interviewed to gain a general understanding of CUNY’s College Discovery Program and the controls in place over the processes that were significant to achieving the audit objective. The auditors interviewed the Vice Chancellor of Student Affairs and Enrollment Management; the University Dean, the Administrative Executive Coordinator, and the Academic Support Coordinator from the Office of Special Programs; and the University Deputy Director of Financial Aid from the Office of Student Financial Assistance.

The auditors also reviewed the College Discovery Program organizational charts for each of CUNY’s six community colleges and identified key officials to be interviewed to gain an understanding of how each college manages the College Discovery Program. The following key College Discovery Program individuals were interviewed from the community colleges: a Dean of Enrollment Management; five directors; an Associate Director/Academic Support Manager; and six counselors/counseling coordinators.^[16]

To familiarize the auditors with CUNYfirst and its various features and functions as they relate to the handling of the academic and financial aid processes for College Discovery Program students, the auditors observed demonstrations of the system. During the demonstrations, the auditors learned about the College Discovery Program student application, enrollment, and transfer processes. The auditors also reviewed the related three CUNYfirst operating manuals: PeopleSoft Campus Solutions 9.2: Recruiting and Admissions, PeopleSoft Campus Solutions 9.2: Student Records, and PeopleSoft Campus Solutions 9.2: Financial Aid.

To obtain a general understanding of the processes involved with the reporting of the College Discovery Program’s statistical information (e.g., retention and graduation rates) from CUNYfirst at the community-college level, the auditors judgmentally selected three community colleges and interviewed two Directors of Institutional Research and a Dean of Institutional Effectiveness.^[17]

In addition, the auditors wanted to familiarize themselves with the process of how CUNY tracks the tutoring and counseling services provided to College Discovery Program students.  The auditors learned that not all the community colleges use the same system to manage these services.^[18]  Accordingly, the auditors judgmentally selected Bronx Community College and observed a demonstration of its Starfish system, which is the system this college uses to manage its tutoring and counseling services.

Over the course of the audit, three student datasets—each containing the population of students enrolled in the College Discovery Program at each of the six community colleges during the audit scope period of September 2019 (Fall 2019) to June 2022 (Spring 2022)—were received. The following summarizes each of the three datasets received:

• The first student dataset was provided on October 14, 2022, and contained the details for each student, such as the student’s identification number, the student’s preferred email address, the first term that the student was enrolled at CUNY (if it was within the audit scope period), the term(s) that the student was enrolled at a CUNY college (and associated college), the term(s) that the student transferred to another CUNY college (and associated college), and the term that the student graduated with an associate degree (excluding graduations in the summer terms).
• The second student dataset was provided on January 23, 2023, and contained the same information as the first dataset, but had additional information, such as demographic information (e.g., gender and race) and the term(s) that the student was enrolled at a CUNY college either on a full-time or part-time basis.
• The third student dataset was provided on February 17, 2023, and contained the same information as the first dataset, but had additional graduation information to include graduations in the summer terms (Summers 2020, 2021, and 2022).

The auditors relied primarily on the first student dataset to conduct most of their testing, but also used the other two datasets for certain tests (explained in detail below). Each of the student datasets contained a total of 3,857 students who were enrolled in the College Discovery Program at each of the six community colleges at some point during the audit’s primary scope period of September 2019 through June 2022.

Various data reliability tests were conducted of the first student dataset, to check for questionable entries (including duplicates, blank fields, and clearly anomalous information). In addition, as part of the data reliability accuracy testing, the auditors stratified the population by community college and selected a total sample of 78 College Discovery Program students (75 were randomly selected using the Rat-Stats statistical sampling software and three were judgmentally selected) from the first student dataset. The auditors reviewed the information associated with this sample as it was presented in the student dataset and compared it to the information as it was presented on screenshots taken from CUNYfirst.^[19] In addition, for each of the sampled students, the auditors reviewed the screenshots and determined whether CUNY properly recorded when the students started in the program (recorded with an active status) and, if applicable, when the students were no longer part of the program due to graduating, transferring to a four-year college, or withdrawing from the program (recorded with an inactive status).

On February 8, 2023, OAREDA performed a “live reenactment” of how the first student dataset was generated.^[20] Subsequently, on February 22, 2023, OAREDA provided the auditors with the reenacted CUNY student dataset that contained information on all the College Discovery Program students who were enrolled during the audit scope period. As part of the data reliability completeness testing, the auditors used the reenacted dataset and stratified the population by community college. The auditors then randomly selected a total sample of 120 students (using the Rat-Stats software) to determine whether they were also listed in the first student dataset. The auditors also reviewed the information associated with this sample as it was presented in the reenacted student dataset and compared it to the information as it was presented in the first student dataset. Based upon the data reliability accuracy and completeness testing, CUNY’s first student dataset was determined sufficiently reliable for audit testing purposes. Since the other two student datasets contained most of the same information as the first student dataset, the auditors determined that they were also sufficiently reliable for audit testing purposes.

Additionally, for each of the 78 sampled students, the auditors reviewed CUNYfirst screenshots and determined whether any of these students had cumulative Grade Point Averages (GPAs) below 2.0 for more than two terms. If so, the auditors determined whether there was adequate evidence that additional support services (e.g., counseling and tutoring) were provided to help the students graduate on time.  The auditors also determined whether plans were prepared to help these students exit academic probation in conjunction with their being allowed to continue in the program.

The auditors reviewed the second student dataset containing information on part-time terms and identified those students who had a part-time status for two or more terms, other than their graduating term (a student is allowed to have a part-time status for only one term, other than their graduating term).  For purposes of this analysis, the audit focused only on those students who had a part-time status for four or more terms. For each of these students, the auditors determined whether waivers had been granted for each part-time term (beyond the allowed number of terms) either by the University Dean of OSP or the College Discovery Program directors as required. In addition, the auditors determined whether the waivers contained plans for helping the students graduate within seven terms.

In addition, for each of the six community colleges, the auditors calculated the one-year retention rates (the percentage of students who persist from one academic year to the next)  and three-year graduation rates for the full-time first-year College Discovery Program students for the Fall 2019 cohort based on data provided by CUNY.^[21] The auditors then compared these rates to the general student population retention and graduation rates provided by CUNY. The purpose of this analysis was to determine how well the College Discovery Program students were performing in relation to the regularly admitted students.

The auditors reviewed the College Discovery Program budget letters for each of the six community colleges for three Academic Years (2019–2020, 2020–2021, and 2021–2022) to determine the budgeted amounts and maximum number of students allowed to be enrolled in the program. The auditors also calculated the cost per student at each of the colleges based on the budget information and the actual number of students enrolled.

On November 23, 2022, the auditors sent a survey questionnaire, containing 17 questions,  to the 3,857 College Discovery Program students who were enrolled during the audit scope period. The auditors only received 112 responses. Accordingly, in an effort to increase the number of responses to the survey questionnaire, the auditors redistributed the same survey questionnaire on February 16, 2023, to the same students and received an additional 69 responses. In total, the auditors were able to successfully send the survey questionnaire to 3,708 students (the survey questionnaire was unsuccessfully sent to 149 students) and received responses from 181 students. The purpose of the survey was to obtain the students’ feedback, insights, and opinions about the program.

Although the results of the tests involving samples were not projectable to their respective populations, these results, together with the results of other audit procedures and tests, provided a reasonable basis for determining the effectiveness of CUNY’s College Discovery Program in meeting the intended outcomes for program participants.

Appendix I

The following summarizes the full results of the survey questionnaire per community college:

Current status of student in the College Discovery Program

Helpfulness of the required summer program in preparing student for college

Academic, personal, financial, & career counseling services provided to student

Number of times on average student met with their counselor each term

Effectiveness of counseling services in helping student to graduate

Tutoring services provided to student

Number of times on average student received tutoring services each term

Effectiveness of tutoring services in helping student successfully finish classes

Stipend amount received by student

Overall effectiveness of College Discovery Program in helping student to graduate

Ethnicity of student

*Other includes multi-racial, mixed, or Jewish students.

Gender of student

*Other includes non-binary students.

Primary language of student

*Other includes the following languages: Albanian, Bengali, Bosnian, Cantonese, Haitian-Creole, Indonesian, and Urdu.

---

Endnotes

[1] The retention rate is the percentage of students who persist from one academic year to the next (i.e., students who enroll in the Fall term at a college and are still enrolled at the same college in the subsequent Fall term).

[2] There are a total of 25 CUNY colleges throughout the five boroughs: 7 are two-year community colleges, 12 are four-year colleges, and 6 are graduate and professional schools. CUNY only offers a College Discovery Program at 6 of the 7 community colleges. Guttman Community College, which began operating in August 2012, does not offer this program.

[3] SEEK is the acronym used by CUNY for its other opportunity program at a four-year college called Search for Education, Elevation, and Knowledge.

[4] The number of first-year College Discovery Program students enrolled in the Fall 2019 Cohort per community college were as follows: 235 (Borough of Manhattan); 178 (Bronx), 132 (Hostos), 166 (Kingsborough), 152 (LaGuardia), and 93 (Queensborough).

^[5] In its response, CUNY questioned the graduation rates presented in the report, arguing that the dataset auditors used to calculate the graduation percentage only included the term that the students graduated with their associate degrees, not the college from which they graduated. CUNY is incorrect on this. The dataset used by the auditors, which CUNY provided on February 17, 2023, did in fact include information on the community colleges from which the students graduated. CUNY has offered no other reason for questioning the College Discovery Program graduation rates presented in this report. It should be noted that the auditors relied on CUNY’s graduation rates for the general student populations at the six community colleges.

Furthermore, it is important to note that the College Discovery Program graduation rates that the audit presents in this report for the six community colleges exceed CUNY’s College Discovery Program graduation rates for these community colleges, based on numbers provided by CUNY concerning the first-year College Discovery Program students who were enrolled in the Fall 2019 term and CUNY’s numbers concerning how many of these students had graduated from these community colleges by August 2022.

[6] To be eligible to participate in the College Discovery Program, a prospective student must attend the Summer Bridge Program and take non-credit bearing introductory courses in English, mathematics, and writing. In addition, the student must participate in weekly skill building workshops, daily tutorials, several cultural events, and team building exercises to prepare for the college academic experience. Based on each student’s summer academic performance, College Discovery Program staff work with each student to develop an Educational Action Plan that will aid in “mapping out their college journey.”

[7] It should be noted that the audit did not validate the suggestions provided by the students. Rather, the audit simply reports the results at face value. Some of the suggestions might relate more to CUNY as a whole or to the individual community colleges than they do to the College Discovery Program.

[8] If a student’s GPA falls below a 2.0, the student is generally placed on academic probation until the cumulative GPA rises to 2.0 or above. The probationary period for College Discovery Program students is for a maximum of two consecutive terms. According to CUNY officials, it is possible that a student can have a cumulative GPA of less than 2.0 for more than two terms and still be in good academic standing. They stated that the CUNY Board of Trustees’ policy for good academic standing is that students should maintain a cumulative GPA based on the number of credits/units attempted as follows: (1) a minimum GPA of 1.50 if the number of credits attempted by the community college student is between 0.5 and 12; (2) a minimum GPA of 1.75 if the number of credits attempted is between 13 and 24; and (3) a minimum GPA of 2.00 if the number of credits attempted is at least 25. An individual community college can establish a higher standard for remaining in good academic standing (such as maintaining a cumulative GPA of at least 2.0 regardless of the number of credits attempted).

[9] This dataset included 3,857 College Discovery Program students who were enrolled during any of the terms within the audit scope period of September 2019 through June 2022.

[10] For purposes of this analysis, the auditors reviewed cumulative GPA information recorded in CUNYfirst for each student and identified those students who had GPAs below 2.0 for three or more consecutive terms. Next, we determined whether these at-risk students received additional counseling and tutoring services to help them to graduate on time.

[11] The auditors reviewed the second student dataset provided by CUNY on January 23, 2023, containing information on part-time terms to identify those students who had a part-time status for two or more terms, other than their graduating term.

[12] It should be noted that for informational purposes the auditors provided CUNY with a listing of the names and student identification numbers of the 297 students (329 minus 32) who had part-time status for 2 to 3 terms, other than their graduating term. Because these students were excluded from the analysis, the auditors did not determine whether waivers were granted to these students by CUNY.

^[13] In its response, CUNY argued that all of the students had acceptable reasons for being in part-time status and that waivers are optional for every term that a student is in part-time status. This was the first mention by CUNY that waivers are optional; further, this position is at odds with CUNY’s written policies and procedures, which state that waivers are required whenever a student exceeds the limit on the number of part-time terms that are automatically allowed.

[14] To calculate the approximate average cost per College Discovery Program student in the 2021-2022 Academic Year at each of the community colleges, the auditors used the budget and enrollment information provided by CUNY. For each community college, the auditors divided the budgeted amount for Academic Year 2021-2022 by the Fall 2021 enrollment number. In its response, CUNY argued that the auditors’ calculation is incorrect because it took an entire academic year’s funding but only included the fall student population in the calculation, ignoring the spring population. In calculating a conservative (i.e., a lower) estimate of the average cost, auditors applied the fall student population to the entire academic year’s funding and did not reduce the size of the population to reflect the lower enrollment in the program during the spring term.

[15] The financial aid included stipends and student fees.

[16] At Queensborough Community College, the director position was vacant at the time the auditors conducted their interviews. Thus, the auditors selected to interview the Dean of Enrollment Management who was acting as director of the College Discovery Program until the vacancy was filled.

[17] The three judgmentally selected community colleges were Bronx, Kingsborough, and LaGuardia.

[18] Three of the community colleges (Bronx, Hostos, and Queensborough) use the Starfish system to track both tutoring and counseling services. Kingsborough and LaGuardia community colleges use the Scholarship system and Student Engagement Management System respectively to track both tutoring and counseling services. Borough of Manhattan Community College uses the Upswing system for tutoring services and the Titanium system for counseling services.

[19] CUNY did not provide the auditors with read-only access to CUNYfirst. Accordingly, OSP officials, in the presence of the auditors, retrieved certain information for the 78 sampled students from CUNYfirst on January 13, 2023 and on February 10, 2023. The auditors then took screenshots from CUNYfirst.

[20] CUNY provided the auditors with step-by-step instructions on how data was extracted from CUNYfirst into its own local database (Institutional Research Database) to create the first student dataset.

[21] The auditors used the second student dataset to calculate the retention rates and the third student dataset to calculate the graduation rates.

---

Addendum

Date: July 3, 2023

To: Maura Hayes‐Chaffe, Deputy Comptroller for Audit, Office of the City Comptroller

From: Reine T. Sarmiento, Vice Chancellor for Enrollment Management

RE: CUNY’s Response to the College Discovery Program Audit (Report #ME22‐081A)

CUNY would like to thank the Office of the City Comptroller (OCC) for its commitment to its fiduciary responsibility to the College Discovery Program (CDP) through the College Discovery audit, which lasted from August 2021 to June 2023. The mission of the CDP “is to provide higher education equality to students who otherwise would not have such access. Program efforts are intended to increase education, social capital, and workforce expertise in our City and State.” Program services include mandatory summer connection, need‐based financial aid, academic, personal, career, and financial counseling, peer mentoring program, tutoring services, CD seminars, skills building, and career development workshops.

The audit review period of September 2019 to June 2022 occurred primarily during the national state of emergency brought on by the COVID‐19 pandemic. The University needed to pivot its normal business operations to serve students in a manner not previously done in higher education. Despite the challenges, the CDP staff at the campuses and the Central Office (The Office of Special Programs) have achieved remarkable success through the Program’s embedded support systems‐counseling, tutoring, financial assistance, and academic support. Survey results conducted by the OCC indicate that the CDP students agreed that the Program was beneficial and that support services were excellent, even in a national emergency.

CUNY appreciates the valuable feedback from the OCC audit. The University was pleased to note that OCC’s recommendations are aligned with practices already in place or implemented during AY 2022‐2023.

Before the final draft of the audit, CUNY would like to request that the items below be corrected on pages 4, 5, and 12. CUNY has clarified these items and previously corrected the errors in the preliminary draft; however, these errors remain unchanged.

1. Pg. 4 – College Discovery Program Generally Improves Retention and Graduation Rate.
   CUNY’s response: Within this section heading, the auditors erroneously cited that CUNY’s Office of Special Programs (OSP) reports public data. OSP does not publish data for the public; it is CUNY’s Office of Applied Research, Evaluation, and Data Analytics (OAREDA).
2. Pg. 5 – Graduation Rates
   CUNY’s response: The 3‐Year Graduation Rate – College Discovery Program Students for Fall 2019 Cohort – is an incorrect calculation by the auditors.

The dataset the auditors used to calculate the graduation percentage only included the term that the students graduated with their associate degrees, not the college from which they graduated. The auditors did not request this information. Thus, the auditors erroneously concluded that the first CUNY college the student attended was the college from which they received their associate’s degree. This incorrect assumption has been corrected by CUNY on numerous occasions, both verbally and in emails.

CUNY first corrected this error in an email dated March 9, 2023, and again in another email dated May 23, 2023. At no point after these corrective emails did the auditors request the correct data to ensure that their comparison was accurate and supported.

CUNY noted this concern for the third time in the post‐exit summary meeting held on May 18, 2023, and again on May 30, 2023.

CUNY is concerned about the auditor’s incorrect calculation of the 3‐Year Graduation Rate for College Discovery Program Students for the fall 2019 Cohort. The University, for the fifth time, is respectfully requesting that this information be removed from the report as it is misleading and presents false and inaccurate analytics of comparison on graduation rates. This error must be removed.

Additionally, the statement “Bronx Community College stood out as a clear outlier in improving graduation rates – by more than 12 percentage points over the general student population” lacks any supporting statistical analysis. Firstly, the definition of outlier is not aligned with that of OAREDA. OAREDA defines outliers based on the distribution of data. To identify an outlier, OAREDA separates the data into quartiles (each including ~25% of the sample) and then examines the range/spread of the data and where extraordinarily high or low values fall relative to that range. Technically, the formula used to identify an extremely high outlier is Q3 + 1.5*IQR, where Q3 represents the 75th percentile of the distribution, and IQR represents the “interquartile range” or the difference between Quartile 1 (25th percentile) and Quartile 3. In a data set of the differences between CD and non‐CD student graduation rates at the six community colleges (‐0.9, ‐0.3, 5.8, 5.8, 7.6, and 12.5), a high outlier would be greater than 16% points. Using OREADA’s calculation, Bronx Community College’s graduation rate for CD and non‐CD students is not an outlier.

Secondly, the statement made that within‐campus (CD v. non‐CD) and across‐campus (Bronx Community College v. others) comparisons using descriptive data attributes any difference to the CD program. Descriptive data is the simplest form of data analysis. In this case, it does not account for differences in populations based on their selection into the CD program and/or campuses that could also be related to the outcomes around retention and graduation. Further, it does not account for the Program’s eligibility criteria (CD students are less academically prepared and financially resourced than the general student population). In sum, the comparison made by auditors does not account for the scale of disparity within or across campuses.

3. Pg. 12 – Fiscal Accountability Plan

Regarding the statement, “Interestingly, the average cost per student in the College Discovery Program was approximately $2,205 in the 2021‐2022 Academic Year. 12 The two community Colleges with the highest average cost per student – LaGuardia ($3,223) and Queensborough ($2,442) – were also the community colleges for which the graduation rates of students in the program were slightly lower than they were for the students in the general population.” And, regarding footnote #12 – “to calculate the approximately average cost per College Discovery Program student in the 2021‐2022 Academic Year at each of the community colleges, the auditors used the budget and enrollment information provided by CUNY. The auditors divided the Academic Year 2021‐2022 budget for each community college by the Fall 2021 enrollment numbers.” CUNY’s response: CUNY has noted that the calculation performed by the auditors to determine the 3‐ year Graduation Rate for the CDP students for the fall 2019 Cohort needs to be revised. Further, the footnote determining the average cost per student states that the auditors took an entire academic year’s funding but only dispersed that to the fall student population and ignored the entire spring population who also benefits from the funding.

At the exit conference, CUNY inquired how the auditors determined the average cost per student, as this was not a value that CUNY had provided to the auditors. The auditors provided their worksheet for CUNY’s review. In an email dated May 24, 2023, CUNY wrote: “CUNY reviewed the auditor’s calculation of average cost per student. It appears that an overly simplified calculation was done – i.e., OCC took the full academic year funding and divided it by the number of spring students only (as noted in the Excel file under auditors note: “For purposes of our analysis, we used the number of students enrolled during the spring 2022 terms as it represents the number of students who stayed in the program through most of the year”). The total number of students enrolled in the spring term (1,384) is much lower than the total number of students enrolled in the fall term (1,754). Without consideration of the real and actual number of students, the values you have computed for the average cost per student in AY21‐22 and the cost per student at LaGuardia and Queensborough are both inaccurate and misleading. The report does not mention that these numbers are “guesstimate” but rather appear in the Exit Conference Summary as verified and precise, which it is not.” It now appears that the auditors took a full year’s funding, divided it with the fall enrollment number, and ignored the spring enrollment population. This calculation continues to be an inaccurate calculation of the average cost per student. Students receive funding in both the fall and spring semesters, which has been negated by the auditors.

Both these calculations need to be corrected and removed from the report.

CUNY also provides their detailed responses to the recommendations in the report.

Recommendations:

1. Develop best practices to be applied in all six CUNY community colleges, including aligned General Plans for each fiscal year, common academic goals (including retention and graduation goals), and practices to improve outcomes and the ability to measure success.CUNY’s response: CUNY had already identified an aligned improvement in reporting in 2019. During the pandemic, the Office of Special Programs (OSP) pressed forward with identifying what data should be captured that would be beneficial for the College staff as well as CUNY Central Office (OSP) to make informed decisions about the College Discovery Program (CDP). A General Plan was developed with a working committee comprised of OSP staff, CDP Directors, and other college personnel. The General Plan is a comprehensive document that outlines the programs’ operations, goals, budget, services, and students. The General Plan contains a description of both administrative and academic management for CDP. It covers all components defined in the CDP Guidelines, along with regular and special instruction (including summer sessions). The General Plan will aid in developing common academic goals and practices. CDP Directors submitted their General Plan to OSP on Friday, March 31, 2023.

2. Consider conducting its own surveys of College Discovery Program students on a periodic basis to help determine the effectiveness of the program and identify areas for improvement.CUNY’s response: CUNY agrees with the auditor’s statement in this report: “Surveys can serve as a vital resource that colleges can use to enhance their programs and operations. By regularly collecting and analyzing student feedback, a college can make more informed decisions to help improve student performance and achieve institutional goals. Students have a valuable perspective on the quality of their education.”CUNY currently engages its student population, which includes CDP students, as they are first and foremost a CUNY student, via different surveys. The topic of surveys was never a discussion point by the auditors with OSP, CDP Directors, or other College or Central Office personnel. The results from both OCC surveys conducted during this 2‐year audit were extremely encouraging to CUNY. The surveys targeted the CDP population in the audit scope period of September 2019 to June 2022 during a national state of emergency brought on by the COVID‐19 pandemic. During this unprecedented period for CUNY, the perseverance of CUNY staff, especially the CDP staff and services, was evident in the students’ responses to the survey questions as reported by the auditors (i.e., 95% of respondents found counseling services to be effective, 93% found tutoring services to be effective, and 91% indicated that the CDP program helped them to graduate). These results indicate that the services provided by CDP are effective and provide the pathway to help students graduate with a degree and be productive contributing members of society, even in the most difficult of times.

3. Develop mechanisms to more closely monitor the cumulative GPAs of College Discovery Program students that fall below 0 and develop plans for these students to obtain the support services they need to improve their GPAs and graduate on time.CUNY’s response: CUNY was given the opportunity to research the students that the auditors found exception to and concluded that none of the exceptions were justified or supported. CUNY communicated that none of the auditor’s exceptions were valid on May 24, 2023. Each student was placed on Satisfactory Academic Progress (SAP), at the appropriate time by the SAP committee of their respective College, which is a separate department and separate process completely outside ofCDP. The auditors did not conduct any interviews or have discussions with any SAP committee at the College level to determine what the process is to place a student on SAP, the appeal process, or how their academic plan is developed (this would include the number of credits to attempt and support services that are recommended or required).Furthermore, the auditors put together their own analysis on which they determined that a student did not reach good academic standing. CUNY informed the auditors in an email dated April 17, 2023, that their analysis was not correct¹:

• • “As an example, …, the first student on the Excel file. CUNY already explained on the March 24, 2023, response, that this student was enrolled in fall 21 but had a combination of “W” grades (i.e., no credits or grades were factored for that term). Therefore, the 997 cgpa, from spring 21, was a carryover cgpa to fall 21, and should only be counted once, not twice, as it currently is on the excel file. Please recheck the formula that was used to determine the number of terms for which a student’s gpa was less than 2.0, taking the above into account and the below information on SAP. We also acknowledge that the calculation included terms outside of the audit scope period.
  • Additionally, it is possible that a student can have less than 0 for more than two terms and still be in good academic standing (see below). This Excel file was set up to not record the terms where a student’s GPA was 2.0 or above, which is essential to fully understand the student’s academic progress.”

CUNY also had an extensive conversation with the auditors and came to understand that the auditors misinterpreted the basic requirements of SAP (i.e., SAP is determined based on both cumulative credits attempted and cumulative gpa), and thought that if a student did not reach 2.0 gpa each semester that they should be on SAP, regardless to the number of credits accumulated. CUNY provided another email on June 5, 2023, to clear up this misconception: “SAP is based on cumulative credits attempted (attempted credits are courses you registered for and did not drop before the last official withdrawal date) and cumulative GPA, not by how many credits are attempted each semester.”

SAP guidelines issued by the Board of Trustees (which the auditors footnoted in this report as #7) closely aligns with both the Federal Title IV and HESC guidelines, which all clearly explain that good academic standing is based on the number of cumulative credits earned and cumulative grade point average.

Supporting documentation provided by the Colleges for the students placed on SAP (at the correct and appropriate times) showed that CDP Directors worked closely with the students to ensure that the created academic plan would help them succeed in their classes and resume good academic standing. These students were informed of their required participation in the RISE program. This program explicitly targets all CDP students on probation or special probation and includes workshops and interventions designed to help students get back on track. These students continued to have access to both tutoring and counseling resources in CDP. CDP Directors went above and beyond to engage the student during this national emergency brought on by the COVID‐19 pandemic by texting, emailing, phone calls, and setting up meetings via in‐person and Zoom to discuss their progress and help them be successful. CDP Counselors and Peer Leaders worked with other departments that could help the student, for example, Disability Services, to obtain appropriate accommodations.

The auditors noted in their report, “College Discovery Program directors at four of the six community colleges stated that students should maintain a cumulative GPA of 2.0 per term to maintain their spot in the program. The directors also stated that if students’ cumulative GPA falls below 2.0, they are placed on academic probation. The directors at the remaining two community colleges (Borough of Manhattan and Queensborough Community Colleges) stated that they determine whether a student should be on academic probation based on the cumulative GPA and the number of credits the student has attempted to earn since enrolling in the community college.” The only mention of SAP was during interviews with CDP Directors, who may have been under the impression that the auditors understood SAP, as the auditors needed to inquire as to how one gets placed on SAP or any follow‐up questions related to SAP. The auditors did not request a copy of the grading policy or consider how students with non‐passing grades would factor into the cumulative GPA. CUNY also provided the Board of Trustees approved policy, and while it’s been footnoted in this report, it was not applied in their analysis.

4. Consider requiring the use of a formal agreement between a counselor and an academic probationary student, such as the College Discovery Program Academic Probation/High Risk contract, so that students are aware of the program’s expectation or them in the upcoming term and of the support services that are available.

CUNY’s response: CUNY already identified and included this requirement in the General Plan. The General Plan includes contracts for students on Academic Probation and contracts for first‐year, sophomore, junior, senior, and transfers.

5. More closely monitor the part‐time enrollment status of the College Discovery Program students so that: (1) required waivers are obtained; 2) the waivers include plans to help students stay on track to graduate (i.e., within seven terms); and (3) the students receive the counseling and tutoring needed to help them graduate on time (i.e., within seven terms).

CUNY’s response: During interviews, meetings, and emails as recently as May 24, 2023, CUNY explained that a waiver is optional for every term a student is in part‐time status. As explained but not acknowledged by the auditors, a waiver is filled out when a student is knowingly starting the semester in a PT status. As further explained to the auditors but disregarded, other factors require a student to be PT, for which a waiver will not be filled out:

• • As explained during the auditors’ interviews with CDP Directors, there are specific majors (eq. nursing, Vtech, radiology, etc.) whose degree maps have course load restrictions, which have been approved by Middle States² – CUNY’s accrediting body, and therefore are not held against the student.
  • The student intended to start the semester with a full‐time workload, but at some point, in the semester, withdrew from courses and assumed a part‐time status.
  • Those students can combine a Spring+Summer session or a Fall+Winter session with being a full‐ time status (i.e., each semester, when viewed as a stand‐alone semester, will be a part‐time status)

These valid exceptions were not considered by the auditors when they reviewed the data and decided that there were 329 exceptions³. Of the 32 exceptions that the auditors requested information for, CUNY provided the documents and support on April 17, 2023, and then again on May 24, 2023, that supported 26 students⁴ had one of the considerations listed above, which again was disregarded by the auditors. Of the 32 students CUNY was requested to support, CUNY only found an exception for seven students.

CUNY already identified the need to track part‐time status for all students and worked alongside CUNY’s OAREDA team to develop a centralized database that tracks student enrollment status by term and academic year. The database will go live for the CDP Directors and Office of Special Programs staff in FY24.

6. Ensure that the date a student becomes inactive in the College Discovery Program is accurately recorded in CUNYFirst in a timely manner.

CUNY’s response: CUNY agrees that it is vital to record the date of inactive status promptly. Updates to a student group’s status are done at the College level. The Office of Special Programs has been working with CUNY’s OAREDA team to develop a centralized database that will track program services, assess variation in program delivery across campus, and measure the differential impact of program services. A centralized database component will include accurate student coding across the programs. The database will go live for the CDP Directors and Office of Special Programs staff in FY24.

Addendum Footnotes

1. Footnote #9 in this report states “for purposes of this analysis, the auditors reviewed cumulative GPA information recorded in CUNYFirst for each student and identified those students who had GPAs below 2.0 for three or more consecutive terms”. CUNY corrected their inaccurate analysis at the time.
2. The Middle States Commission on Higher Education (MSCHE) is an institutional accreditor recognized by the United States Department of Education (USDE) that serves as an independent voice in higher education. MSCHE establishes equitable standards for accreditation with member institutions that advance the impact of accreditation in the ever‐changing landscape of higher education and promotes educational excellence through innovation across diverse institutions. As an institutional accreditor, MSCHE requires evaluation of all aspects of the institution holistically, including modality of instruction, and everything done in the name of the institution, including all academic programs and services, fall within their scope of accreditation. (https://www.msche.org/accreditation/)
3. In CUNY’s email response on May 24, 2023 to the 329 exceptions “the auditors identified a total of 329 (9%) students who had part‐time status for 2 or more terms, other than their graduating term” should have context to explain how only 32 students were requested by OCC to be vetted by CUNY, but not the other 297 students (as any of the above explanations may have been applicable, and would substantiate a PT status).
4. 9/26 students (35%) where in degrees that had course load restrictions, 1/26 was a student who was only enrolled PT for one semester, 1/26 student was previously communicated on April 17, 2023, to being PT once leaving the CDP program, 8/26 (30%) where registered with disability services – Office of Accessibility Services or Accessibility Resources Center, the remaining 6 had a combination of required non‐credit bearing remedial courses or started as FT and withdrew from courses.