Audit of the Human Resources Administration/Department of Social Services’ Career Pathways Program

AUDIT IMPACT

Summary of Findings

The audit found that the Human Resources Administration (HRA) appropriately reimbursed Career Pathways providers for achieving aggregate milestones and interacted with providers in monitoring the services provided to clients. However, the auditors also found that HRA’s administration of the Career Pathways program could be improved.

While the Career Pathways program was negatively impacted by COVID-19, the audit found that HRA did not respond to this challenge as well as it could have by (1) better controlling costs as the number of participants in the program dropped and (2) more quickly opening the program to new applicants on a voluntary basis. The audit found that per person expenditures rose from $870 in FY2020 to $7,544 in FY2022.

The actual retention rate for clients placed in jobs by career services vendors was below the 80% target set by HRA for the program at the 180-day mark, and the overall performance goals HRA set for the program did not extend beyond 180 days from job placement.  This is the case even though the Career Pathways contracts provide for milestone payments to the contractors for their clients retaining employment at the 180-day and one-year marks. The audit recommends that HRA establish a target retention rate for the one-year mark.

The audit also included a client satisfaction survey of participants. Although the response rate was low, only 42% of respondents expressed overall satisfaction with the program.

Intended Benefits

The audit made several recommendations designed to help HRA enhance the effectiveness of the employment services its Career Pathways contractors provide.

INTRODUCTION

Background

HRA, which was integrated with the Department of Homeless Services in 2017 under the management structure of the Department of Social Services, serves over three million New Yorkers annually through programs that help lower-income individuals and families, such as cash and food assistance programs and Medicaid. HRA also promotes work opportunities through various employment training and placement programs.

HRA’s Career Services program offers clients who are receiving cash assistance the opportunity to increase their job skills and build careers. Career Services works closely with clients to find opportunities that match their skills, needs, and career goals, and help them access a broad range of educational, training, and job placement services.

Career Pathways, which is the focus of this audit, is one of the major components of the Career Services program.^[1] Three individual programs—CareerCompass, CareerAdvance, and YouthPathways—comprise Career Pathways. The CareerCompass program provides assessment and coordination services for cash-assistance clients ages 25 and older. Service providers work with job-ready adult clients to assess their skills and experience and create Individual Service Plans. Clients then generally move on to the CareerAdvance program, which provides educational opportunities, vocational training, and job placement and retention services. The YouthPathways program provides similar assessment, training, and job placement and retention services for clients ages 18 to 24.

During Fiscal Years 2020 through 2022 (July 1, 2019 through June 30, 2022), HRA awarded 34 contracts to 10 providers to provide career services to cash-assistance clients through Career Pathways. According to HRA, it paid $146.3 million to its Career Pathways contractors for the services provided during these three fiscal years.

HRA uses the New York City Work, Accountability and You (NYCWAY) system for several purposes. The system tracks eligibility and employment information about its cash-assistance clients; manages client appointments, assignments, and exemptions; and facilitates supportive services, such as childcare. The agency also uses the Daily Timekeeping System to track and monitor client attendance in Career Services programs.  In addition, HRA uses the Payment and Claiming System (PaCS) to track job placement and retention milestones for provider reimbursement purposes.  Further, HRA uses the Health and Human Services Accelerator to review providers’ monthly invoices and reimburse them for their expenses.

On March 16, 2020, as the COVID-19 pandemic began affecting many government services, the New York State Office of Temporary Disability and Assistance (OTDA) suspended the mandatory requirement that cash-assistance clients must participate in employment-related activities. HRA subsequently stopped referring new clients to Career Pathways providers, though clients who were already enrolled in the program could continue to receive career services on a voluntary basis.

According to HRA, 50,758 clients received career services in the Career Pathways program during FY2020. In FY2022, 7,324 clients received career services in the program.  HRA officials stated that the agency planned to resume the requirement that cash-assistance clients participate in the program in September 2023.

As a result of the change from mandatory to voluntary participation and the expenses incurred because of COVID-19, HRA modified the contract terms for provider payments in April 2020.   This coincided with a renewal of the contracts through the end of March 2023. Prior to the modification, the payment structure for the CareerAdvance program (the largest Career Pathways program) was as follows: 63% of the total payment amount related to line-item expenditures and 37% of the amount related to providers meeting individual and aggregate performance milestones.^[2]   Examples of line-item expenditures eligible for reimbursement included salaries, rent, utilities, professional services, and equipment. Once modified, reimbursement changed from a hybrid structure (line-item and performance-based reimbursement) to a 100% line-item structure.

Objective

The objective of this audit was to determine whether HRA effectively oversees the Career Pathways program.

Discussion of Audit Results with HRA

The matters covered in this report were discussed with HRA officials during and at the conclusion of this audit. An Exit Conference Summary was sent to HRA and discussed with HRA officials at an exit conference held on May 18, 2023. On June 5, 2023, we submitted a Draft Report to HRA with a request for written comments. We received a written response from HRA on June 20, 2023.

In its response, HRA thanked “NYC Comptroller for their partnership as we continue our critical mission.” HRA agreed with two of the four audit recommendations and disagreed with two.

HRA’s written response has been fully considered and, where relevant, changes and comments have been added to the report.

The full text of HRA’s response is included as an addendum to this report.

DETAILED FINDINGS

The audit found that HRA appropriately reimbursed Career Pathways providers for achieving aggregate milestones. The agency also generally interacted with providers and monitored services being provided to clients, as called for in the contract. However, the auditors also found that HRA’s administration of the Career Pathways program could be improved.

Inevitably, the Career Pathways program was impacted by COVID-19; however, HRA did not respond to this challenge as well as it could have in two important ways: (1) as the number of participants in the program dropped, HRA should have made greater efforts to control costs;  and (2) HRA was slow to open the program to new applicants on a voluntary basis. Because of this, per person expenditures rose from $870 in FY2020 to $7,544 in FY2022.

In March 2020, participation in the program became voluntary and the number of participants plummeted. In response, HRA revised the contracts with providers to eliminate incentive milestone payments (making them 100% cost based) but it took few steps to reduce costs in line with reduced service levels. HRA, in fact, increased allowable costs to cover certain COVID-related expenses.

In addition, when the program was made voluntary, it only included existing clients who had already enrolled in the program. New applicants for cash assistance were not given access to career services for 15 months, between March 2020 and June 2021.

Despite the very high level of costs incurred, the retention rate for clients placed in jobs by career services vendors was well below the 80% target set by HRA for the program at the 180-day mark, even once the most challenging period of the pandemic had passed, and the performance goals set for the program by HRA did not extend beyond 180 days from job placement.

Although the Career Pathways contracts provide for milestone payments to the contractors for their clients retaining employment at the 180-day and one-year marks, HRA has not set an overall program goal for job retention at the one-year mark. The audit recommends that HRA establish an overall performance goal for the program at the one-year mark, consistent with its job retention payments to contractors.

The following sections of this summary discuss these findings in further detail.

Amounts Paid to Contractors Increased While Client Participation Decreased

While the number of clients participating in the Career Pathways program dropped from 50,758 in FY2020 to 7,324 in FY2022, the amount of money paid to providers increased from $44 million to $55 million per year over the same period.

Before the COVID-19 pandemic, participation in employment-related activities was mandatory for cash assistance clients. On March 16, 2020, the New York State OTDA suspended this requirement due to the public health crisis.

While the number of clients participating in the program dropped significantly from FY2020 to FY2022, there was no corresponding drop in the amount paid to providers during the period.  As shown in Table I below, HRA spent a substantial amount of money on career services in FYs 2021 and 2022, even though the program was entirely voluntary during those years and, in FY2021, was only available to those who had been clients as of March 2020.

Table I

As shown in Table I, the number of clients participating in the program dropped by 86%, but the amounts paid to providers increased during that period, going from $44 million to $55 million (a 25% increase). On a per client basis, HRA paid more than eight times in FY2022 what it paid in FY2020.

As a partial explanation for the disparity between the number of clients served and the amounts paid to providers, HRA officials stated that providers were allowed to submit separate invoices during the pandemic for COVID-related expenses (in addition to the amounts indicated in their contracts). That may account for certain increases, but it does not explain why overall vendor expenditures did not fall as the number of clients served also fell, dramatically.

As a result of the modification in the payment structure (which went from a hybrid structure based on both line-items and performance to a structure based entirely on line-items), providers were reimbursed solely on the basis of costs actually incurred to provide services to HRA clients.  A review of several contract budgets shows that personal service expenditures (salaries and fringe benefits) accounted for about 55% of the contract budgets. Given the massive decrease in the number of clients served, it is reasonable to expect this expenditure, along with various other expenditures, to have similarly fallen.   As stated in Comptroller’s Directive #2, Cost Reimbursable Contract Payment Request Audits, costs reimbursed under a contract must be reasonable in that they are necessary to carry out the contract.

HRA officials stressed that steps were taken to contain costs during the COVID emergency. However, officials noted that opportunities were somewhat limited in that certain costs associated with the contract were fixed (e.g., rent, utilities, and equipment) and were payable regardless of the number of clients served. Additionally, officials stated that the contractors provided extensive assistance to their clients during the COVID emergency to ensure that they received the support services they needed.

The auditors nonetheless question the large increase in payments to the contractors over this three-year period. The other than personal services costs (including fixed costs) in the contract represent only about 45% of the overall costs paid to the contractors.   As per Directive #2, as noted above, expenditures should reasonably reflect the payments needed to carry out a contract. Since career services were being provided to significantly fewer clients, one would have expected that expenditures (especially personal services expenditures) would also have decreased substantially during this period. On the contrary, during this time, total expenditures actually increased substantially.

HRA Did Not Invite New Clients to Voluntarily Participate Until June 2021

Although OTDA suspended the requirement that clients must participate in Career Pathways in March 2020, existing clients were still allowed to receive services on a voluntary basis. However, HRA did not invite new clients to Career Pathways until June 28, 2021—over 15 months after OTDA lifted the participation requirement. In this time, the agency encouraged existing clients to continue with the program (and even offered services to the family members and friends of existing clients).

HRA officials stated that the voluntary process for new clients needed to be approved by OTDA, but provided no evidence that this was required. Officials provided no evidence that they asked OTDA for approval before sending formal invitation letters on June 28, 2021. Moreover, in its announcement of the suspension, OTDA did not exclude new clients from participating in the program voluntarily.

HRA officials also informed auditors that the invitation letters were not actually sent to all new clients due to various mailing issues (including a shortage of envelopes). Nonetheless, the agency had many alternative ways to communicate with clients, such as email.  As noted below, HRA provided the auditors with the emails of many program participants.

New clients who were eligible to participate in the Career Pathways program were not given the opportunity for up to 15 months. Had HRA included new clients in the program during this time, per person cost would potentially have been reduced, and useful services may have been provided to a new set of clients.

HRA Did Not Meet Its Retention Goals

HRA set a goal in the Mayor’s Management Report (MMR) that at least 80% of clients that its providers helped find jobs would remain employed (or at least stay off cash assistance) for at least 180 days after job placement.  During the period covering FYs 2020 through 2022, the percentage of people still employed at the 180-day mark ranged from 51% to 69%, falling well short of HRA’s goal. This shortfall occurred even though program headcount decreased and costs per participant increased, as detailed above.

The following tables show the Career Pathways program’s results for FYs 2020 through 2022, as provided by HRA. As shown, Career Pathways providers did not meet HRA’s goal of 80% continued employment within 180 days of job placement.

As shown in Table II, below, only 69% of clients were employed or stayed off cash assistance for 180 days in FY2020; that rate dropped to 51% in FY2022. In addition, the percentage of clients placed in jobs declined from 29% in FY2020 to 13% in FY2022.

Table II

On the positive side, the percentage of those who stayed employed or off cash assistance for one year improved slightly from 60% in FY2020 to 64% in FY2022. The percentage of those who returned to cash assistance within one year of job placement also improved slightly, declining from 40% to 36%.

When the performance of the Career Pathways program is compared to other HRA employment programs, the results are somewhat similar. Table III, below, shows the Career Pathways program’s results for FYs 2020 through 2022, as compared to the results for other HRA employment programs.

Table III

* These programs include all other HRA employment programs except for the Subsidized Employment Program, which was excluded by HRA for these two MMR indicators.

In FYs 2020 and 2021, clients in Career Pathways fared slightly worse (within four percentage points) than clients in other HRA employment programs, in terms of job retention, or at least not relying on cash assistance, at the 180-day mark. In FY2022, the percentage of clients in Career Pathways who retained their jobs or stayed off cash assistance was 19 percentage points less than it was for other HRA employment program clients at the 180-day mark, but five percentage points higher at the one-year mark.

At the exit conference, HRA officials stressed that the clients typically served by this program are hard to place even when the local economy is strong and were even more difficult to place during the COVID emergency. They also stated that the restaurant, hospitality, and retail sales sectors were especially hard hit by the emergency, and that these are the industries in which many Career Pathways clients are placed during normal times.

The auditors accept the arguments raised by HRA in relation to the pandemic and acknowledge the general challenges in providing employment services to often hard-to-place clients. These challenges obviously impacted both job placements and retention and the ability to meet goals set for the 180-day period.

Nonetheless, the auditors recommend that the Career Pathways program establish a longer-term job retention goal to supplement the goal that it already has in place at the 180-day mark.  This is consistent with the contract requirements and with the MMR. Both the MMR and Career Pathways contracts identify the one-year retention mark as a key performance indicator for milestone payments to its contractors, but the agency has not established a corresponding retention goal for the program.

A longer-term retention goal is important for HRA to assess the effectiveness of its Career Pathways program and, when there are shortcomings, to then consider possible operational improvements. For that reason, the agency should establish a one-year retention goal to supplement its 180-day goal.

Survey Respondents Had Mixed Views of Program

To determine how program participants viewed the services they received from the Career Pathways program, auditors conducted a client satisfaction survey.

On December 22, 2022, and again on February 9, 2023, auditors sent survey questionnaire emails to over 3,000 clients and received 84 responses. Responses received were anonymous. The survey contained 23 questions, a number of which asked respondents to rate their levels of satisfaction with services provided as either excellent, good, fair, or poor.

The audit acknowledges that the participants who responded to the survey represent a very small percentage of the people who participated in the program. Consequently, the degree to which their responses reflect the general opinions of the Career Pathways program participants as a whole is unknown. A survey with a higher response level would allow HRA to gather feedback from participants and help the agency identify overall satisfaction levels and potential shortcomings in the services provided.

Overall, only 42% of respondents had a favorable opinion of the quality of services received in the program. (The full survey results for questions in which respondents were asked to indicate whether they received certain services and to rate their levels of satisfaction with the services received are included as an appendix to the report.)

The key points of the survey results are as follows:

• Of the 84 participants who responded to the survey, 35 (42%) participants stated that the overall quality of service offered by Career Pathways providers was good or excellent, 18 (21%) stated that overall quality was fair, and 31 (37%) stated that overall quality was poor or unsatisfactory.
• Of the 48 participants who stated that the provider had assessed their skills and abilities, 41 (85%) were satisfied and 7 (15%) were dissatisfied with the assessment services.
• Of the 43 participants who stated that they had received training services from the providers, 33 (77%) were satisfied and 10 (23%) were dissatisfied with the training provided.
• Of the 36 participants who stated that they had received job search assistance from the providers, 31 (86%) were satisfied and 5 (14%) were dissatisfied with the assistance provided.
• Of the 33 participants who stated that they had received guidance from the providers on how to retain a job, 32 (97%) stated that the guidance was very helpful or somewhat helpful.

Most respondents who were asked to rate specific services from providers had a favorable view of them. However, many respondents indicated that they did not receive one or more services. The reasons for this vary; for example, a client may have exited the program before a certain service would have been provided. Of the 84 respondents:

• 36 (43%) stated that they did not receive skills assessments;
• 41 (49%) stated that they did not receive training services; and
• 48 (57%) stated that they did not receive assistance in finding a job.

The clients who responded to the survey questionnaire also provided the following suggestions for improving the Career Pathways program:

• the contractors should offer services that are better tailored to individuals’ interests and skill levels;
• the contractors should offer more training classes that help clients start new careers;
• the contractors should provide better assistance in helping clients obtain permanent jobs; and
• the contractors should expand the types of positions that they help clients obtain, rather than primarily focusing on maintenance, nurse aide, and security jobs.

HRA said that they have not conducted surveys of program participants, though some of its contractors have occasionally done so. HRA did not request or view the results of contractor surveys.

In its response, HRA expressed concerns about the validity and representativeness of the client survey, pointing to the low response rate, and objecting to the reported results of the survey.

However, the results are provided in full above, and the impact of the low response rate is already noted above.  Moreover, the auditors recommended that HRA conduct its own client satisfaction surveys.  HRA agreed with this recommendation.

RECOMMENDATIONS

To address the abovementioned findings, the auditors propose that:

1. In the future, HRA should ensure that opportunities for all eligible clients to receive employment services are maximized.

HRA Response: HRA disagreed with this recommendation, stating that the agency “did the best it could to continue ensuring that opportunities for all eligible clients to receive employment services were maximized,” and that it continued to provide support services to clients while ensuring their participation in a number of activities, including job fairs, educational services, job placement and retention services, and financial planning and counseling.

Auditor Comment: The recommendation relates to the finding that HRA did not open the program to many eligible clients for a period of 15 months. This occurred in a context in which the number of clients served was drastically reduced and the cost of the program continued to grow.  This can hardly represent maximization.     

2. HRA should develop and formally document strategies for regulating and containing contract costs when emergencies impacting the performance of deliverables occur during the course of a contract.

HRA Response: HRA disagreed with this recommendation, stating that it “did develop strategies for regulating and containing contract costs” and further claimed that it generated over $20.8 million in program-related savings.

Auditor Comment:  The auditors found no evidence of cost containment strategies being effectively implemented for the Career Pathways program. Despite the fact that the number of Career Pathways clients dropped dramatically, payments to Career Pathways vendors grew by $11.1 million between FY2020 and FY2022.

3. In conformance with the contract, HRA should establish performance targets for clients placed in jobs who stay employed, or at least do not return to cash assistance, within one year of employment.

HRA Response: HRA agreed with this recommendation.

4. HRA should endeavor to assess client satisfaction on a larger scale, review contractors’ client satisfaction survey results, and consider the opinions of clients in making changes to the program.

HRA Response: HRA agreed with this recommendation.

Recommendations Follow-up

Follow-up will be conducted periodically to determine the implementation status of each recommendation contained in this report. Agency reported status updates are included in the Audit Recommendations Tracker available here: https://comptroller.nyc.gov/services/for-the-public/audit/audit-recommendations-tracker/

SCOPE AND METHODOLOGY

We conducted this performance audit in accordance with Generally Accepted Government Auditing Standards (GAGAS). GAGAS requires that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objective. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions within the context of our audit objective. This audit was conducted in accordance with the audit responsibilities of the City Comptroller as set forth in Chapter 5, §93, of the New York City Charter.

The scope period for this audit was Fiscal Years 2020 through 2022.

To obtain an understanding of the career services offered to cash-assistance clients, the auditors interviewed the Deputy Commissioner of Career Services; the Assistant Deputy Commissioner for Provider, Client and Business Services; the Assistant Deputy Commissioner for Education and Youth Services; and the Assistant Deputy Commissioner for Program Development, System Innovation and Engagement. To understand how HRA tracks provider performance, the employment-related activities of its cash-assistance clients, and its reimbursements to providers for meeting job placement and retention milestones, auditors attended demonstration sessions for the different computer systems used by the Career Pathways program.  To understand how HRA evaluates provider performance and determines whether the Career Pathways program is meeting the intended outcomes for the program participants, auditors held several discussions with the Executive Deputy Commissioner of the Employment and Support Services Administration, and the Assistant Deputy Commissioner of the Office of Planning and Performance Management.

To better understand and to assess the adequacy of HRA’s oversight of Career Pathways providers, auditors reviewed and used the following criteria:

• New York State Office of Temporary and Disability Assistance’s General Information System Messages—Transmittal 20 TA/DC016, dated March 16, 2020; Transmittal 20 TA/DC023, dated April 1, 2020; and Transmittal 20 TA/DC048, dated May 14, 2020;
• HRA Policy Bulletin #2021-014: Career Pathways Voluntary Engagement, dated March 31, 2021;
• HRA’s Manual for CareerCompass, CareerAdvance, and YouthPathways, issued on April 9, 2018;
• HRA Policy Bulletins—#2019-017(updated), dated April 19, 2019; #2019-23, dated June 29, 2019; #2019-24, dated September 26, 2019;and #2019-016, dated December 4, 2019;
• New Employment Services PaCS Web Milestones (PaCS rules), issued on January 10, 2018;
• Comptroller’s Directive #2, Cost Reimbursable Contract Payment Request Audits; and
• The New York City Mayor’s Management Report.

To determine whether HRA effectively oversaw its Career Pathways providers’ provision of career services to cash-assistance recipients, the auditors randomly selected for review 12 contracts of the 32 Career Pathways contracts that were active in both Fiscal Years 2020 and 2021.^[3]   Auditors reviewed HRA’s bi-weekly and quarterly provider meeting notes, site visit reports, annual review reports, and provider performance evaluations pertaining to these 12 contracts for the pre-pandemic period (from July 2019 through February 2020). Additionally, auditors reviewed documentation related to HRA’s monitoring efforts during the pandemic (from March 2020 to June 2021), such as the agency’s meeting notes for virtual network meetings and regular check-in calls with providers, and its observations and assessments of virtual webinars and online workshops that providers offered to cash-assistance clients.

To determine whether HRA appropriately reimbursed its Career Pathways providers for their achievements of performance-based aggregate milestones, the auditors reviewed payment records relating to the milestones achieved during the pre-pandemic period (from July 2019 through March 2020).   The auditors stratified a total of 234 payments by milestone type, then selected for review all of the 171 payments issued for the aggregate assessment milestone and randomly selected for review 28 of the 63 payments issued for the aggregate Strategic Program Operation Target (SPOT) milestone. Auditors then reviewed the monthly career statistical reports and the predetermined payment schedules specified in the career service contracts with providers to determine whether the payments were appropriate.

To assess clients’ satisfaction with the career services they received through the Career Pathways program, auditors conducted client satisfaction surveys. From a population of 50,758 clients, auditors randomly selected 5,000 clients for the satisfaction surveys. On December 22, 2022,  and again on February 9, 2023,  auditors successfully delivered 3,364 survey questionnaire emails to the 3,655 clients for whom HRA had email addresses.  Auditors received a total of 84 responses.

Although the results of the audit tests that involved samples were not projectable to their respective populations, these results, together with the results of other audit procedures and tests, provided a reasonable basis to assess the effectiveness of HRA’s oversight of the Career Pathways program.

APPENDIX I

The following is a breakdown of Career Pathways program client survey questions/topics and results:

Overall quality of services received from Career Pathways providers

Assessment services

Training services

Job search assistance services

Employment services

Voluntary Engagement

Addendum

June 20, 2023

Ms. Maura Hayes-Chaffe
Office of the City Comptroller
1 Center Street, Room 1100
New York, NY 10007

Re: Agency Response to the Draft Report on the Human Resources
Administration/Department of Social Services’ Career Pathways Program ME22-069A

Dear Ms. Hayes-Chaffe,

We have received the draft report for the New York City Comptroller’s Audit of the Human Resources Administration/Department of Social Services’ Career Pathways Program (MD22- 105S). Please find enclosed our Agency response, which identifies the actions taken in accordance with the plan to address the recommendations noted in the report, or the reason why the Agency disagrees with the recommendations.

We would first like to provide context which is critical for understanding this report. The COVID-19 Public Health Emergency (PHE) had substantial impact on the Career Pathways Program operations and the New York City (NYC) economy more broadly. For example, mandatory employment referrals were suspended on March 16, 2020, in response to the PHE. Many of the job sectors, which had previously employed Career Pathways participants, such as retail, food industry, and home health care, experienced massive layoffs during the pandemic and did very little hiring. NYC unemployment rate was one of the highest in the country. Many companies went out of business entirely and while NYC’s economy has been improving, these sectors continue to lag behind and the unemployment rate, especially for New Yorkers of color and the clients HRA’s programs serve, remains disproportionally high.

This context is also crucial for understanding the increase in providers’ per-participant costs. DSS had a purposeful and directed plan to keep our providers solvent even though the Career Services Program was operating at a very different level due to the pandemic. Based on the March 18, 2020, guidance provided by the Mayor’s Office of Contract Services (MOCS), the Mayor declared a State of Emergency that created flexibility in contract performance while maintaining organizational cash flow and financial security. This meant that the City would reimburse providers for contract expenses even if usual service delivery was suspended or modified, as long as this was done in consultation with the Agency. In addition, the guidance

noted that the City would reimburse providers for additional personnel expenditures related to overtime and temporary staff to address shortages if staff could not come to work.

Furthermore, while the number of participants dramatically declined during the public health emergency, many costs, such as rent, utility, insurance, and salaries, were fixed. HRA’s Career Services Program quickly adapted to the enhanced new needs of our participants, which could not have been anticipated. The Career Services Program worked with the providers to pivot, offering our clients more supportive services, supplying them with laptops and mi-fi devices, as well as conducting remote job-readiness webinars (which necessitated converting all in-person trainings and workshops to the online format). To ensure food security for some of the most vulnerable New Yorkers, Career Services providers also offered community resource mapping of local resources and their utilization and shared lists of locations that distributed free boxes of fresh food and groceries. Further complicating cost control was the worldwide uncertainty around the duration of the pandemic and when operations would normalize. Requesting providers give up leases or layoff staff would impact the resumption of services as well as provide even greater harm to the larger NYC economy.

We continue to have significant concerns with the validity and representativeness of the client survey included in this report. While we applaud the NYC Comptroller’s effort to seek client feedback directly, we believe the low survey response rate and resulting small sample size prevent the survey from yielding meaningful results. The survey was sent to 3,364 clients with email addresses but resulted in just 84 responses (response rate 2%). This small sample

size limits the generalizability of the survey responses, undermining the report’s broader claims, such as “only 42% of respondents had a favorable opinion of the quality of services received in the program.” Furthermore, when reviewing the detailed responses, we found that many of those clients marking the overall quality of services as “poor” did not appear to receive any service or did not at all rate a specific service. By contrast, the satisfaction ratings of participants who did receive a service are much higher (according to the Comptroller’s Draft Report, 85% were satisfied with assessment services, 77% were satisfied with provided training, 86% were satisfied with job search assistance and an impressive 97% were satisfied or somewhat satisfied with guidance received from the provider as it relates to job retention). The focus on the low response rate in the first section, while not stressing the high satisfaction rate of those who did receive services, seems misleading. Nevertheless, we do not disagree with the Comptroller’s recommendation to regularly conduct our own customer satisfaction surveys and will take that under consideration.

We are confident that our progress in returning to some normalcy after the worldwide pandemic, and our response to this review, demonstrate the Agency’s commitment to continually improving our operations. To that end, the Draft Report indicates that HRA will resume mandatory engagement in April 2023. Please be aware that mandatory engagement is currently scheduled to begin in September 2023. We respectfully request that this be updated on the Final Report accordingly.

Should you have any questions regarding the enclosed, please contact Victoria Arzu, Assistant Director of the DSS External Audit Facilitation at 929-221-7067. We thank NYC Comptroller for their partnership as we continue our critical mission.

Yours sincerely,

Christine Maloney
Deputy Commissioner, Office of Audit & Quality Assurance Services

Enclosures

---

Endnotes

[1] Other Career Services programs include Business Link; the City University of New York Educate, Develop, Graduate, and Empower program (CUNY EDGE); Internship Placement Services; and the Work Progress Program.

[2] The payment structure for the CareerCompass program was 72% for line-item expenditures and 28% for performance.   The payment structure for the YouthPathways program was 68% for line-item expenditures and 32% for performance.

[3] Two contracts were in effect in Fiscal Year 2020 but not in Fiscal Year 2021.