The Pennsylvania Department of Labor and Industry (DLI) is responsible for the administration and operation of the state’s unemployment compensation program, which provides temporary income to replace lost wages for qualified workers. DLI employs over 500 people and has approximately 200 offices statewide to serve Pennsylvania’s 6.4 million workers and nearly 300,000 employers. Unemployment compensation (UC) claims are usually filed online or by telephone or mailed to a UC service center.
DLI had a legacy mainframe system for processing unemployment benefits that was over 40 years old. However, it became increasingly expensive to maintain and difficult to modify, with limited functionality for case management and integrating newer tools and technologies to enhance productivity.
In June 2006, DLI awarded IBM a fixed price contract totaling $109.9 million for the Unemployment Compensation Modernization System (UCMS), which would replace the antiquated mainframe system.
The initial contract with IBM called for more modern and efficient technology and business processes for (1) maintaining wage records, (2) processing employer taxes, and (3) claims processing, payment, and appeals, to be completed by February 2010. IBM won the UCMS contract after a three-year bidding process, claiming to be the only vendor with the type of proprietary databases capable of supporting a totally integrated computer system.
However, this project experienced significant delays and cost overruns, ultimately costing nearly $180 million, with much of the system never completed when the contract expired in September 2013. By that time, the project was 45 months behind schedule and $60 million over budget. Pennsylvania taxpayers had paid IBM nearly $170 million for what was supposed to be a comprehensive, integrated, and modern system that it never got. IBM’s contract was not renewed. In March 2017, Pennsylvania sued IBM for breach of contract, fraudulent and negligent misrepresentation, and charging taxpayers for services it did not provide. IBM said Pennsylvania’s claims had no merit and that it would fight the lawsuit. A spokesman for the company laid some of the blame for the project’s problems on the state, saying that there was responsibility on both sides for system performance and service delivery. How did all of this happen?
Phase 1 of UCMS (wage records) was implemented in May 2008. Phase 2, which included the employer tax portion of the system, went live in March 2011 but required additional work, which took years to fix. Phase 3 for benefit claims processing, payment, and appeals continued to lag behind with problems and ultimately never went live.
In 2012, DLI enlisted the Carnegie Mellon Software Engineering Insitute to conduct an independent assessment of the UCMS. The study was completed in July 2013, and recommended continuing work on remaining Phase 2 problems, but stopping work on Phase 3. Many of the problems it identified for Phase 3 could not be solved.
The Carnegie Mellon study found many flaws in the systems development process. IBM had extensive systems experience and technology knowledge but its proposal underestimated the project’s scope and complexity. DLI lacked sufficient staffing and experience for effective oversight and management of the contract and project. There was no formal delegation of roles and responsibilities for managing the project. No one at DLI was held accountable. DLI essentially relied on the contractor to self-manage.
UCMS was considered a large-scale software project due to its complexity, large number of information requirements and business rules, and its cost. DLI’s solicitation for vendor proposals for UCMS exhibited ambiguity in communicating all of these requirements, and also neglected to define and describe quantitative and qualitative performance measures and metrics for the proposed system.
A large-scale software-intensive system such as UCMS requires a rigorous and disciplined testing strategy, but this was not implemented. IBM decided to use DLI users to help develop test scripts. They provided the business expertise, but IBM did not use IT test experts on its end. User acceptance testing was initiated before completing system tests for Phase 2 and Phase 3. Rigorous testing came too late in the project. DLI did not specify a minimum of metrics for UCMS system performance so that there were no identifiable criteria and evidence for determining that Phase 2 and Phase 3 application releases were stable.
DLI staff had approved IBM’s representation of business system requirements without fully understanding what they were approving. IBM’s software development and testing program for this project lacked rigor. This resulted in a higher number of software defects than industry norms, software code that was excessively complex (which makes testing too difficult), and late discovery of missing business requirements.
The vast majority of the software defects were serious, and 50 percent were not discovered until the User Acceptance test, very late in the system development cycle. Without thorough and complete testing throughout the development process, there is no way to know how many of the total defects residing in software will be discovered as a system is being used. Carnegie Mellon also found that IBM had not performed a stress test to determine the performance limits of the UCMS system.
IBM’s software development plan was supposed to use industry and company standards and practices, but there was no ongoing discipline to execute these standards and practices during the project. DLI accepted Phase 2 prematurely for production in March 2011 with known defects impacting system performance, including software defects, unresolved data conversion issues, and problems with batch processing operations.
A project of this complexity and magnitude requires a high degree of continuity in knowledge throughout the system development cycle, but this was never achieved. During requirements determination, DLI didn’t have enough user subject matter experts to participate in joint application design (JAD; see Chapter 13) sessions with technical members of the project team. Thirty-six JAD subcontractors were prematurely removed from the project, leaving IBM with incomplete understanding of unemployment claims processing business requirements. System design and testing staff were not included in the JAD process, running counter to sound business practice. Including them was essential to ensure UC business requirements were defined in sufficient detail to be testable. DLI staff often approved JAD requirements documents and Detailed System Design documents under pressure to meet short deadlines for approval.
Ineffective project management and constant changes in the contractor’s workforce prevented transfer of essential knowledge for the entire project, a loss of “project memory.” Since the UCMS project’s start, 638 different contractors and staff members worked on the project. The majority of the project workforce spent less than one year on the project and 75 percent spent less than 2 years. All of these discontinuities and workforce churn most likely contributed to IBM’s schedule delays and inability to provide an accurate picture of the state of the project.
Work on Pennsylvania’s UC system continued without IBM. In 2013, the Pennsylvania Legislature passed Act 34, which created a Services Infrastructure Improvement Fund (SIIF) as a temporary supplemental funding source to improve UC services and systems. A total of $178.4 million was authorized and spent during calendar years 2013 through 2016. Even then the project stumbled. Pennsylvania Auditor General Eugene A. DePasquale initiated an audit in January 2017 to determine how the $178 million in SIIF funds were spent. The auditors found that DLI did not use proper accounting methods to record specific SIIF expenditures. DLI comingled unemployment compensation administrative funds from all sources, including federal funds for unemployment compensation administration and interest on unemployment compensation tax money as well as outlays from SIIF.
On a more positive note, there were noticeable improvements and efficiencies from 2013 through 2016 in services provided to UC claimants and in UC system infrastructure. For example, the percentage of first payments paid promptly increased from 81.6 percent to 93.4 percent. However, DLI was unable to show how exactly the SIIF expenditures contributed to these outcomes.
When SIIF funding was not reauthorized and supplemental funding ended in December 2016, DLI was forced to cut $57.5 million from its UC administrative budget for 2017, causing the immediate closure of three of the state’s eight UC service centers in December 2016 and the elimination of 521 positions. Customer service declined significantly with claimants not being able to get through on the phone lines and delays in processing claims.
Despite earlier setbacks, DLI is determined to complete the modernization of its unemployment compensation benefits delivery system. In June 2017, DLI signed a $35 million contract with Florida- based Geographic Solutions to create a system that enhances customer service, improves quality, is more operationally efficient, and is sustainable into the future. Geographic Solutions specializes in designing, developing, and maintaining web-based systems for the workforce development and unemployment insurance industries and has developed over 80 workforce systems for state and local agencies across the United States. Geographic Solutions was scheduled to begin work on the system on August 1, 2017 with a projected 18 to 24 months for completion.
In 2015, DLI had hired Chicago-based CSG Government Solutions for $6.1 million to assist with planning for and monitoring this project. CSG specializes in planning, managing, and supporting complex projects that modernize the information technology and business processes of large government programs. CSG analyzed existing systems and workflows, developed the project strategy and technology roadmap, and gathered business and technical requirements to develop an RFP. CSG also established a full-service Project Management Office to monitor project progress, and is providing technical oversight, UC subject matter expertise, requirements management, and testing support throughout the system modernization. Once the new system has been fully implemented, cost savings from benefit modernization are estimated to range from 5 to 10 percent of total UC administrative costs.
Source: Laudon Kenneth C., Laudon Jane Price (2020), Management Information Systems: Managing the Digital Firm, Pearson; 16th edition.
It’s an amazing article designed for all the web users;
they will take benefit from it I am sure.