Content written by Michael Acton, Chief Technology Officer at Array, Joe Amsden, Senior Solutions Architect at Array, and Brad Barker, Client Executive at Array.
On behalf of the United States Air Force (USAF), our multi-company team—comprised of staff from ARRAY, NTT Data, TSRI, and Datum—delivered a successful modernization of a COBOL-based system running on aged mainframes to a Java-based system running on x86 Red Hat Enterprise Linux (RHEL).
The goals were to introduce agility, reduce costs, modernize the technical stack, and enhance security by leveraging Amazon Web Services (AWS). We did this using automated code conversion tools and techniques while maintaining all functionality, screens, and reports.
The modernization target was AWS GovCloud (US) in order to address the USAF’s stringent security and compliance requirements such as U.S. citizenship.
The system we modernized is an ACAT III Major Defense Acquisition Program and mission-critical system used by more than 18,000 users at over 260 global locations. It provides daily supply chain and equipment support for USAF missions and is the accountable data system for over $30 billion in inventory.
AWS provides the system’s capabilities for reliability, scalability, and availability with features such as Availability Zones (AZs), Auto Scaling Groups, and Amazon Relational Database Service (Amazon RDS).
In this post, we describe the objectives, approach, solution, lessons, and customer benefits realized from this experience. ARRAY Information Technology is an AWS Partner Network (APN) Consulting Partner.
Context and Objectives
A major component of the system is 54 years old, written in COBOL, and provides retail-level business logic. The component runs on mainframes that have proven to be extremely difficult to change and manage, and the USAF needed to modernize the component to drive down operating costs and move to an open platform, while retaining all functionality.
In the past, attempts made to modernize the component failed due to the massive size and complexity of the task. In fact, modernizing the component was regarded as such a difficult task that it was highlighted in the 2003 book Modernizing Legacy Systems: Software Technologies, Engineering Processes, and Business Practices.
After 54 years of operations, maintenance, and extensions, the component’s code had become poorly documented. The technical design of the existing system, which was needed to support the modernization effort, had to be derived from the existing system and code.
Key characteristics of this component included:
- Annual operating costs over $30 million, largely attributable to mainframe hosting and maintenance costs.
- 1.3 million lines of COBOL Source Lines of Code (SLOC).
- A data management system, comprised of 1.2 million lines of COBOL code, supporting approximately 500,000 transactions per day.
- Few remaining subject matter experts (SMEs).
- Difficulty finding qualified, affordable COBOL programmers.
The component needed to be modernized and migrated to an affordable open system and hosting environment, with no down-time, data loss, functionality loss, performance loss, and minimal mission risk. The USAF had never done anything like this.
Modernization Approach and Solution
Our team developed a trusted relationship with the USAF’s Project Management Office over several years by delivering solutions that transformed and improved their supply mission. In 2015, the USAF contracted our team to modernize the component to become an integrated and sustainable part of the overall system.
We started the project by identifying and evaluating solution options:
- A total manual rewrite and re-architecting solution failed to meet the program’s time constraints, had historically low success rates (high risk), and would have been too costly.
- A replacement solution was not selected because the USAF needed to retain all the current business rules.
- A COBOL emulator re-host solution was a stopgap measure that failed to reach the J2EE/RHEL/SQL DB architectural future state requirement.
This analysis led to our decision to use a COBOL-to-Java code automated refactoring solution. This option would take a low-risk, incremental approach and apply a blended agile/traditional methodology and tools to ensure rapid, high-quality software delivery.
Component Modernization Phases
Once the COBOL-to-Java code automated refactoring solution was selected, a three-phase approach emerged to meet the entirety of the USAF’s requirements and cost, schedule, and risk constraints.
- Phase 1 (18 months): COBOL-to-Java code automated refactoring to x86/RHEL platform
This was the most complex and risky phase, as we automatically refactored COBOL code from mainframes to Java code on a virtualized x86 RHEL platform while not losing any functionality or performance. The resulting Java code contained design remnants of COBOL, and development and test environments were moved to the AWS Cloud. - Phase 2 (12 months): Code advanced refactoring to remove COBOL design overtones
We refactored the Java codebase even more to remove residual COBOL remnants, overtones, and design elements to improve maintainability. - Phase 3 (3 months): Infrastructure moved to AWS GovCloud (US)
We moved all remaining environments to AWS GovCloud (US) including staging and production. AWS GovCloud (US) allowed us to meet the many cyberthreat security requirements for the USAF.
Figure 1 shows our three-phrase modernization approach. The two Java logos illustrate the different Java phases. At the end of Phase 1, the Java program is “dirty” with remaining COBOL coding practices. At the end of Phase 2, the Java program is “clean” without COBOL remnants.
Figure 1 – ARRAY’s three-phase modernization approach met the USAF’s stringent requirements.
Phase 1: Automated Refactoring of COBOL to Java on x86
The modernization of the component involved a conversion of 1,260,679 lines of COBOL code and 10,078 lines of C code to Java to maintain current application capabilities, Graphical User Interface (GUI), and performance while migrating to an affordable and sustainable x86 RHEL platform.
The component’s COBOL online and batch applications were automatically refactored to run on a JEE platform using Java object-oriented software layers (data access, presentation, business logic) and patterns to enable migration to a standard x86 architecture.
The modernized Java software reused identifier names from the original COBOL application, allowing the component SMEs to read and understand the new Java code and routines more easily.
Rather than simply transliterating source COBOL code to target Java code, the tool executes a mature automated conversion and refactoring process by first constructing a comprehensive Intermediate Object Model of the legacy system in an intermediate translation language.
Once modeled within the tool engine, SMEs employ an iterative process of applying rules and tuning to output the transformed code into the target Java language.
We completed the work of reintegrating items like schedulers and other utilities, testing the initial construction iterations of the code, and providing defects. In response to each construction delivery evaluation, we adapted the transformation rules and regenerated improved construction iterations of the code according to internal evaluations.
During this phase, the COBOL Data Management System network database code was transformed to COBOL with SQL. This COBOL and SQL code was then transformed to Java and SQL code.
Figure 2 – The component COBOL automated refactoring to Java.
Phase 2: Advanced Refactoring to Remove COBOL Design Overtones
After we completed Phase 1, the resulting converted Java code contained COBOL paradigm design remnants, or COBOL overtones, that required personnel to have specialized skills to maintain the codebase. A plan was developed to identify and correct COBOL overtones with standard Java solutions. This part of the effort was considered low-risk because we used approaches, processes, and techniques proven in Phase 1.
Our refactoring approach used the TSRI JANUS Studio tool and a semi-automated refactoring method that performs further code optimization, naming changes, and other enhancements to improve architecture, design, security, sustainability and performance.
We used an iterative approach consisting of tool-driven automated refactoring, regression testing, and customer review to address the four most significant COBOL overtones existing in the component’s Java codebase:
- Refactoring the COBOL Memory Model to Java
- Refactoring the COBOL data mapping layer to native Java and SQL
- Removing COBOL-style control-flow/GOTO Logic
- Identifying and removing redundant code sections
These techniques, along with the improved method synthesis algorithm, greatly improved the maintainability of the Java codebase.
Figure 3 – The component refactored to maintainable Java.
Phase 3: Infrastructure Moved to AWS
After Phase 2 was completed, the entire component was moved to AWS GovCloud (US) including the staging, production, and all support environments.
AWS GovCloud (US) was selected because it aligned with the future technical direction of the system. This included cybersecurity, DevOps, and automated continuous integration (CI) and continuous delivery (CD) tools. The component’s architecture was installed in AWS GovCloud (US) and continues to evolve.
Figure 4 – The component moved to AWS.
Customer Benefits
For the USAF, the component has been transformed from an expensive mainframe COBOL legacy system to an affordable, modern maintainable Java-based system. All valuable existing business rules have been preserved in the modernized system, while development, test, and production environments were migrated to AWS providing flexibility at reduced cost.
The USAF’s projected cost savings is $25 million per year, and they are now able to use popular Java programmers to maintain and enhance the critical component.
Lessons Learned
This project taught the USAF that automated refactoring projects are a viable approach to modernize major legacy systems, including complex COBOL-based mainframes, to preserve the existing business rules within a cost-effective, open system.
We can highlight specific lessons learned during this project:
- Code conversions via automated refactoring are a low-risk approach to modernize away from legacy platforms and migrate to the AWS Cloud.
- Breaking efforts into phases (modernize, migrate, etc.) keeps risk low and ensures mission success.
- Breaking away from legacy COBOL enable a completely new set of sustainment opportunities, such as labor and tooling.
- Agile development models enable real-time course corrections and reduce delivery risk.
- Maximum automation is crucial in order to transform millions of lines of code while preserving functional equivalence, reducing risks, and reducing the project timeline.
- The Java language target allows choosing from a wide range of cost-efficient reliable compute options. In this case, it allowed the elasticity and scalability to meet the mission-critical USAF system needs.
- Moving away from legacy proprietary mainframe and COBOL to an open Java platform allows access to a large pool of talented architects and specialists for design and operation.
- Suppressing proprietary COBOL technologies allows more cost-efficient and reliable application servers and databases.
Next Steps
ARRAY specializes in transformative IT solutions for government agencies and the commercial sector. For more information about our mainframe and legacy migration to AWS capabilities, please contact your dedicated AWS account team at aws-california-team@amazon.com.
