A new paper on arXiv proposes an agentic method for deterministically validating legacy code migration from COBOL to Java[reference:36][reference:37]. The approach, called the Locksmith Loop, addresses a critical problem: migrating decades-old COBOL systems to modern Java requires extensive testing, but test data is often lost, and validating all corner cases is extremely difficult[reference:38].
The Locksmith Loop
The Locksmith Loop follows a teacher-student pattern[reference:39]. The "student" is a deterministic set of tools that carry out the migration, while the method prepares two runtime environments: the COBOL source and the generated Java target, both instrumented with mocks and executed on commodity hardware[reference:40]. An iterative agentic loop then performs a Witness Search over input mocks to penetrate program branches, followed by parity-preserving mutations[reference:41].
When routing boundaries are reached, an analyzer identifies a Locked Paragraph: a condition preventing deeper exploration[reference:42]. This process allows the system to systematically explore the program's behavior and validate the migration.
The Results
The researchers tested the Locksmith Loop on three COBOL-Java case studies, ranging from 430 to 4,114 source lines. The results were impressive:
- The method consistently improved coverage beyond input-search plateaus.
- It achieved nearly complete coverage on the two open-source programs.
- It reached 91.90% branch coverage on the internal production-like COBOL program[reference:43].
- The generated Java matched the COBOL reference under deterministic parity checks in all accepted test cases.
The Challenge
A 2025 Gartner forecast labels legacy modernization a $500 billion market, driven by COBOL's resource drain[reference:44]. A 2023 Stack Overflow survey found that 62% of developers are seeking AI tools to tackle such legacy systems[reference:45].
However, as the paper's findings show, the challenge is not just about achieving high coverage but about handling the remaining cases. The remaining 8.10% of branches represent the truly difficult corner cases that have plagued COBOL migrations for decades. As one commentator put it, "the remaining 8% is the real problem"[reference:46].
The Locksmith Loop demonstrates that AI can play a significant role in legacy modernization, but the hardest problems—the ones that require deep understanding of business logic and edge cases—still require human expertise. The paper represents a step forward, but it also highlights that AI is not yet a silver bullet for the COBOL problem.