Refactoring a Codebase with and without Generative AI: A Comparative Study

Abstract

Anna Hou, Skylar Myers, Rachael Poffenberger, and Dr. Weidong Liao (Faculty Advisor), Dept of Computer and Information Sciences, Shepherd University, Shepherdstown, WV, 25443. Refactoring a Codebase with and without Generative AI: A Comparative Study.

The objective of this study is to compare the efficiency and accuracy of software refactoring workflows with and without Generative AI tools.

Maintaining legacy software is traditionally a time-intensive process involving manual code comprehension, test generation, and structural alteration. We conducted a comparative study of the workflows for refactoring an existing legacy codebase. The first workflow is the traditional and manual one that relies on standard static analysis and IDE features. The second workflow utilizes Generative AI assistants, specifically Google AntiGravity and Gemini Code Assist. Our comparative analysis measured the time expended, the number of logical errors introduced, and the overall improvement in code readability. The results indicate that the AI-assisted workflow significantly reduced the time required for initial code comprehension and boilerplate unit test generation. The AI tools rapidly identified technical debt and generated functional refactoring snippets. However, the traditional manual workflow, while slower, would result in fewer architectural misalignments during complex dependency injections. We conclude that Generative AI significantly accelerates the refactoring process, particularly in the discovery and test generation phases, but rigorous human oversight remains critical to ensure structural integrity and prevent AI-generated logic errors. This project was supported by the National Science Foundation S-STEM grant awarded to Shepherd University.