AUITestAgent: Automatic Requirements Oriented GUI Function Testing
Yongxiang Hu, Xuan Wang, Yingchuan Wang, Yu Zhang, Shiyu Guo, Chaoyi Chen, Xin Wang, Yangfan Zhou
2024-07-18
Summary
This paper introduces AUITestAgent, an innovative tool that automates the testing of mobile app interfaces using natural language instructions.
What's the problem?
Testing the graphical user interface (GUI) of mobile apps is essential to ensure they work correctly, but it can be very time-consuming and labor-intensive. Traditional methods require testers to manually check each feature based on written requirements, which is challenging due to the large number of screens and rapid updates in modern apps. This makes it hard to keep up with testing needs efficiently.
What's the solution?
AUITestAgent solves this problem by automatically interpreting natural language test requirements and performing the necessary GUI interactions without human intervention. It extracts commands from the requirements and uses a structured approach to verify if the app behaves as expected. The system has been tested on various popular apps and has shown a high accuracy rate of 94% in verifying functions, significantly improving the efficiency of the testing process.
Why it matters?
This research is important because it streamlines the testing process for mobile applications, allowing developers to identify bugs and issues more quickly and accurately. By automating GUI testing, AUITestAgent can save time and resources, leading to better quality apps and a smoother user experience. This advancement is particularly valuable in today's fast-paced tech environment where apps are frequently updated.
Abstract
The Graphical User Interface (GUI) is how users interact with mobile apps. To ensure it functions properly, testing engineers have to make sure it functions as intended, based on test requirements that are typically written in natural language. While widely adopted manual testing and script-based methods are effective, they demand substantial effort due to the vast number of GUI pages and rapid iterations in modern mobile apps. This paper introduces AUITestAgent, the first automatic, natural language-driven GUI testing tool for mobile apps, capable of fully automating the entire process of GUI interaction and function verification. Since test requirements typically contain interaction commands and verification oracles. AUITestAgent can extract GUI interactions from test requirements via dynamically organized agents. Then, AUITestAgent employs a multi-dimensional data extraction strategy to retrieve data relevant to the test requirements from the interaction trace and perform verification. Experiments on customized benchmarks demonstrate that AUITestAgent outperforms existing tools in the quality of generated GUI interactions and achieved the accuracy of verifications of 94%. Moreover, field deployment in Meituan has shown AUITestAgent's practical usability, with it detecting 4 new functional bugs during 10 regression tests in two months.