Xinyu Wang earns NSF CAREER Award to democratize web automation

Xinyu Wang, assistant professor of computer science at the University of Michigan, has received a National Science Foundation (NSF) CAREER Award to democratize web automation tools. The project is titled “Interactive Program Synthesis for Web Automation.”

AI and automation involves a number of complex algorithms and techniques that, to this point, have been the domain of specialists and enthusiasts. But automation is a powerful tool for day-to-day life, and many work, finance, and personal tasks can be put on autopilot with the right know-how. Simple scripts free up time spent on things like logging changes to an account, scheduling meetings on a calendar app, and doing repetitive tasks for work.

Wang’s project aims to democratize these web automation tools by developing new technologies that enable non-experts to build their own web automation programs. These include any programs that can automate interactions between data (like data stored in a spreadsheet) and a web browser. 

“This research is motivated by the fact that an increasing number of populations struggle with such tedious web-related tasks, such as web scraping or data entry,” Wang says.

In his CAREER research, Wang will develop new program synthesis algorithms that automatically generate web automation programs from user interactions with the browser. In practice, this means that users can demonstrate a tedious task they’d like to have automated in their web browser, and the AI-based programming assistant will generate a program that can carry out the necessary steps. The algorithms will ultimately be implemented in a browser extension, with an emphasis on usability and transparency.
Wang is doing related work as part of a collaboration with software company UiPath, as well as with IBM Research. Ultimately, he hopes to make AI-based automation comprehensible for users at every level and enable these effort-saving tools to be adopted widely.