DevAug Meeting Report: How AI Will Evolve Software Testing

We recently invited our product advisors to the first ever DevAug (the augmentation of software development) meetup. The engineering leaders below discussed the future of AI in the Software Development Lifecycle and how Autify Genesis will play a role in software QA.

Product Advisors

Dhara Patel - https://www.linkedin.com/in/dharapateldesai/ 

Heemeng (Chris) Foo - https://www.linkedin.com/in/heemeng/ 

Naveen Singhal - https://www.linkedin.com/in/naveen-singhal-48397/ 

Prasanna Ganapule - https://www.linkedin.com/in/prasanna-ganapule-6448a6/ 

Vilas Veeraraghavan - https://www.linkedin.com/in/vilas-veeraraghavan/ 

Ravi Ayeetha - https://www.linkedin.com/in/ravi-ayeetha/ 

The future of AI in software testing

Our CEO, Ryo Chikazawa, began the meetup with a short presentation on Autify’s vision of how software testing will evolve through AI. Ryo showed the diagram below of the current software testing process and how AI will affect them: 

• Test Design: AI will validate specs created by testing individuals to make sure they are thorough and provided adequate testing coverage
• Test Case Creation: AI will take created specifications and supporting documentation to generate natural language test cases
• Test Implementation: AI will generate test code by partnering with a test engineer based of the test cases it is given
• Test Execution: Visual recognition and intelligent selection and prioritization of elements by AI will lead to flexible tests
• Test Maintenance: AI will continue to advance and provide updates to tests and analyze failures to determine the cause, decreasing maintenance time

‍
How Autify provides services to achieve the above vision?

Autify NoCode and Playwright currently supports test execution and maintenance areas whereas Autify Genesis's aim is to approach the early stages of test strategy and design. Test Case Creation is where Autify Genesis and Autify/Playwright meet. Autify Genesis currently produces playwright code. We chose playwright as it is the fastest growing E2E testing tool, it’s lightweight and our no code tool will eventually be transferred over and built on top of playwright, giving more flexibility and freedom. We have plans to change the no code tool to a local system which will then integrate directly with Autify Genesis. 

Autify Genesis Roadmap

The Autify Genesis roadmap was a hot topic of conversation for people. Our first goal was for Test Case and Test Script Generation to be completed and at a level that could be at least 80% accurate. That goal should be achieved by the end of September upon which Autify Genesis will be opened to the public!

Once open beta begins, we hope to take feedback and analyze data in order to increase the quality of generated Gherkin and Playwright code. We also are trying to add Gherkin dictionaries, which is a place for an organization to help Autify Genesis understand organization specific terminology.

The biggest surprise in the DevAug meetup came in the form of test management. Autify’s original intent was to provide custom Test case labeling as a way to help with test management. Every person attending the meeting disagreed and made it clear that as customers, they want to be able to integrate with their own test case management, JIRA ticketing, and Code repositories. The rationale discussed boiled down to making Autify Genesis easier to access and fit into their own testing processes. We have decided to change our roadmap to accommodate for these valuable feedbacks. 

Once GA is released, Autify will be focused on integrating our NoCode product with Autify Genesis the same way playwright is and better user/organization management within Autify Genesis.

The partnership of AI and Test Engineers

There was a question about why anyone should care and review generated playwright code and why a person can’t just take the code and run with it. We think it is important that we highlight this question and answer. 

AI should be treated as a partner, not an entity that does something entirely on its own. AI is not perfectly accurate and should not be treated as such. Autify’s goal is for an engineer and Autify Genesis to partner together, where Autify Genesis works with the engineer to help speed up the test creation process with the help of his/her expertise. By reviewing generated gherkin and code, an engineer can help catch the mistakes that Autify Genesis may make and guide it when it needs to input from a user. It is through this partnership that companies can benefit by speeding up their testing process while avoiding costly mistakes that an AI might make. 

If you and your company has interest in speeding up the test creation process then please visit Autify Genesis and sign up for our open Beta coming on Oct 8th!