Panel Discussion: How to Decide What Automation Technology to Use [Testμ 2022]

To decide what automation technology to use, we brought together Joe Colantonio, Founder of TestGuild, Sneha. V, Director of Quality Engineering, Everfi, and Carlos Kidman, Director of Engineering, Stealth Startup. The panel discussion was hosted by Mudit Singh, Marketing head at LambdaTest. Mudit decided to take a step backwards and let the panel discussion happen.

Here is the full video link:

Joe Colantonio took the lead in introducing the other two panel members. Sneha gave a glimpse about her Code stories podcast, where she interviews people to learn about their passions and hobbies.

He described how people struggle to know which automation technology we can use. Joe quizzed Sneha on having a solid automation strategy. Sneha started telling about how different companies she worked with used strategies based on skill set, budget, and cost. She spoke about how automation is an important step in DevOps. Without a strategy, Sneha feels we will miss out on many benefits of automation testing. Carlos agreed with Sneha and delved further into how he is often questioned by people about which AI tool they should use. He stresses that before choosing an automation technology, you should know the problem for which you will find a solution.

Joe spoke about how he failed to look at the roadmap while working for a certain big company. He spoke on how they struggled with the application when it rolled out new features.

Carlos also described how people are nowadays more curious about security testing.

Joe then enquired if Sneha ever had to put different silos together to strategize automation. Sneha explained how she has yet to experience something like that and mentioned that it might happen when acquisitions occur. She states this as a reason for scalable and standardized processes in place. When we have to determine which features and tools to use, it’s important to have a different skill set in the team. This is what makes our framework simple and easy to use.

Carlos was able to relate to it since he was in a similar situation when his previous company, Workfront, got acquired by Adobe. At Workfront, they were using AWS. They had to support GCP, Google Cloud Platform and Azure at Adobe. Hence they had to create a roadmap for what they wanted to tackle first and get more return on time. To ensure this happens, they formed a platform engineering team with a team of architects representing a specific domain. They created it to prioritize different features and standards. They were able to get things done this way successfully.

Joe asked Sneha if a particular strategy would work for her team. For that, she answered that she would go about checking the requirement, problem statement, and proof of concept. She described how she needed visual UI testing, audio testing, and video testing since she works in EdTech. Having a common standard of coding practices and doing a PoC also has helped her.

Carlos spoke about the importance of understanding what we need to have, want to have, like to have, and don’t want to have. He mentioned how Selenium tool and Cypress dominated the tech domain, competing for the first place a few years back, and then Playwright appeared. Hence he speaks about delving deeper into the tech radar to know what you need and what you don’t.

When asked which one among Selenium, Cypress, and Playwright testing he would choose, he spoke on how it’s entirely team dependent. According to him, you need to know what your infrastructure team needs.

Joe asked his two fellow panelists how they expected everyone to know about every test. Luckily for Sneha, one of her directors helped her learn about the axe core plugin with Cypress for accessibility testing. They also had an accessibility testing team. She insists on the importance of research on different tools.

Sneha also described how working along with developers can help her a lot.

Joe spoke about the importance of build vs buy. Carlos explained how his team could prioritize tasks at hand when he could buy the product instead of building it. He doesn’t prefer creating the data dog when he has less time in hand. He believes this can help him save time, energy, priority, business objectives, and deadlines.

Joe asked about the panel’s thoughts on Artificial Intelligence and Machine learning for automation testing.

“Quality is about the journey, not destination. There are going to be new requirements. If you need to buy an AI/ML tool, find out how it can fix the problem at hand right now.” Sneha replied.

Carlos described how powerful the technology is. He spoke about how all the companies would have a facility to try the trial version. He recommends everyone try the trial version to visualize things.

“When we say AI-driven automation or AI-driven testing, that doesn’t mean everything about that system is AI-based. Maybe a small part of it is AI-based. For example, looking at the test reports. In most cases, it’s just an outsourced company doing most of the automation. That’s why I recommend you to use the trial version since each company has a different flavor when they say AI-driven,” Carlos revealed.

Then the panel moved on to the lightning round to answer audience questions.

They answered the following questions:

How much multi-language support is important while picking up a browser automation tool?

Sneha: It depends. Some people think that they need to stick to the programming language the application uses. You can use whatever test automation framework you want.

Carlos: The one who’s going to maintain the things and the ones writing it should be picking it up. What do you think about the future of Python in automation testing?

Carlos: Python is a beast. It’s the language of automation. Hence it’s a very powerful language to learn. Is it good to have code coverage in automation?

Sneha: Yes, it’s good to have code coverage in automation. But you need to decide if you want coverage for the entire code. I don’t think it’s necessary to cover the whole code space.

Carlos: If it’s used as a tool for collaboration and conversation, it’s a great tool. I don’t use it as a tool for vanity metrics.

Do you think there is value in manual testing in the DevOps model, where efficiencies are believed to be driven by test automation?

Carlos: You cannot do testing without exploration. Hence manual testing is a must.

Sneha: I don’t think we can do 100% automation. We have to look into voiceover testing, content testing, and many different types of testing. Hence we would need manual testing. Why is there no sufficient budget for testing compared to development?

Sneha: It depends on the company’s culture and attitude. They need to realize that quality is everyone’s responsibility. Hence we need to bring in the engineering budget. Yes, we must fight for it, but we must convince the management.

How to know the feasibility of automation?

Carlos: You would have to evaluate a lot of things like the difficulty of the test and if the environment is ready for it.

Sneha: We do automation tests in an elementary manner. For example, we look at the name and size of the file during video testing.

Carlos: For someone in the leadership team, it’s their responsibility to evaluate the strategy by aligning with the team members.

Which programming language is best for API automation testing?

Carlos: It’s Python!

I would love to take your opinion on shift left testing.

Carlos: When you are doing BDD and TDD, there will be things like unit testing and many other things like risks and acceptance criteria. You will have a better idea of how things are going to be. That’s when shift left plays to help testers identify which test to focus on.

It was indeed an insightful session with our panel members. We hope that you would have got a gist on which automation technology you need to choose after listening to them.

Happy testing!