QA 2.0: Deep dive Part 1.2:Parasoft Selenic
If you haven’t checked my previous articles, I recommend you start from the index page to understand the full context.
AI-Integrated Test Automation Tool Consideration and Classification article
Now, let’s dig deep into Streamline Selenium Testing With Parasoft Selenic.
As per the official website
Summary
When your team is using Selenium to develop and test the UI for your software applications, you need confidence that your testing process is identifying real issues, creating meaningful and appropriate tests, and reducing test maintenance. While Selenium offers many benefits, you want to get more out of your UI testing while leveraging your current practice. Find the real UI issues and get quick feedback on test execution so you can deliver better software faster with Parasoft Selenic.
Improve your existing library of Selenium web UI tests, or quickly create new ones, with a flexible Selenium companion that integrates seamlessly with your Agile DevOps environment. Parasoft Selenic fixes common Selenium problems with AI-powered self-healing to minimize runtime failures, test impact analysis to dramatically reduce test execution time, and the ability to record new scenarios using a Chrome extension to create Page Object Model Selenium code.
Main advantages
- Keep Using Selenium
- Focus on Regressions Instead of Test Maintenance
- Reduce the Effort to Create Reliable Selenium Tests
- Eliminate 50% of Your Build Failures
- Reduce Maintenance Time by 80%
- Jumpstart UI Test Creation
How it works?
Leveraging AI to solve these challenges, Parasoft Selenic is a unique solution that works with regular Selenium tests, rather than locking users into a proprietary “Selenium-powered” platform like all other available solutions. Simply integrate Selenic into your current testing processes to see immediate results with your Selenium projects — no need to migrate or recreate the tests you’re already using. Parasoft Selenic accelerates automated testing and improves the stability of Selenium tests without changing your existing Selenium practice.
Demo
Let’s Find out if these claims are true.
For this demo purpose, I will be using locally hosted https://github.com/browserstack/browserstack-demo-app.
So that I can change element names and create flaky tests to check the self-healing capabilities of the Selenic.
First impression
While I was reading the details of the Selenic product on their website one of their customer agents connected with me and redirected me to request a trial version of Selenic. After discussing my requirements with the marketing team, they have provided binaries as well as trial keys, not only once, but twice since I couldn’t explore the full features of the product within my first trial period.
Very professional and quick responsive support team.
Installation
Installation files come in separate installers. Plugins for IDEs, Recorder browser plugin etc. However, as per the build number they have provided, they haven’t made any improvements to the software after 2022.
However, they planned to release a major software update in 2024, which includes support for the latest versions of IDEs and fixes for known issues.
The installation process is straightforward and well-documented. But I prefer an all-in-one installer as many other vendors do.
Also, I had to downgrade my IntelliJ Idea version to a lower version since the Selenic plugin only supports IntelliJ 2020.1 to 2021.1. (This issue will be addressed in up coming release)
After the installation process, we have to do some configs like setting up the license and downloading browser drivers.
All well documented in https://docs.parasoft.com/display/SEL20222/Getting+Started
The Recorder
Selenic comes with a Chrome browser plugin that helps us record user actions and generate test scripts based on user actions and assertions.
Simple and user-friendly design that helps users to understand the functionality without referring to documents.
Check the following link to see the Selenic recorder in action.
After recording user actions you can simply export the recording as a JSON file to import into the IDE.
The IDE Plugin
After downloading the recording we can easily import the recording file to the IDE. In my case IntelliJ IDEA.
After importing it will create a Java project based on the Unit testing framework that you set up during the IDE plugin configuration process.
Also, the import process will create page object classes. Check the following video to see the project structure and generated classes.
Test Execution
After you configure the test run you can run your tests with Selenic.
As mentioned in the official documentation, users have to use Java 11+ to support Selenic.
If you encounter the following issue, please update the Selenium version. It’s fixed in the latest version of Selenium.
org.openqa.selenium.remote.http.ConnectionFailedException: Unable to establish websocket connection to http://localhost:64830/devtools/browser/a823106b-f77e-4625-b33c-628191698e8c
Build info: version: '4.7.2', revision: '4d4020c3b7'You are ready for the test execution !!!
However, in my scenario, Selenic couldn’t recognize one of the recorded actions to insert a password for my demo page.
After correcting the element I was able to execute the tests successfully.
The test report was created in a temp file location which is not mentioned in logs. However, if you select “Open HTML report after execution” report will be automatically loaded in the web browser.
It contains all the required details including screenshots of healed or failed screens.
Another nice additional feature in Selenic is to show failed assertion points in the IDE. So we can easily correct them.
org.opentest4j.AssertionFailedError:
Expected :CONTINUE SHOPPING »
Actual :Continue Shopping »
<Click to see difference>
The Self Healing
This is where Selenic comes in handy.
As you can see in the above test execution report it healed on the element which couldn’t be found with the locator that we have provided.
Since the demo site is locally hosted, I played with different elements to check Selenic’s self-healing ability.
<div className="form-field">
<label for="firstNameInput" className="form-label optimizedCheckout-form-label">First Name 1</label>
<input required autoComplete="given-name" id="firstNameInput" className='form-input optimizedCheckout-form-input' type="text" defaultValue="" />
</div>
Selenic was able to self-heal the issue.
You can click on the points in the Test to Review section to check how the healing process worked.
It gives a detailed view of the locators used instead of just failing the test.
Also, they have a cool feature to replace flaky or incorrect locators with recommended locators by reusing the Selenic Recommendations console.
Using Parasoft DTP users can easily integrate test scripts with major defect management and ALM solutions like Jira, Azure DevOps etc. Parasoft DTP seamlessly integrates with defect management software to give faster feedback and reporting.
Check their official documentation to see more features regarding ALM solutions.
https://docs.parasoft.com/display/DTP20232/Integrating+with+ALM+Tools
Issues
Since I ran a demo web application on my local machine, I saw some delay in loading pages. So based on the loading time one button with the text “Continue Shopping” shows different behaviours.
Ex: When there are delays in loading time, the button looks like this.
However, when the page is loaded properly the button shows in all capital.
So, confirmationPage.getButtonFieldText() provides two different outputs based on page loading time.
assertEquals("CONTINUE SHOPPING »", confirmationPage.getButtonFieldText());
//assertEquals("Continue Shopping »", confirmationPage.getButtonFieldText());This is easily resolved by improving the performance of the hosted website or by increasing timeouts.
The Parasoft marketing team informed me that they have added an enhancement request to address this issue. So most probably we will have a permanent fix for this issue in the next major release.
The Evaluation
I have recently added the world’s first AI-integrated tool evaluation criteria. If you haven’t checked that article please check that article to understand the evaluation criteria I used to evaluate Selenic.
The overall score for Parasoft Selenic is 88/120 based on my evaluation criteria.
Conclusion
Selenium tests lack the capability to flexibly adjust to modifications in web applications or browsers. Selenic addresses this limitation by offering self-healing mechanisms for repairing broken locators and wait conditions in real time, ensuring the continuity of tests. Additionally, Selenic provides post-execution recommendations, facilitating teams in advancing with releases and fulfilling the Agile and DevOps principles. Specifically designed for enterprise application software like Salesforce, Selenic introduces enhanced locator and wait condition strategies. Furthermore, it enables the capture of assertions during recording, seamlessly integrating them into Selenium tests using pure Java.
Also, as I mentioned earlier they have a proactive support team to address your technical issues.
