It is time to learn new frameworks to improve your code quality and decrease the time of your testing phase. I have selected six testing frameworks that sound promising. Some exist for quite a long time but I have not heard about them before.
At the end of the article, tell me what do you think about them and what are your favorite ones?
Test framework RoboFramework
Robot Framework is a generic open-source automation framework. It can be used for test automation and robotic process automation (RPA).
Robot Framework is open and extensible and can be integrated with virtually any other tool to create powerful and flexible automation solutions. Being open-source also means that Robot Framework is free to use without licensing costs.
The RoboFramework is a framework to write test cases and automation processes. It means that it may replace your classic combo Selenium + Cucumber + Gherkins. To be more precise, the Cucumber Gherkins custom implementation you wrote will be handled by RoboFramework and Selenium invoked below.
For the Java developers, this framework can be executed with Maven (link) or Gradle(link) but less mature for the latter solution.
Testcafé
Testcafé is another E2E testing framework like Protractor or Cypress.
It provides a light DSL over the Browser API to write concise tests. It does not require WebDrivers, it uses Node.JS and the browsers installed on the machine. In a sense, it may not be the best solution for CI/CD environments but it is quite easy to use. It is still possible to use headless browsers for advanced users ( and for CI/CD). It also supports mock HTTP requests that are also great to make tests on a smaller portion of your scenario. Last but not least, the framework is supposed to handle the timeouts and sleep pause between the browser actions automatically.
JUnit Extension Random beans
https://glytching.github.io/junit-extensions/randomBeans
This JUnit extension integrates RandomBeans to enable your tests to integrate POJO generation with dynamic values.
Person person = random(Person.class);
Too often, tests are using fixed static data and are too long to read and maintain. With this extension, you may fill out your POJO with random values and ensures that your tests are validating properly their results.
@ExtendWith(RandomBeansExtension.class)
public class MyTest {
// injected with a random String
@Random private String anyString;
// injected with a random, fully populated instance of DomainObject
@Random private DomainObject fullyPopulatedDomainObject;
// injected with a random, partially populated instance of DomainObject
@Random(excludes = {"wotsits", "id", "nestedDomainObject.address"})
private DomainObject partiallyPopulatedDomainObject;
// injected with a List of random strings
@Random(type = String.class)
private List<String> anyList;
}JUnit Quickcheck
https://github.com/pholser/junit-quickcheck
junit-quickcheck is a library that supports writing and running property-based tests in JUnit, inspired by QuickCheck for Haskell.
To be more concrete, you define properties or invariants, and they are checked on random values produced by generators. So you can use define properties on objects and provide a generator that will initialize the POJOs with random values if possible.
Rather than testing such properties for all possible inputs, junit-quickcheck and other QuickCheck kin generate some number of random inputs and verify that the properties hold at least for the generated inputs. This gives us some reasonable assurance upon repeated test runs that the properties hold true for any valid inputs.
import com.pholser.junit.quickcheck.Property; import com.pholser.junit.quickcheck.runner.JUnitQuickcheck; import org.junit.runner.RunWith; import static org.junit.Assert.*; @RunWith(JUnitQuickcheck.class) public class StringProperties { @Property public void concatenationLength(String s1, String s2) { assertEquals(s1.length() + s2.length(), (s1 + s2).length()); } }
Several examples are located there to get an idea of how to use the framework and which benefits you may earn: https://github.com/pholser/junit-quickcheck/tree/master/examples/src/test/java/com/pholser/junit/quickcheck/examples
Framework Playwright
https://playwright.dev/docs/why-playwright
Playwright is another E2E JS testing framework publishing by Microsoft in 2020.
Among the interesting features provided by the framework :
- Test for mobile. Use device emulation to test your responsive web apps in mobile web browsers.
- Auto-wait APIs. Playwright interactions auto-wait for elements to be ready. This improves reliability and simplifies test authoring.
- Timeout-free automation. Playwright receives browser signals, like network requests, page navigations, and page load events to eliminate the need for sleep timeouts that cause flakiness
My personal opinion is to have a look at it since the integration with Robot Framework is confirmed
JQF
https://github.com/rohanpadhye/JQF
JQF is a java Fuzzy test library to perform fuzzy testing on your code by varying the values of the parameters and potentially discover bugs you have never thought about before.
@RunWith(JQF.class) public class PatriciaTrieTest { @Fuzz /* The args to this method will be generated automatically by JQF */ public void testMap2Trie(Map<String, Integer> map, String key) { // Key should exist in map assumeTrue(map.containsKey(key)); // the test is invalid if this predicate is not true // Create new trie with input `map` Trie trie = new PatriciaTrie(map); // The key should exist in the trie as well assertTrue(trie.containsKey(key)); // fails when map = {"x": 1, "x\0": 2} and key = "x" } }
The JQF Maven Plugin documentation shows how to run mvn jqf:fuzz.
PODAM
https://github.com/mtedone/podam
PODAM is a lightweight tool to auto-fill Java POJOs with data. This comes in handy when developing unit tests.
// Simplest scenario. Will delegate to Podam all decisions PodamFactory factory = new PodamFactoryImpl(); // This will use constructor with minimum arguments and // then setters to populate POJO Pojo myPojo = factory.manufacturePojo(Pojo.class); // This will use constructor with maximum arguments and // then setters to populate POJO Pojo myPojo2 = factory.manufacturePojoWithFullData(Pojo.class); // If object instance is already available, // Podam can fill it with random data Pojo myPojo3 = MyFactory.getPojoInstance(); factory.populatePojo(myPojo3);
It is another version of a Random library like RandomBeans for which I am a frequent user. PODAM is extensible and you can create your own generators. If you do not like Randombeans, have a look at PODAM.
In this article, we discussed 6 test frameworks I didn’t know and if we should remember one, it would definitely by RobotFramework. And you, which framework do you want to use in 2021?