Dealing with unreliable API endpoints can be a significant challenge. In this blog post, we'll explore how to implement custom retry logic and mock responses using Playwright, ensuring our tests remain robust and reliable even when APIs fail. Learn how to seamlessly handle API inconsistencies and verify data on the UI, ensuring your applications run smoothly and efficiently.

The Problem: Verifying API Data

Verifying data fetched from APIs can be a daunting task, especially when dealing with unreliable or slow endpoints. Here are some common challenges:

• Inconsistent Responses: APIs can sometimes return incomplete or incorrect data due to server-side issues or network problems.
• Timeouts and Failures: Network instability or server overloads can cause API requests to fail or time out, leading to incomplete test results.
• Data Variability: API data can change over time, making it difficult to create consistent test cases.
• Dependency on External Services: Relying on third-party APIs means your tests are dependent on their uptime and reliability, which can vary.

These challenges make it difficult to ensure the data displayed on the UI matches the data returned by the API. This is where retries and mocking come into play.

Importance of Retries and Mocking

Retries and mocking are crucial techniques for handling unreliable APIs in automated tests:

• Retries: Implementing retry logic allows our tests to make multiple attempts to fetch data from the API, increasing the chances of getting a successful response. This is particularly useful for intermittent network issues or temporary server problems.
• Mocking: When all retry attempts fail, mocking allows us to simulate API responses. This ensures our tests can proceed with predefined data, providing a fallback mechanism to verify UI behavior without relying on real-time API responses.

By incorporating retries and mocking, we can create more resilient and reliable tests, reducing the impact of external factors on our test results.

Use Case: University Data

To illustrate the importance of retries and mocking, let's consider a specific use case: verifying university data. In this scenario, we aim to fetch data about universities from an API and display it on a web page.

For our test, we need to ensure that the data fetched from this API is correctly displayed on the UI. The expected data includes:

• University Name: The name of the university as returned by the API.
• Source: The website of the university.

×

Test data representing university information

Given the potential unreliability of the API, we implement a test script that attempts to fetch the data three times. If all attempts fail, we use mocked data to verify that the UI still displays the information correctly.

Implementing the Test Script

We start by importing the necessary modules from Playwright and defining our test case. Here, we specify the API URL and mock data to be used if the API calls fail.

                                       
const { test, expect } = require("@playwright/test");

test("Verify university data with retries and mock", async ({ page }) => {
    const url = "http://universities.hipolabs.com/search?name=middle&country=turkey"; // API URL
                            
    const mockData = {
        name: "Mocked University",
        source: "http://mockeduniversity.com",
    };
                      

Next, we implement the retry logic. The test will attempt to fetch data from the API up to three times. If the API call succeeds, we break out of the loop. If all attempts fail, we use the mock data.

To begin, we initialize a retry counter (retries) and a variable (fetchedData) to store the fetched data. We then use a for loop to attempt the API call up to three times. Within the loop, if the API call is successful, we log the success and break out of the loop. However, if the API call fails, we log the error message and continue to the next attempt. This approach ensures that we have multiple opportunities to fetch the data before resorting to the mock data.

                                       
let retries = 3;
let fetchedData = null;
let attempt = 0;
                          
for (let i = 0; i < retries; i++) {
    try {
        attempt++;
        const response = await page.evaluate(() =>
            fetch(
            "http://universities.hipolabs.com/search?name=middle&country=turkey"
            ).then((res) => res.json())
        );
        fetchedData = {
            name: response[0].name,
            source: response[0].web_pages[0],
        };
        console.log(`Attempt ${i + 1}: Successfully fetched data`);
        break;
    } catch (e) {
        console.error(`Attempt ${i + 1} failed: ${e.message}`);
    }
}
                      

If all retry attempts fail, we use the predefined mock data. We also mock the API response to return the mock data.

If fetchedData is still null after all attempts, we log a message indicating that we are using mock data. We then set fetchedData to the mock data. To ensure the mock data is used, we use page.route to intercept the API call and return the mock data instead of making a real API request. This way, even if the API is unreliable, we can still perform the verification using consistent and predictable data.

                                       
if (!fetchedData) {
    console.log("All attempts failed. Using mock data.");
    fetchedData = mockData;
                            
    await page.route(url, (route) =>
        route.fulfill({
            status: 200,
            contentType: "application/json",
            body: JSON.stringify([
                { name: fetchedData.name, web_pages: fetchedData.source },
            ]),
        })
    );
    console.log("Verification done with mocked data");
} else {
    console.log("Verification done with real data");
}
                      

We navigate to the web page and wait for the elements containing the data to load. This ensures that the data is available before we attempt to verify it.

                                       
await page.goto("http://127.0.0.1:5500/resources/index.html"); //your website url

await page.waitForSelector("#name");
await page.waitForSelector("#source");
                      

Finally, we verify that the data displayed on the UI matches the data fetched from the API or the mock data.

                                       
    const universityName = await page.textContent("#name");
    const universitySource = await page.textContent("#source");
                          
    console.log(`Total attempts: ${attempt}`);
                          
    expect(universityName).toBe(fetchedData.name);
    expect(universitySource).toContain(fetchedData.source);
});
                      

×

Test output comparison

Conclusion

In this blog post, we explored the importance of handling unreliable API endpoints by implementing custom retry logic and mocking data responses. We started by identifying the challenges of verifying data from unreliable APIs and highlighted why retries and mocking are crucial for ensuring test reliability. We then walked through a specific use case of verifying university data, detailing the API endpoints and expected data.

We demonstrated the implementation of retry logic, where the test attempts to fetch data from the API up to three times, logging success or failure at each step. If all attempts fail, we showed how to use mock data by intercepting the API call and returning the predefined mock response. This approach ensures that our tests can still run and verify expected outcomes even when the API is unreliable.

We encourage you to implement and modify the script provided in this blog post to suit your specific use cases. Experiment with different retry strategies, mock data, and API endpoints to see how they impact your test outcomes. The script, as well as the mock website, are also available on our GitHub page.

Remember, thorough testing is vital in software development. By adopting best practices for writing robust and reliable tests, we can ensure the quality and stability of our applications. Happy testing!