Testing

Whenever new features are implemented for WTP, we should add tests to ensure this feature still works with further changes to the code. Whenever code is merged, the tests have to be able to pass. Testing makes sure that existing functionality is not accidentally broken and that the application remains stable over time.

E2E testing

To break it down, End to end testing is a type of testing that verifies the entire software application from start to finish. We implement workflows and our end to end tests ensure that they work as they should. For end to end testing, we use a tool which is called Playwright.

Writing E2E tests

This guide explains how to write new Playwright test cases for the WTP canvas. It focuses on the essential steps needed to create reliable end-to-end tests.

At the top of your test file, import Playwright APIs and the WTPPage fixture:

import { expect, test } from "@playwright/test";
import { WTPPage } from "./fixtures";

Create a suite for the application and initialize a fresh WTPPage before each test. Setting a fixed viewport and clearing the snapshotSuffix ensures that screenshots remain consistent across local and CI runs.

test.describe("Application", () => {
  let WTPpage: WTPPage;

  test.beforeEach(async ({ page }, testInfo) => {
    WTPpage = new WTPPage(page);
    await WTPpage.goto();
    await page.setViewportSize({ width: 800, height: 600 });
    // omit platform suffix so CI snapshots match locally
    testInfo.snapshotSuffix = "";
  });

  // Tests go here...
});

Each feature should be covered by a clearly named test case. For example, to test the makeTurtle function:

test("makeTurtle", async ({ page }) => {
  // steps go here
});

First, make sure the runtime is fully initialized:

await WTPpage.awaitPyodideReady();

This will insert the code snippet into the editor:

await WTPpage.updateEditorContent("from gturtle import *\nmakeTurtle()");

Run the code and wait until execution finishes

await WTPpage.runCode();
await WTPpage.execDone();

Visual testing

To test some features, we can use Visual Testing. Visual testing is by done by capturing the program state with Screenshots. Screenshots can be updated, if the desired program behaviour changes. For visual testing we use the following Playwright function, which lets us screenshot the whole app or single components and then compare the last screenshot with the expectation .

await expect(WTPpage.canvas).toHaveScreenshot("makeTurtle.png");

Complete Example:

import { expect, test } from "@playwright/test";
import { WTPPage } from "./fixtures";

test.describe("Application", () => {
  let WTPpage: WTPPage;

  test.beforeEach(async ({ page }, testInfo) => {
    WTPpage = new WTPPage(page);
    await WTPpage.goto();
    await page.setViewportSize({ width: 800, height: 600 });
    testInfo.snapshotSuffix = "";
  });

  test("makeTurtle", async ({ page }) => {
    await WTPpage.awaitPyodideReady();
    await WTPpage.updateEditorContent("from gturtle import *\nmakeTurtle()");
    await WTPpage.runCode();
    await WTPpage.execDone();
    await expect(WTPpage.canvas).toHaveScreenshot("makeTurtle.png");
  });
});

Running E2E tests

To run E2E tests

npx playwright test

This runs all available tests. For a better developer experience, run E2E tests in UI Mode

npx playwright test --ui

This opens a visual interface where you can see tests running and debug them more easily. If you want to run a specific test, pass in the file name want to run

npx playwright test turtle_tests.spec.ts

This is useful when you only want to check one feature instead of the whole test suite.

For further details and advanced usage of Playwright, please refer to the official Playwright documentation

Note:

If the screenshot file does not exist yet, Playwright will generate it during the first run. In this case, the test will fail the first time and you need to run it again so that the new screenshot is used as the baseline for comparison.