Introduction

"I want to automate tests, but writing test code is a hassle..."

"It's hard to modify test code every time the UI changes..."

"Errors occur and it doesn't work..."

Many developers might have these concerns. In this article, we'll introduce a tool that super-efficiently combines generative AI and Laravel Dusk for browser testing.

Table of Contents

1. Laravel's Test Automation Tools
2. Browser Testing × Generative AI
3. Environment Setup
4. Test Case Generation
5. Test Execution
6. Summary

If you don't need the introduction, please proceed to 3. Environment Setup.

1. Laravel's Test Automation Tools

Laravel has two built-in test automation tools:

Unit Testing with PHPUnit

Unit testing is a testing method that verifies individual parts of an application independently. In Laravel, PHPUnit is integrated as the default testing framework, allowing developers to automate unit tests and improve code quality.

Laravel 11.x Testing

Browser Testing with Laravel Dusk (WebDriver)

Browser testing is a type of E2E (End to End) testing that verifies the flow of web applications. Laravel Dusk is a browser test automation tool developed for the Laravel framework, using WebDriver to automate actual browser operations. With this tool, you can smoothly execute E2E tests from UI operations like button clicks and form submissions to verifying dynamically changing screens.

Laravel 11.x Laravel Dusk

Characteristics of Unit Testing and Browser Testing

2. Browser Testing × Generative AI

Despite its drawbacks, browser testing is very powerful as it can test E2E including JavaScript. We'll improve the issues of test script creation and maintainability in browser testing by utilizing generative AI (GPT-4o).

Automatically Generate Code from Natural Language

Using generative AI, you can automatically generate code by instructing user operations in natural language.
For example, if you write this instruction:

    public function testQiita(): void
    {
        $this->browse(function (Browser $browser) {
            // Navigate to qiita.com
            // Enter "blocs" in the search form and press return
            // Click the link of the top article
        });
    }

It will automatically generate code like this. You can run browser tests by executing the generated code.

    public function testQiita(): void
    {
        $this->browse(function (Browser $browser) {
            // Navigate to qiita.com
            $browser->visit('https://qiita.com');

            // Enter "blocs" in the search form and press return
            $browser->type('input[name="q"]', 'blocs')
                ->keys('input[name="q"]', \Facebook\WebDriver\WebDriverKeys::ENTER)
                ->pause(1000);

            // Click the link of the top article
            $browser->scrollIntoView('.style-1kxl4ny')
                ->clickLink('【Laravel】Template engine を作ってみた')
                ->pause(1000);
        });
    }

Specify Elements in Natural Language like "Confirm Button" or "Search Form"

The generative AI analyzes the HTML of the displayed page and retrieves CSS selectors for input fields or buttons specified in natural language. This allows you to specify elements in natural language like "search form" or "confirm button". Even if UI changes cause errors in browser tests, you can resolve errors by simply regenerating the code without changing the instructions.
For example, in this code:

// Click the confirm button, then click the update button in the modal
$browser->press('Confirm')
    ->waitFor('#modalUpdate')
    ->press('#modalUpdate .btn-primary');

Even if the button class changes, it will generate code that matches the UI:

// Click the confirm button, then click the update button in the modal
$browser->press('Confirm')
    ->waitFor('#modalUpdate')
    ->press('#modalUpdate .btn-secondary');

We have developed a package that supports Laravel Dusk code generation with generative AI.

https://github.com/blocs/dusk

You can super-efficiently perform browser testing in just these 3 steps. Generative AI only supports step 2 (Test Script Creation). While using GPT-4o incurs costs per token, once test script creation is complete, there are no costs for test execution.

• Environment Setup
• Test Script Creation
• Test Execution

3. Environment Setup

When using this package, you don't need to install it directly in the Laravel project you're testing. As long as the project you're testing is accessible from a browser (running as a development server or staging environment), you can build a separate Laravel project for testing and run tests from there.

LLM usage is required for test code generation, so if you don't have one, please obtain an OpenAI API key. How to obtain an OpenAI API key

Required Environment

• PHP 8.1 or higher
• Composer
• Laravel 10 or higher

Setup Commands

1. Create a Laravel Project for Testing

composer create-project laravel/laravel [project name]
Example: composer create-project laravel/laravel dusk-web-test

2. Install blocs/dusk

cd dusk-web-test
composer require --dev blocs/dusk

3. Install Laravel Dusk and Open AI

php artisan dusk:install
php artisan openai:install

4. Run Laravel Dusk

php artisan dusk

Errors may occur on the first execution. In that case, please run it again.

5. Set up env

OPENAI_API_KEY=your-api-key-here

This completes the environment setup.

4. Test Script Creation

The directory structure related to testing is as follows:

dusk-web-test/
├─ config/
│   └─ openai.php/
├─ tests/
│   ├─ Browser/
│   │   ├─ ExampleTest.php
│   │   ├─ QiitaTest.php
│   │   ├─ blocs/
│   │   │   ├─ system.md
│   │   │   ├─ user.md
│   │   │   ├─ sample.md
│   │   │   └─ assistant.md
│   │   ├─ screenshots/
│   │   └─ ...(other directories omitted)
│   ├─ Feature/
│   ├─ Unit/
│   └─ DuskTestCase.php
└─ ...(other directories omitted)

Browser test scripts (like ExampleTest.php) are placed under tests/Browser. Files under tests/Browser/blocs have roles set for code generation. By editing these files, such as sample code, you can improve the accuracy of code generation for each project. What is "role" in OpenAI API

4.1 Prepare Test Script

Write instructions for the operations you want to perform as comments.

    public function testBasicExample(): void
    {
        $this->browse(function (Browser $browser) {
            // Navigate to http://dev.local/login
            // Enter ID:admin, Password:admin and login
            // Click the User link in the side menu, specify the link with a CSS selector
            // Click the new button from the list screen
            // Enter a random email in userID and check the admin checkbox
            // Enter a random password in password, enter the same password in repassword
            // Click the Confirm button then press the submit button
        });
    }

You can also include multiple test cases in one test script.

    private function testQiita($browser): void
    {
        $this->browse(function (Browser $browser) {
            // Navigate to qiita.com
            // Enter "blocs" in the search form and press return
            // Click the link of the top article
        });
    }

    private function testYahoo($browser): void
    {
        $this->browse(function (Browser $browser) {
            // Navigate to yahoo.co.jp
            // Enter "generative AI" in the input form and click the search button
            // Confirm that ChatGPT is displayed
        });
    }

4.2 Create Test Script

Run the support tool with this command. When you run the support tool, command selection will be displayed. (You can select with ↑↓.)

php artisan blocs:dusk tests/Browser/ExampleTest.php

Code generation for test cases is done interactively step by step. If code for a step has already been generated, that step is automatically executed. If code has not yet been generated, it automatically generates code for the operation instructed in the comment. Select the command for the automatically generated code.

After automatic generation of all steps is complete, you can add steps.

Specifying the LLM Model

In the default settings when introducing the package, it combines gpt-4o-mini and gpt-4o to generate code. Basically, it uses gpt-4o-mini, and queries gpt-4o when adding instructions to regenerate code. If you want to specify the model to use, write it in config/openai.php. If specified like this, it will always use gpt-4o-mini.

'model' => 'gpt-4o-mini',

4.3 Dry Run

You can perform a dry run of the test by executing the run command of the support tool. When actually executed, it looks like this.

5. Test Execution

Executing this command will run the test script with Laravel Dusk. Laravel Dusk executes browser tests in headless mode, so it operates in the background without displaying a screen. As long as there are no specification changes such as UI changes, you can perform regression testing by executing this command. If you need to modify the test script, go back to test script creation and make modifications.

php artisan dusk tests/Browser/ExampleTest.php

6. Summary

In this article, we introduced a tool that super-efficiently performs browser testing with Laravel Dusk × GPT-4. If you're interested, please give it a try!

Conclusion

Our company is also developing a template engine to make Laravel development more convenient.

• BLOCS Official Reference
• 【Laravel】I Created a Template Engine

References

• Laravel Dusk Official Documentation
• OpenAI API Reference