Unlocking AI Superpowers in Test Automation: It All Starts with Your Framework

Tried using AI agents to generate tests but ended up frustrated and ready to give up? Hold on — this might just change the game for you. Keep reading until the End! Let’s talk about something that’s been buzzing for a while now, AI in test automation. Yep, it’s cool, powerful, and kind of magical when it works! But here’s the catch: most people assume AI is some all-knowing oracle that just writes perfect test scripts if you ask nicely. Spoiler alert: It doesn’t‼️ In my experience, the real magic happens only when you pair AI with a well-structured test automation framework and a proper prompt. That’s when things start to click. So let’s unpack that a bit. The AI Isn’t the Sole Hero Here I’ve played around with AI agents like ChatGPT, GitHub Copilot, JetBrains Junie and a few custom LLM setups to help generate tests. And I’ll be blunt: the output varies wildly depending on ‘from-where’ you prompt. Which means the environment where you are in. Which means in this case, specifically, your test framework. Let me explain.. If your test automation framework is a chaotic mess with test files dumped all over the place, no clear naming convention, or structure — AI has no idea what to do with that. It’ll guess. And most of the time, it guesses wrong. But when the framework follows a clear architecture, like the Page Object Model (POM), or even better, a layered design with base test classes, reusable commands, and folder separation for different test types — AI can start making sense of it. It’s kind of like giving the AI a map. Without it, it’s lost in the woods. With it? It can run a marathon blindfolded. Share the Blueprint: Feed the Architecture to AI Here’s something that works like a charm.. Before asking AI to generate any tests, I give it a basic overview of my framework architecture. Not a 20-page documentation dump, but something like: - /tests: test specs go here - /pages: all page objects live here - /utils: helper functions - We use TypeScript + Cypress Note: Above is just a hint on how to give the information Then I’ll follow up with a simple example test and page object file, so the AI can get the “pattern.” After this little orientation tour, the AI becomes surprisingly accurate. It knows where to place new tests, how to reference page objects, how hooks work, and even when to reuse utility functions. It’s like onboarding a new teammate. You don’t just throw them into the deep end, you show them around first. The Prompt Is Your Steering Wheel Okay, so now your framework is neat and structured, and you’ve handed over the blueprint. What’s next? Prompting. This part is half art, half science. If you say “Write a login test,” you’ll probably get something generic that may or may not fit your actual setup. But if you say something like: “Write a Cypress test in TypeScript to verify the login flow using the Page Object Model. Use the LoginPage class from /pages/LoginPage.ts. The test should check for successful login and redirection to the dashboard.” Boom! Now you’re talking! ❤ Add test data examples, expected behaviors, or edge cases into your prompt, and AI will often generate tests that are eerily spot-on! Real-World Result? AI Writes, You Refine When the framework is well-organized and you guide the AI properly, the tests it generates aren’t just dummy scripts. They’re often 80–90% production-ready. You still need to polish a few edges — naming, assertion tweaks, maybe a bit of test data handling — but the base is there. And that’s a massive time-saver. I’ve even had AI generate test cases for new features on the same day they were developed — saving hours of manual work and letting me focus on edge case validation and exploratory testing instead. Make AI Work with You, Not for You! So yeah, AI can be a great assistant in test automation, but only if you do your part first: Keep your test framework clean, modular, and structured Share your framework structure with the AI agent Write prompts that are clear, detailed, and contextual The AI isn’t your silver bullet. But with the right setup, it’s a serious productivity booster! Gift for the Finishers! If you’ve made it this far, thank you! As a little treat, I’ve got something for you. I recently worked on a Cypress boilerplate framework tailored for beginners and teams who want a clean, scalable starting point. It follows the Page Object Model, is structured for scalability & maintainability. And this will play well with AI agents when generating tests. I’ve bundled it all up and published it as an npm package so you can kickstart your next project with ease. Grab it here: cypress-bootstrap Give it a spin, and let me know what you think!

May 11, 2025 - 20:50

Unlocking AI Superpowers in Test Automation: It All Starts with Your Framework

Tried using AI agents to generate tests but ended up frustrated and ready to give up? Hold on — this might just change the game for you. Keep reading until the End!

Let’s talk about something that’s been buzzing for a while now, AI in test automation. Yep, it’s cool, powerful, and kind of magical when it works! But here’s the catch: most people assume AI is some all-knowing oracle that just writes perfect test scripts if you ask nicely.

Spoiler alert: It doesn’t‼️

In my experience, the real magic happens only when you pair AI with a well-structured test automation framework and a proper prompt. That’s when things start to click. So let’s unpack that a bit.

The AI Isn’t the Sole Hero Here

I’ve played around with AI agents like ChatGPT, GitHub Copilot, JetBrains Junie and a few custom LLM setups to help generate tests. And I’ll be blunt: the output varies wildly depending on ‘from-where’ you prompt. Which means the environment where you are in. Which means in this case, specifically, your test framework.

Let me explain..

If your test automation framework is a chaotic mess with test files dumped all over the place, no clear naming convention, or structure — AI has no idea what to do with that. It’ll guess. And most of the time, it guesses wrong.

But when the framework follows a clear architecture, like the Page Object Model (POM), or even better, a layered design with base test classes, reusable commands, and folder separation for different test types — AI can start making sense of it.

It’s kind of like giving the AI a map. Without it, it’s lost in the woods. With it? It can run a marathon blindfolded.

Share the Blueprint: Feed the Architecture to AI

Here’s something that works like a charm..

Before asking AI to generate any tests, I give it a basic overview of my framework architecture. Not a 20-page documentation dump, but something like:

- /tests: test specs go here
- /pages: all page objects live here
- /utils: helper functions
- We use TypeScript + Cypress

Note: Above is just a hint on how to give the information

Then I’ll follow up with a simple example test and page object file, so the AI can get the “pattern.”

After this little orientation tour, the AI becomes surprisingly accurate. It knows where to place new tests, how to reference page objects, how hooks work, and even when to reuse utility functions.

It’s like onboarding a new teammate. You don’t just throw them into the deep end, you show them around first.

The Prompt Is Your Steering Wheel

Okay, so now your framework is neat and structured, and you’ve handed over the blueprint. What’s next?

Prompting.

This part is half art, half science. If you say “Write a login test,” you’ll probably get something generic that may or may not fit your actual setup.

But if you say something like:

“Write a Cypress test in TypeScript to verify the login flow using the Page Object Model. Use the LoginPage class from /pages/LoginPage.ts. The test should check for successful login and redirection to the dashboard.”

Boom! Now you’re talking! ❤

Add test data examples, expected behaviors, or edge cases into your prompt, and AI will often generate tests that are eerily spot-on!

Real-World Result? AI Writes, You Refine

When the framework is well-organized and you guide the AI properly, the tests it generates aren’t just dummy scripts. They’re often 80–90% production-ready.

You still need to polish a few edges — naming, assertion tweaks, maybe a bit of test data handling — but the base is there. And that’s a massive time-saver.

I’ve even had AI generate test cases for new features on the same day they were developed — saving hours of manual work and letting me focus on edge case validation and exploratory testing instead.

Make AI Work with You, Not for You!

So yeah, AI can be a great assistant in test automation, but only if you do your part first:

Keep your test framework clean, modular, and structured
Share your framework structure with the AI agent
Write prompts that are clear, detailed, and contextual

The AI isn’t your silver bullet. But with the right setup, it’s a serious productivity booster!

Gift for the Finishers!

If you’ve made it this far, thank you! As a little treat, I’ve got something for you.

I recently worked on a Cypress boilerplate framework tailored for beginners and teams who want a clean, scalable starting point. It follows the Page Object Model, is structured for scalability & maintainability. And this will play well with AI agents when generating tests.

I’ve bundled it all up and published it as an npm package so you can kickstart your next project with ease.

Grab it here: cypress-bootstrap

Give it a spin, and let me know what you think!