Agent browser automation works beautifully until it meets a real website.
The agent clicks through a demo page and looks unstoppable. Then you point it at a logged-in dashboard, or a form whose selectors moved after last week's redeploy, or a login that wants an SMS code. The reasoning model is fine. The layer underneath it was never built for this.
I spent a few days running BrowserAct through a JS-rendered page and a full login flow, then wrote the same login as a Playwright script to find where the two approaches diverge. Everything below is output I captured on Windows through WSL, including the run that failed while telling me it had worked.
What Playwright is built for
Playwright is deterministic. You know the page, you write selectors against it, you assert on outcomes. For CI end-to-end tests or scripted flows against a UI your team owns, it's hard to beat, because the script encodes knowledge you already have before the run starts.
That assumption is where agent workflows diverge, mostly because the agent doesn’t know the page.
In an agentic workflow, an agent arrives at a page it has never seen, on a DOM that may have shifted since the last run. Hand it raw HTML and you’re handing it thousands of tokens of nested markup with no stable handle to act on. It infers which node is the search box, guesses a selector, and hopes the guess survives the next render. When the layout changes, the guess breaks silently.
A script solves this by knowing the answer in advance. An agent can't, so it needs the page described in a form it can reason over. I wanted to see what that description actually looks like, so I ran some tests.
What the agent receives
I installed the CLI first, expecting the usual dependency archaeology:
uv tool install browser-act-cli --python 3.12
It resolved cleanly and dropped a single executable.
Another thing that caught my attention next was that the CLI refuses to ship the agent's instructions as a static file. The skill stub points at a command instead:
browser-act get-skills core --skill-version 2.0.2
Workflow content comes back matched to the installed version, so the guidance can't drift away from the binary driving it. For a tool whose commands change across releases, I thought that was a real answer to instruction rot.
The loop it served me is Open, State, Interact, Verify, Close. State is the part that matters, and here's what I got back against quotes.toscrape.com/scroll:
url=https://quotes.toscrape.com/scroll
|SCROLL|<html /> (0.0 pages above, 0.7 pages below,
left=0,top=0,width=1905,height=1716)
[1]<a />
Quotes to Scrape
[2]<a />
Login
"The world as we have created it is a process of our thinking..."
by
Albert Einstein
Tags:
[3]<a class=tag />
change
Every actionable element carries an index. To reach login, the agent clicks [2], and it does that without writing a selector or holding a DOM model between turns. I also liked that scroll position arrives as page-fractions, since the agent knows 0.7 of a page sits below it without measuring anything.
I captured the same moment visually with screenshot --full:
The viewport was 1716 pixels tall and had not been scrolled, yet the capture runs all the way to the footer. Between them these are two views of one moment, the screenshot for a human auditing what happened and the indexed state for the agent deciding what to do next.
Where network capture earns its place
State told me what was rendered, but I wanted to know where it came from, so I pulled the XHR traffic:
method,status,mime_type,url
POST,,,https://ogads-pa.clients6.google.com/$rpc/...GetAsyncData
GET,200,application/json,https://quotes.toscrape.com/api/quotes?page=1
That second row is the interesting one. The page renders its cards from a JSON endpoint, which means an agent can read api/quotes?page=1 directly rather than parsing rendered markup back into structured data. The first row is a Google ads call the page also fired, and I'd note that real capture always includes noise the agent has to filter.
One gotcha I hit: without --format json the CLI writes CSV, which will bite you if you're piping into a parser.
The failure that reported success
The scroll taught me more than anything that worked.
I opened the page, read state, ran scroll down --amount 3, waited, then read state again. Both snapshots came back identical. Same top=0, same tag indices [1] through [19]. Only page=1 had been fetched, when a successful scroll should have pulled page=2.
I assumed a race condition, so I re-ran the whole thing with wait stable --timeout 15000 and a second settle pass. Same result.
What threw me is that the command had reported success both times, echoing back scrolled=down amount=3 with exit code 0. That confirms the command executed, which turns out to be a very different claim from the viewport having moved. When I read the page markdown, Loading... was still sitting at the bottom.
Calibration fixed it. Running --amount 3000 gave me this:
|SCROLL|<html /> (0.7 pages above, 2.1 pages below, height=3794)
Compare that against the earlier height=1716. The document had more than doubled, which only happens when new content loads, and the network capture confirmed where it came from:
json
{"url": "https://quotes.toscrape.com/api/quotes?page=1", "status": 200}
{"url": "https://quotes.toscrape.com/api/quotes?page=2", "status": 200}
So the amount behaves like pixels, and my original 3 meant three pixels of movement on a 1716-pixel viewport. That's a documentation gap rather than a bug.
The lesson sits in the first attempt, though. Nothing errored anywhere in that sequence. Every command returned success, every state snapshot looked valid, and the page underneath had not moved at all. Verification has to come from the page's own evidence, whether that's a changed scroll position, a taller document, or a new network call. An agent trusting exit codes here would have confidently reported reading a list it never scrolled.
Crossing a login boundary
A public page is the easy case. I wanted to know whether state survives authentication, so I pointed the loop at saucedemo.com, a Sauce Labs sandbox with published credentials.
Here's what I got before logging in:
{"url":"https://www.saucedemo.com/","title":"Swag Labs",
"text":"[1]<div id=root />\n\t[2]<input placeholder=Username id=user-name />\n\t[3]<input placeholder=Password type=password />\n\t[4]<input id=login-button type=submit />"}
I typed into [2] and [3], then clicked [4]:
url=https://www.saucedemo.com/inventory.html
[2]<button id=react-burger-menu-btn /> Open Menu
[3]<a class=shopping_cart_link />
[9]<a id=item_4_title_link /> Sauce Labs Backpack $29.99
[10]<button id=add-to-cart-sauce-labs-backpack /> Add to cart
The URL moved, the indices renumbered against a completely different page, and the session carried authentication forward. [10] is now a real add-to-cart action, and nothing in my setup had to know in advance that a product grid was coming.
I did fumble this on the first attempt, which is worth passing on. I picked indices 1, 2, and 3 for username, password, and submit, when the correct mapping was 2, 3, and 4, because [1] was the root <div id=root /> wrapper. The state had shown me that correctly and I misread it. Re-reading state fixed it in a single turn, which I'd contrast against a stale selector, since that one needs a human to open the script and rewrite it.
The network capture then surfaced something I wouldn't have predicted:
json
{"url": "https://events.backtrace.io/api/unique-events/submit", "status": 401}
No authentication call at all. Sauce Demo does its auth client-side, so the only traffic I captured was Backtrace telemetry returning 401. If I'd built an agent that confirms login by watching for an auth response, it would have failed here silently, and one command told me that.
Running the same login in Playwright
Then I wrote the same flow as a script: goto, fill #user-name and #password, click #login-button, wait for the inventory URL. Headless Chromium, exit code 0, six products found.
Both tools passed. I'd add context, though, because Sauce Demo is a test sandbox with stable documented IDs, built so that selector-based scripts have something predictable to hit. It's the most favourable page Playwright will ever see.
What interested me more is what Sauce Demo couldn't test. It never challenged either tool, with no fingerprinting and no interstitial anywhere in the flow. So my run measured whether a form can be driven, and told me nothing about whether the browser reaches the page at all.
So I pointed both at deviceandbrowserinfo.com/are_you_a_bot. BrowserAct's stealth browser came back with this:
json
{"isBot": false, "details": {
"hasBotUserAgent": false, "hasWebdriverTrue": false,
"isPlaywright": false, "isHeadlessChrome": false,
"isAutomatedWithCDP": false, "hasSuspiciousWeakSignals": false}}
My Playwright script came back with this:
json
{"isBot": true, "details": {
"hasBotUserAgent": true, "hasWebdriverTrue": true,
"hasWebdriverInFrameTrue": true,
"isAutomatedWithCDP": true, "isAutomatedWithCDPInWebWorker": true}}
Five separate flags tripped. The user agent announced itself, navigator.webdriver read true in the main frame and inside frames, and the CDP automation was visible from both the page and a web worker. The page rendered "You are a bot" where the other run had said "You are human!"
I should be clear that this is default headless Playwright. Stealth plugins exist for exactly this reason and would change the result, so the fair framing is that hardening is work you take on yourself, while the stealth browser arrived that way.
bot.sannysoft.com agreed on the BrowserAct side, showing WebDriver missing, the Chrome object present, and PluginArray correct. I noticed one nuance in the fp-collect output, where webDriver: true sits alongside webDriverValue: false, and that reflects the property existing while set to false, which is standard Chrome rather than a leak.
| BrowserAct | Playwright (default) | |
|---|---|---|
| How it clicks | Index from live state | Selector written in advance |
| Needs the page mapped first | No | Yes |
| Sauce Demo login | Reached inventory | Reached inventory |
| Detection verdict | isBot: false |
isBot: true, five flags |
Two housekeeping notes from these runs. My stealth quota ran out partway through, so the scroll confirmation used local Chrome, though scroll behaviour is identical either way. And when I ran under WSL without sudo, the apt install of Chromium libraries failed, at which point BrowserAct unpacked the dependencies into ~/.local/share/ and loaded them through LD_LIBRARY_PATH rather than dying. I'd call that a good default, though expect a slow first run while it fetches.
When BrowserAct is the right tool
Four situations came up across these runs where BrowserAct did something a plain script would have struggled with, and I hit each one directly.
On quotes.toscrape.com/scroll I never wrote a selector. I read indexed state, acted on [2], and re-read after the page changed. When I point an agent at a site it hasn't seen, or at a UI that shifts between runs, that loop is what lets it keep going instead of stalling on a selector that no longer matches. This is the case for research and monitoring work, where the target is rarely a page you built.
The detection run is the one I keep coming back to. My stealth browser cleared deviceandbrowserinfo and sannysoft as human on the same page where my headless Playwright tripped five flags and got "You are a bot." An agent that reasons perfectly is useless when the site fingerprints it and never serves the page. On protected targets that's the reason I'd reach for the stealth browser rather than a raw headless script.
The SauceDemo login showed me the third. I authenticated, landed on /inventory.html, and the session carried forward into a page with real add-to-cart actions I hadn't seen before the login. For a dashboard check or an account operation, the part that only exists after login is the whole job, and the session held across that boundary without me scripting the transition.
The fourth came out of the network capture. On the quotes run it handed me the api/quotes endpoint the page was reading from, and on SauceDemo it showed me the auth was client-side with no login call at all. When the data I want arrives over XHR, reading it from the network beats parsing rendered cards, and it told me how each page actually worked rather than how it looked.
I'll keep one thing honest, though. If the page is one you own, the selectors are stable, and nothing is trying to keep you out, a plain Playwright script is simpler and cheaper than an agent loop. My SauceDemo login proved that much, since both tools passed and the script had less to reason about. BrowserAct earned its cost on the runs where the page was unfamiliar, protected, gated behind a session, or feeding data through a network call, and that describes production more often than it describes a sandbox.
Try it against one workflow
If your agent works in clean demos but falls over on real sites, the test that told me the most was a small one.
Point it at a single workflow you care about, then check the page's own evidence rather than the exit code.
That's where a browser layer either earns its keep or shows you that a script was fine all along.
Top comments (1)
Good tutorial