WebRunner

Cross-platform web automation: Selenium + Playwright, plus a JSON-driven action executor with batteries included.

PyPI Version Python Version License Documentation Status

繁體中文 | 简体中文

WebRunner (je_web_runner) started as a Selenium wrapper and grew into a full automation platform: a Selenium and a Playwright backend behind one JSON-driven action executor, plus modules for reporting, observability, orchestration, security, and AI assistance. Every executor command has a deterministic name (WR_*) and a single dispatch point, so an action JSON can mix browser, HTTP, database, and webhook calls in the same script.

Auto-generated reference — every registered WR_* command (signature + summary) is exported under docs/reference/command_reference.md, and a JSON Schema for action JSON files lives at docs/reference/webrunner-action-schema.json.

Table of Contents

Highlights

Installation

Stable:

pip install je_web_runner

Development:

pip install je_web_runner_dev

Optional dependencies (each enables a slice of features; install only what you use):

pip install playwright           # Playwright backend
python -m playwright install     # Browser binaries for Playwright
pip install Pillow               # Visual regression
pip install faker                # Random test data (WR_faker_*)
pip install sqlalchemy           # Database validation (WR_db_*)
pip install opentelemetry-sdk    # Distributed traces (WR_set_action_span_factory)
pip install Appium-Python-Client # Mobile native (WR_appium_*)
pip install testcontainers       # Spin up Postgres / Redis (WR_tc_*)
pip install locust               # Load testing (WR_locust_*)

Hard requirements: Python 3.10+, selenium>=4.0.0, requests, python-dotenv, webdriver-manager, defusedxml, Pillow.

Architecture

System overview

flowchart LR
  subgraph Authoring
    A1["Action JSON files"]
    A2["Programmatic Python API"]
    A3["Browser recorder<br/>(JS injection)"]
    A4["LLM NL → action draft"]
  end

  subgraph Core
    EXE["Executor<br/>event_dict"]
    REC["Test record<br/>singleton"]
    LDG["Run ledger /<br/>flaky detection"]
  end

  subgraph Backends
    SEL["Selenium<br/>WebDriverWrapper"]
    PW["Playwright<br/>PlaywrightWrapper"]
    APM["Appium<br/>Mobile"]
    HTTP["HTTP API<br/>requests"]
    DB["Database<br/>SQLAlchemy"]
  end

  subgraph Outputs
    REP["Reports<br/>HTML/JSON/XML/JUnit/Allure"]
    OBS["Observability<br/>OTel · dashboard · replay"]
    NOT["Notifiers<br/>Slack · webhook · GH · JIRA · TestRail"]
  end

  A1 --> EXE
  A2 --> EXE
  A3 --> A1
  A4 --> A1
  EXE --> SEL
  EXE --> PW
  EXE --> APM
  EXE --> HTTP
  EXE --> DB
  SEL --> REC
  PW --> REC
  APM --> REC
  HTTP --> REC
  DB --> REC
  REC --> LDG
  REC --> REP
  REC --> OBS
  REC --> NOT

Action lifecycle

flowchart LR
  IN["Action<br/>[cmd, args, kwargs]"] --> VAL["JSON validator<br/>(WR_validate_*)"]
  VAL --> ENV["${ENV.X} / ${ROW.x}<br/>placeholder expansion"]
  ENV --> SPAN["OTel span factory<br/>(optional)"]
  SPAN --> RETRY["Retry policy<br/>retries × backoff"]
  RETRY --> GATE["Arbitrary-script<br/>gate"]
  GATE --> DISP["event_dict[cmd](*args, **kwargs)"]
  DISP --> RECORD["test_record_instance<br/>append()"]
  DISP -- failure --> SHOT["Auto-screenshot<br/>(failure dir)"]
  RECORD --> DONE["Result dict"]
  SHOT --> DONE

Backend dispatch

flowchart TB
  CMD["Action command name"] --> ROUTE{"prefix?"}
  ROUTE -- "WR_pw_*" --> PW["Playwright backend<br/>(PlaywrightWrapper)"]
  ROUTE -- "WR_pw_element_*" --> PWE["Playwright element<br/>(PlaywrightElementWrapper)"]
  ROUTE -- "WR_appium_*" --> APM["Appium driver"]
  ROUTE -- "WR_http_*" --> HTTP["requests wrapper"]
  ROUTE -- "WR_db_*" --> DB["SQLAlchemy validator"]
  ROUTE -- "WR_pw_a11y_* / WR_a11y_*" --> AXE["axe-core audit"]
  ROUTE -- "WR_pw_throttle / WR_throttle" --> THR["Network throttling<br/>(CDP)"]
  ROUTE -- "WR_pw_route_*" --> ROUTE_MOCK["Playwright route mock"]
  ROUTE -- "WR_*<br/>(default)" --> SEL["Selenium backend<br/>(WebDriverWrapper)"]
  ROUTE -- "WR_element_*<br/>(default)" --> SE["Selenium element<br/>(WebElementWrapper)"]

Module map

je_web_runner/
├── __init__.py
├── __main__.py                    # CLI: --execute_dir / --watch / --tag / --shard / --migrate ...
├── element/web_element_wrapper.py
├── manager/webrunner_manager.py
├── webdriver/
│   ├── webdriver_wrapper.py             # Selenium core
│   ├── webdriver_with_options.py
│   ├── playwright_wrapper.py            # Playwright sync backend (full)
│   ├── playwright_element_wrapper.py
│   └── playwright_locator.py            # TestObject ↔ Playwright selector
└── utils/
    ├── ab_run/                  # A/B run mode (run_ab + diff_records)
    ├── accessibility/           # axe-core audit
    ├── ai_assist/               # Pluggable LLM scaffold
    ├── api/                     # HTTP API testing commands
    ├── appium_integration/      # Mobile native via Appium
    ├── auth/                    # OAuth2 / OIDC token helpers
    ├── callback/                # Callback executor
    ├── cdp/                     # Raw CDP passthrough
    ├── ci_annotations/          # GitHub Actions ::error::
    ├── cli/                     # CLI parser, watch mode, dispatch
    ├── cloud_grid/              # BrowserStack / Sauce Labs / LambdaTest
    ├── dashboard/               # Live progress HTTP server
    ├── database/                # SQL validation (SQLAlchemy)
    ├── data_driven/             # CSV/JSON dataset + ${ROW.x}
    ├── docs/                    # Auto-generated command reference
    ├── dom_traversal/           # Shadow DOM / iframe helpers
    ├── env_config/              # .env loader + ${ENV.X}
    ├── exception/               # Exception hierarchy
    ├── executor/                # Action executor + retry/screenshot/gate
    ├── extensions/              # Browser extension loaders
    ├── factories/               # Factory pattern helpers
    ├── file_process/            # File utilities
    ├── file_transfer/           # Upload / download helpers
    ├── generate_report/         # HTML/JSON/XML/JUnit/Allure + manifest
    ├── har_diff/                # HAR file diff
    ├── json/                    # JSON I/O + validator (length 1/2/3)
    ├── lighthouse/              # Lighthouse CLI runner
    ├── linter/                  # action_linter + migration
    ├── load_test/               # Locust wrapper
    ├── logging/                 # Rotating file handler
    ├── multi_user/              # Multi-user matrix runner
    ├── network_emulation/       # Throttling presets via CDP
    ├── notifier/                # Slack / generic webhooks
    ├── observability/           # Console+network capture · OTel
    ├── package_manager/         # Dynamic plugin loader
    ├── perf_metrics/            # FCP / LCP / CLS / TTFB
    ├── pom_generator/           # POM skeleton from URL/HTML
    ├── project/                 # Project template generator
    ├── recorder/                # JS-injection recorder + PII mask
    ├── replay_studio/           # HTML timeline studio
    ├── run_ledger/              # ledger · flaky · classifier
    ├── schema/                  # Action JSON Schema export
    ├── scheduler/               # stdlib-sched scheduled runner
    ├── secrets_scanner/         # Hard-coded credential scanner
    ├── security_headers/        # HTTP headers audit
    ├── selenium_utils_wrapper/  # Keys / Capabilities
    ├── self_healing/            # Fallback locator registry
    ├── service_worker/          # SW unregister + cache clear
    ├── sharding/                # Deterministic test sharding
    ├── snapshot/                # Text/DOM snapshot testing
    ├── socket_server/           # TCP server with token + TLS
    ├── storage/                 # localStorage / session / IDB
    ├── test_data/               # Faker integration
    ├── test_filter/             # Tag filter + dependency graph
    ├── test_management/         # JIRA + TestRail
    ├── test_object/             # TestObject + record
    ├── test_record/             # Action recording
    ├── testcontainers_integration/   # Postgres / Redis / generic
    ├── visual_regression/       # Pillow-based image diff
    └── xml/                     # XML utilities

Cookbook

The examples/ directory ships runnable recipes that exercise the new helpers against real Chrome / network. Each is invoked from the repo root:

Example Demonstrates
counting_stars.{py,json} WR_sleep, WR_set_driver with Chrome flags, autoplay-policy override, JS-driven video.play(), skip-ad polling.
google_search.py Consent dismissal, search-box typing, ENTER submit, result heading scrape.
form_submit.py form_autofill.plan_fill_actions + state_diff.capture_state round trip against httpbin/forms/post.
smart_wait_demo.py wait_for_fetch_idle + wait_for_spa_route_stable + memory_leak.detect_growth against a real page.
fanout_demo.py fanout.run_fan_out parallel HTTP preflights.
pii_redact_demo.py pii_scanner.scan_text + redact_text + assert_no_pii (pure logic).
quick_smoke.json Minimal WR_set_driver → WR_sleep → WR_execute_script → WR_quit_all smoke via the executor CLI.

Run a Python example directly:

python examples/google_search.py

Run an action JSON example through the executor:

python -m je_web_runner -e examples/quick_smoke.json

Test Tiers

test/
├── unit_test/         # 1200 mock-based unit tests (~12s)
├── integration_test/  #   30 wired-modules tests with real I/O (~6s)
└── e2e_test/          #    6 real-browser tests; skips without Selenium Grid

Thematic API Façade

The 80+ utility helpers live under je_web_runner.utils.<area>; for discoverability they are also re-exported under je_web_runner.api:

from je_web_runner.api import (
    authoring,      # action_formatter, md_authoring, templates, sel_to_pw, bootstrap
    debugging,      # cross_browser, pr_comment, extension_harness
    frontend,       # device emulation, geo/locale, multi-tab, shadow pierce, …
    infra,          # driver pin, k8s runner, pipeline, lock, watch_mode, …
    mobile,         # Appium gestures
    networking,     # api_mock, contract_testing, GraphQL, mock services, har_replay
    observability,  # timeline, failure bundle, trace recorder, OTLP, BiDi, cdp_tap
    quality,        # a11y_diff, a11y_trend, perf budgets/drift, trend, failure cluster
    reliability,    # adaptive retry, browser pool, smart wait, throttler, supervisor
    security,       # PII, license, CSP, cookie consent, header tampering
    test_data,      # DB fixtures, fixture record/replay, form auto-fill
)

The original Selenium-flavoured top-level surface (webdriver_wrapper_instance, execute_action, TestObject, …) is unchanged.

Quick Start

Direct API

from je_web_runner import TestObject, get_webdriver_manager, web_element_wrapper

manager = get_webdriver_manager("chrome")
manager.webdriver_wrapper.to_url("https://www.google.com")
manager.webdriver_wrapper.implicitly_wait(2)

search_box = TestObject("q", "name")
manager.webdriver_wrapper.find_element(search_box)
web_element_wrapper.click_element()
web_element_wrapper.input_to_element("WebRunner automation")

manager.quit()

JSON action list (modern aliases)

from je_web_runner import execute_action

actions = [
    ["WR_new_driver", {"webdriver_name": "chrome"}],
    ["WR_to_url", {"url": "https://www.google.com"}],
    ["WR_implicitly_wait", {"time_to_wait": 2}],
    ["WR_save_test_object", {"test_object_name": "q", "object_type": "NAME"}],
    ["WR_find_recorded_element", {"element_name": "q"}],
    ["WR_element_click"],
    ["WR_element_input", {"input_value": "WebRunner automation"}],
    ["WR_quit_all"],
]
execute_action(actions)

The legacy names (WR_get_webdriver_manager, WR_SaveTestObject, WR_quit, WR_input_to_element, …) still work — see Quality & Security for the one-shot migration helper.

Mixed positional + keyword arguments

[
    ["WR_to_url", ["https://example.com"], {"timeout": 30}],
]

The validator accepts length-1, length-2 ([cmd, dict_or_list]), and length-3 ([cmd, [positional], {kwargs}]) actions.

Core API

The original Selenium-flavoured API remains the canonical entry point for programmatic use. Sections preserved from the original README:

Programmatic examples for each surface are kept identical to the previous edition; see the relevant Sphinx pages under docs/source/Eng/doc/ for full code snippets.

Action Executor

The executor maps a string command name to a Python callable. Every backend, integration, and helper registers under event_dict.

Action shapes

["command"]                                    # no args
["command", {"key": "value"}]                  # kwargs
["command", [arg1, arg2]]                      # positional
["command", [arg1], {"key": "value"}]          # positional + kwargs (length 3)

Length-3 example

[
    ["WR_pw_evaluate", ["() => document.title"], {"arg": None}],
]

Pacing actions

WR_sleep blocks the executor thread for a given number of seconds — useful when the page needs settle time, when a JS animation needs to finish, or when an example wants to hold the browser open for the user to watch:

[
    ["WR_to_url", {"url": "https://example.com"}],
    ["WR_sleep", {"seconds": 2.5}],
    ["WR_get_screenshot_as_png"],
]

Negative or non-numeric seconds raise ValueError. For pacing inside JavaScript (e.g. waiting on a custom event from the page) use WR_execute_async_script with a setTimeout-driven callback.

Top-level shapes

[ ...actions... ]                                                  # bare list

{
  "webdriver_wrapper": [ ...actions... ],
  "meta": {"tags": ["smoke", "fast"], "depends_on": ["login"]}     # optional
}

meta.tags and meta.depends_on are picked up by the CLI for filtering and topological execution.

Adding custom commands

from je_web_runner import add_command_to_executor

def my_step(name: str) -> None:
    print(f"hello {name}")

add_command_to_executor({"my_command": my_step})

Retry, screenshots, scripts

from je_web_runner.utils.executor.action_executor import executor

executor.set_retry_policy(retries=2, backoff=0.5)             # global retry
executor.set_failure_screenshot_dir("./failures")              # auto PNG on raise
executor.set_allow_arbitrary_script(False)                     # gate WR_execute_script / WR_pw_evaluate / WR_cdp

Backends

Selenium (default)

Selenium is the original backend. Every legacy command (and its modern alias) routes here unless an explicit WR_pw_* / WR_appium_* prefix is used.

Playwright (full)

The Playwright backend mirrors the operational surface of the Selenium wrapper under WR_pw_*:

Existing scripts can move to Playwright incrementally; TestObject records are translated to Playwright selectors automatically (CSS_SELECTOR → as-is, XPATH → xpath=…, ID → #…, NAME → [name="…"], LINK_TEXT → text=…, PARTIAL_LINK_TEXT → :has-text("…")).

Cloud Grid

from je_web_runner import (
    connect_browserstack,
    build_browserstack_capabilities,
)

connect_browserstack(
    username="...",
    access_key="...",
    capabilities=build_browserstack_capabilities(
        browser_name="chrome",
        browser_version="latest",
        os_name="Windows",
        os_version="11",
        project="WebRunner",
        build="ci-2026-04-26",
    ),
)
# All existing WR_* commands now run against the cloud session.

connect_saucelabs and connect_lambdatest follow the same shape.

Appium (mobile)

from je_web_runner import (
    start_appium_session,
    build_android_caps,
    build_ios_caps,
)

start_appium_session(
    "https://appium.example/wd/hub",
    capabilities=build_android_caps(app="/path/to/app.apk"),
)
# WR_* commands now drive the mobile session.

Reports

from je_web_runner import (
    generate_html_report,
    generate_json_report,
    generate_xml_report,
    generate_junit_xml_report,
    generate_allure_report,
)
from je_web_runner.utils.generate_report.report_manifest import generate_all_reports

# Run every generator + write a manifest binding all outputs:
result = generate_all_reports("run_2026_04_26", allure_dir="allure-results")
print(result["manifest_path"])  # → run_2026_04_26.manifest.json
Format Output shape Spec-driven?
JSON <base>_success.json + <base>_failure.json split
HTML <base>.html single
XML <base>_success.xml + <base>_failure.xml split
JUnit XML <base>_junit.xml single
Allure <allure_dir>/<uuid>-result.json (× N) directory

The manifest captures the actual paths produced — CI globs no longer need to know the per-format conventions.

Observability

from je_web_runner import (
    test_record_instance,
    summarise_run,
    notify_run_summary,
)
from je_web_runner.utils.executor.action_executor import executor
from je_web_runner.utils.observability.otel_tracing import install_executor_tracing
from je_web_runner.utils.dashboard.live_dashboard import start_dashboard
from je_web_runner.utils.replay_studio.replay_studio import export_replay_studio

executor.set_failure_screenshot_dir("./failures")
install_executor_tracing("webrunner")                 # one OTel span per action
start_dashboard("127.0.0.1", 8080)                    # browser-friendly progress UI
test_record_instance.set_record_enable(True)

# … run actions …

export_replay_studio("./run.html", screenshot_dir="./failures")
notify_run_summary("https://hooks.slack.com/services/...")

Failure screenshot, OpenTelemetry tracing, retry policy, and the live dashboard all hook into the same Executor.event_dict so they compose without coupling.

Test Orchestration

# Filter by tag, run in parallel processes, persist a ledger, fail fast on dep breaks.
python -m je_web_runner \
    --execute_dir ./actions \
    --tag smoke,fast \
    --exclude-tag slow \
    --parallel 4 \
    --parallel-mode process \
    --ledger ./.run_ledger.json

# Re-run only the files that failed last time:
python -m je_web_runner --execute_dir ./actions --rerun-failed ./.run_ledger.json

# Watch a directory and re-run on file change:
python -m je_web_runner --execute_dir ./actions --watch ./actions

# Distribute across 4 runners deterministically (per machine):
python -m je_web_runner --execute_dir ./actions --shard 1/4
python -m je_web_runner --execute_dir ./actions --shard 2/4
python -m je_web_runner --execute_dir ./actions --shard 3/4
python -m je_web_runner --execute_dir ./actions --shard 4/4

Companion APIs — WR_run_for_users (multi-user matrix), WR_run_ab (A/B mode), WR_flakiness_stats, WR_classify_failure, WR_schedule + WR_run_scheduler_for.

Quality & Security

Extended Capabilities

Reliability & flake reduction:

Debugging & observability:

Test data & determinism:

API & contract testing:

Security probes:

Browser & locale:

Reporting & CI:

Orchestration & developer experience:

AI assistance:

MCP Server

WebRunner ships a Model Context Protocol server so any MCP-aware client (Claude, IDE plugins, etc.) can drive WebRunner over JSON-RPC stdio.

python -m je_web_runner.mcp_server

The default tool list (22 tools) exposes:

Live browser execution:

Plus the utility tools below (no live browser):

Action JSON authoring & linting:

Code generation:

Quality / triage:

Security & privacy:

Reporting & contract:

Sharding & infra:

from je_web_runner.mcp_server import McpServer, Tool, build_default_tools, serve_stdio

# Or build a custom server
server = McpServer()
for tool in build_default_tools():
    server.register(tool)
server.register(Tool(
    name="my_custom_tool",
    description="…",
    input_schema={"type": "object", "properties": {"x": {"type": "string"}}},
    handler=lambda args: f"hello {args['x']}",
))
serve_stdio(server=server)

The server speaks MCP 2024-11-05: initialize, tools/list, tools/call, resources/list, ping, shutdown.

Action JSON LSP

A standard Language Server Protocol implementation for action JSON files:

python -m je_web_runner.action_lsp

textDocument/completion returns every registered WR_* command; textDocument/publishDiagnostics runs the action linter on didOpen / didChange. Pair with VS Code’s Configure JSON Language Servers or the JetBrains LSP plugin.

Even More Capabilities (polish wave)

CLI & orchestration polish:

Frontend / mobile / coverage:

Even More Capabilities (final wave)

Debugging & reproducibility:

Authoring / scaffolding:

CI reproducibility:

Long-running observability:

Even More Capabilities (newest wave)

Authoring / formatting:

Triage / production observability:

Frontend / component:

Even More Capabilities (latest wave)

Onboarding / migration:

Test authoring:

Quality:

Performance / orchestration:

Browser internals:

Even More Capabilities

Reliability & dev-loop:

Determinism & offline runs:

Quality / privacy:

Test orchestration:

Advanced WebDriverWrapper

The Selenium wrapper is now composed via mixins under je_web_runner/webdriver/_wrapper_mixins/ (lifecycle / element / wait stay in webdriver_wrapper.py; cookies / actions / media / navigation / scripting are the mixin themes). External imports — webdriver_wrapper_instance, WebDriverWrapper, the _options_dict / _webdriver_dict / _webdriver_manager_dict patch targets — are unchanged.

Stealth / anti-bot launch

from je_web_runner import webdriver_wrapper_instance

webdriver_wrapper_instance.set_driver(
    "chrome",
    options=[
        "--disable-blink-features=AutomationControlled",
        f"--user-data-dir={profile_dir}",
    ],
    experimental_options={
        "excludeSwitches": ["enable-automation"],
        "useAutomationExtension": False,
    },
    extension_paths=["/path/to/extension.crx"],  # optional
    enable_bidi=True,                            # for add_console_listener etc.
)
webdriver_wrapper_instance.add_script_to_evaluate_on_new_document(
    "Object.defineProperty(navigator, 'webdriver', {get: () => undefined});"
)

Attach to a manually started Chrome (session reuse)

# Step 1 — user starts Chrome themselves:
#   chrome.exe --remote-debugging-port=9222 --user-data-dir="C:/temp/profile"
webdriver_wrapper_instance.attach_to_existing_browser("127.0.0.1:9222")

CDP shortcuts (emulation, network, downloads)

w = webdriver_wrapper_instance
w.set_timezone("Asia/Tokyo")
w.set_locale("ja-JP")
w.set_device_metrics(390, 844, device_scale_factor=3, mobile=True)
w.set_user_agent("Mozilla/5.0 (custom)")
w.set_extra_http_headers({"X-Test-Run": "ci-123"})
w.set_geolocation(35.68, 139.69, accuracy=50)
w.set_network_conditions(offline=False, latency=200,
                          download_throughput=50_000, upload_throughput=10_000)
w.block_urls(["*.doubleclick.net/*", "*.googletagmanager.com/*"])
w.set_cache_disabled(True)
w.set_download_directory("./downloads")
w.clear_origin_storage("https://example.com")    # cookies + localStorage + IDB + cache

Session persistence

w.to_url("https://example.com/")
# … log in, etc. …
w.save_cookies("./cookies.json")

# Later (after browser restart):
w.to_url("https://example.com/")
added = w.load_cookies("./cookies.json")          # → number of cookies applied

Fetch interception primitives

w.enable_fetch_interception(patterns=["*/api/*"])
# In a Fetch.requestPaused event callback (subscribe via CDPEventListener):
w.fulfill_request(req_id, response_code=200,
                  body=b'{"ok": true}',
                  response_headers={"Content-Type": "application/json"})
# Or:  w.continue_request(req_id, url=rewritten_url)
# Or:  w.fail_request(req_id, error_reason="AccessDenied")

Page metadata + multi-tab navigation

w.get_current_url(); w.get_title(); w.get_page_source()
w.get_window_handles(); w.get_current_window_handle()
w.new_window("tab")
w.switch_to_window_by_url("checkout")    # restores original if no match
w.close_window()                         # vs quit() which terminates the driver
w.reload(ignore_cache=True)              # CDP Page.reload — Ctrl+Shift+R equivalent
w.bring_to_front()
w.save_full_page_screenshot("./shot.png")   # full page, beyond viewport
w.print_page("./page.pdf")

W3C BiDi listeners (Selenium 4.16+)

webdriver_wrapper_instance.set_driver("chrome", enable_bidi=True)
sub_id = webdriver_wrapper_instance.add_console_listener(
    lambda entry: print(entry.text)
)
err_id = webdriver_wrapper_instance.add_js_error_listener(
    lambda err: print("page exception:", err)
)
# …later…
webdriver_wrapper_instance.remove_console_listener(sub_id)
webdriver_wrapper_instance.remove_js_error_listener(err_id)

Background CDP event loop (independent module)

CDPEventListener opens its own CDP WebSocket on a worker thread so commands and events share the same target session — required because Selenium’s execute_cdp_cmd cannot subscribe to events.

from je_web_runner import CDPEventListener

with CDPEventListener.from_driver(driver) as listener:
    listener.on("Fetch.requestPaused", handle_paused)
    listener.send("Fetch.enable", {"patterns": [{"urlPattern": "*"}]})
    # … drive the browser …

Requires pip install websocket-client (lazy-loaded; a clear CDPEventLoopError is raised if missing).

Performance tracing

from je_web_runner import record_trace

record_trace(
    driver, "perf.json",
    categories=["devtools.timeline", "loading"],
    duration=10.0,
)
# Open perf.json in chrome://tracing or DevTools "Performance".

Cross-browser BiDi network (Selenium 4.16+, Firefox-compatible)

from je_web_runner import (
    bidi_add_request_handler,
    bidi_add_response_handler,
    bidi_clear_network_handlers,
)

sub = bidi_add_request_handler(driver, lambda req: print(req.url))
bidi_clear_network_handlers(driver)

Every method above is also reachable from action JSON via WR_* aliases (WR_set_timezone, WR_save_cookies, WR_enable_fetch_interception, …) so the same surface drives the MCP server too.

Browser Internals

from je_web_runner import (
    selenium_cdp,                 # raw CDP
    pw_emulate, pw_set_locale,    # mobile / locale
)
from je_web_runner.utils.storage.browser_storage import (
    selenium_local_storage_set,
    selenium_indexed_db_drop,
)
from je_web_runner.utils.observability.event_capture import (
    start_event_capture,
    assert_no_console_errors,
    assert_no_5xx,
)
from je_web_runner.utils.dom_traversal.shadow_iframe import (
    selenium_query_in_shadow,
    playwright_shadow_selector,
    selenium_switch_iframe_chain,
)
from je_web_runner.utils.file_transfer.file_helpers import (
    selenium_upload_file,
    wait_for_download,
)
from je_web_runner.utils.extensions.extension_loader import (
    selenium_chrome_options_with_extension,
    playwright_extension_launch_args,
)

Service worker / cache control, console + network event capture and assertions, file upload via element + download dir watcher, browser extension loaders for Chromium-family.

Test Data

from je_web_runner import (
    load_env, get_env, expand_in_action,                   # .env + ${ENV.X}
    load_dataset_csv, load_dataset_json, run_with_dataset, # data-driven + ${ROW.x}
    fake_email, fake_name, fake_credit_card, fake_value,   # faker
)
from je_web_runner.utils.factories.factory import user_factory, order_factory
from je_web_runner.utils.testcontainers_integration.containers import (
    start_postgres,
    start_redis,
    cleanup_all,
)

Every helper is JSON-callable too (WR_load_env, WR_load_dataset_csv, WR_run_with_dataset, WR_faker_email, WR_user_factory, WR_tc_postgres, …).

Auth & APIs

from je_web_runner import (
    http_get, http_post, http_assert_status, http_assert_json_contains,
)
from je_web_runner.utils.auth.oauth import (
    client_credentials_token,
    bearer_header,
)
from je_web_runner.utils.database.db_validate import (
    db_query,
    db_assert_count,
    db_assert_value,
)

token = client_credentials_token(
    "https://idp.example/oauth2/token",
    "client-id", "client-secret",
    cache_key="default",
)
http_get("https://api.example/users/me", headers=bearer_header(token["access_token"]))
http_assert_status(200)
http_assert_json_contains("role", "admin")

db_assert_count(
    "postgresql+psycopg://user:pw@host/db",
    "SELECT 1 FROM orders WHERE user_id = :uid",
    expected=1,
    params={"uid": 42},
)

OAuth2 helpers cache tokens in-process and refresh 30 seconds before expiry.

Recorder

from je_web_runner import (
    recorder_start,
    recorder_stop,
    recorder_save_recording,
)

recorder_start(webdriver_wrapper_instance)
# … user clicks / inputs in the browser …
recorder_save_recording(
    webdriver_wrapper_instance,
    output_path="./recorded.json",
    raw_events_path="./raw.json",  # optional — debugging
)

The recorder injects a static JS listener (no CDP, no eval), so it works on Chrome / Firefox / Edge alike. Sensitive fields are masked by default — type=password, names matching password / card_number / cvv / ssn / secret / token / api_key / otp / passcode, and 13–19-digit numeric values are replaced with ***MASKED***.

CI / Integrations

from je_web_runner.utils.notifier.webhook_notifier import notify_run_summary
from je_web_runner.utils.test_management.jira_client import jira_create_failure_issues
from je_web_runner.utils.test_management.testrail_client import (
    testrail_send_results,
    testrail_results_from_pairs,
)
from je_web_runner.utils.ci_annotations.github_annotations import (
    emit_failure_annotations,
    emit_from_junit_xml,
)

For GitHub Actions inline annotations, run emit_from_junit_xml("run_junit.xml") after generate_junit_xml_report — failed test cases surface as ::error file=…:: lines on the PR diff.

docker/docker-compose.yml ships a Selenium Grid 4 stack (hub + Chrome + Firefox nodes); docker/.env.example exposes the version pin and concurrency settings.

The IDE config examples under docs/ide/ wire VS Code and JetBrains to the action JSON schema produced by WR_export_action_schema.

AI Assistance

from je_web_runner.utils.ai_assist.llm_assist import (
    set_llm_callable,
    suggest_locator,
    generate_actions_from_prompt,
)

# Plug in any callable that returns a string:
def my_llm(prompt: str) -> str:
    # call OpenAI / Anthropic / local Ollama / mock
    ...

set_llm_callable(my_llm)

locator = suggest_locator(html_blob, description="primary submit button")
draft = generate_actions_from_prompt("log in as alice and place an order")

WebRunner intentionally ships no built-in LLM client — the boundary is a single Callable[[str], str] so swapping provider is one line.

CLI Usage

# Original entry points (unchanged):
python -m je_web_runner -e actions.json
python -m je_web_runner -d ./actions/
python -m je_web_runner --execute_str '[["WR_quit_all"]]'

# Newer flags:
python -m je_web_runner -d ./actions --tag smoke --exclude-tag slow
python -m je_web_runner -d ./actions --parallel 4 --parallel-mode process
python -m je_web_runner -d ./actions --ledger ledger.json
python -m je_web_runner -d ./actions --rerun-failed ledger.json
python -m je_web_runner -d ./actions --shard 1/4
python -m je_web_runner -d ./actions --watch ./actions
python -m je_web_runner --report run                          # JSON + HTML + XML + JUnit
python -m je_web_runner --validate ./action_smoke.json
python -m je_web_runner --migrate ./actions --migrate-dry-run

Compose any of the flags above; the dispatcher applies tag filters → ledger / re-run-failed → sharding → dependency-aware ordering before handing files to the runner.

Test Record

from je_web_runner import test_record_instance

test_record_instance.set_record_enable(True)
# … perform automation …
records = test_record_instance.test_record_list
# Each record: {"function_name", "local_param", "time", "program_exception"}
test_record_instance.clean_record()

Exception Handling

WebRunner provides a hierarchy of custom exceptions — every helper raises a domain-specific subclass of WebRunnerException:

Exception Description
WebRunnerException Base
WebRunnerWebDriverNotFoundException WebDriver not found
WebRunnerOptionsWrongTypeException Invalid options type
WebRunnerArgumentWrongTypeException Invalid argument type
WebRunnerWebDriverIsNoneException WebDriver is None
WebRunnerExecuteException Action execution error
WebRunnerJsonException JSON processing error
WebRunnerGenerateJsonReportException JSON / XML / JUnit / Allure report error
WebRunnerHTMLException HTML report error
WebRunnerAddCommandException Custom command registration error
WebRunnerAssertException Assertion failure
XMLException / XMLTypeException XML processing error
CallbackExecutorException Callback execution error
PlaywrightBackendError Playwright backend / element failure
PlaywrightLocatorError TestObject → Playwright selector mapping
RecorderError / VisualRegressionError Recorder / visual regression
HealingError / EnvConfigError / DataDrivenError Self-healing / env / dataset
HttpAssertionError / HttpError HTTP API assertions
AccessibilityError / LighthouseError a11y / Lighthouse
NotifierError / JiraError / TestRailError Notifications / test management
CDPError / StorageError / ServiceWorkerError Browser internals
OAuthError / DatabaseValidationError Auth / DB
NetworkEmulationError / LoadTestError Throttling / Locust
ShardingError / MigrationError / ActionLinterError Orchestration / linting
LLMAssistError / OTelTracingError AI / observability

Logging

WebRunner uses a rotating file handler:

Supported Browsers

Browser Selenium key Playwright
Google Chrome chrome chromium
Chromium chromium chromium
Mozilla Firefox firefox firefox
Microsoft Edge edge chromium
Internet Explorer ie n/a
Apple Safari safari webkit

Supported Platforms

License

This project is licensed under the MIT License.

This site is open source. Improve this page.