WebRunner

Cross-platform web automation: Selenium + Playwright, plus a JSON-driven action executor with batteries included.

PyPI Version Python Version License Documentation Status

繁體中文 | 简体中文

WebRunner (je_web_runner) started as a Selenium wrapper and grew into a full automation platform: a Selenium and a Playwright backend behind one JSON-driven action executor, plus modules for reporting, observability, orchestration, security, and AI assistance. Every executor command has a deterministic name (WR_*) and a single dispatch point, so an action JSON can mix browser, HTTP, database, and webhook calls in the same script.

Auto-generated reference — every registered WR_* command (signature + summary) is exported under docs/reference/command_reference.md, and a JSON Schema for action JSON files lives at docs/reference/webrunner-action-schema.json.

Table of Contents

Highlights

Installation

Stable:

pip install je_web_runner

Development:

pip install je_web_runner_dev

Optional dependencies (each enables a slice of features; install only what you use):

pip install playwright           # Playwright backend
python -m playwright install     # Browser binaries for Playwright
pip install Pillow               # Visual regression
pip install faker                # Random test data (WR_faker_*)
pip install sqlalchemy           # Database validation (WR_db_*)
pip install opentelemetry-sdk    # Distributed traces (WR_set_action_span_factory)
pip install Appium-Python-Client # Mobile native (WR_appium_*)
pip install testcontainers       # Spin up Postgres / Redis (WR_tc_*)
pip install locust               # Load testing (WR_locust_*)

Hard requirements: Python 3.10+, selenium>=4.0.0, requests, python-dotenv, webdriver-manager, defusedxml, Pillow.

Architecture

System overview

flowchart LR
  subgraph Authoring
    A1["Action JSON files"]
    A2["Programmatic Python API"]
    A3["Browser recorder<br/>(JS injection)"]
    A4["LLM NL → action draft"]
  end

  subgraph Core
    EXE["Executor<br/>event_dict"]
    REC["Test record<br/>singleton"]
    LDG["Run ledger /<br/>flaky detection"]
  end

  subgraph Backends
    SEL["Selenium<br/>WebDriverWrapper"]
    PW["Playwright<br/>PlaywrightWrapper"]
    APM["Appium<br/>Mobile"]
    HTTP["HTTP API<br/>requests"]
    DB["Database<br/>SQLAlchemy"]
  end

  subgraph Outputs
    REP["Reports<br/>HTML/JSON/XML/JUnit/Allure"]
    OBS["Observability<br/>OTel · dashboard · replay"]
    NOT["Notifiers<br/>Slack · webhook · GH · JIRA · TestRail"]
  end

  A1 --> EXE
  A2 --> EXE
  A3 --> A1
  A4 --> A1
  EXE --> SEL
  EXE --> PW
  EXE --> APM
  EXE --> HTTP
  EXE --> DB
  SEL --> REC
  PW --> REC
  APM --> REC
  HTTP --> REC
  DB --> REC
  REC --> LDG
  REC --> REP
  REC --> OBS
  REC --> NOT

Action lifecycle

flowchart LR
  IN["Action<br/>[cmd, args, kwargs]"] --> VAL["JSON validator<br/>(WR_validate_*)"]
  VAL --> ENV["${ENV.X} / ${ROW.x}<br/>placeholder expansion"]
  ENV --> SPAN["OTel span factory<br/>(optional)"]
  SPAN --> RETRY["Retry policy<br/>retries × backoff"]
  RETRY --> GATE["Arbitrary-script<br/>gate"]
  GATE --> DISP["event_dict[cmd](*args, **kwargs)"]
  DISP --> RECORD["test_record_instance<br/>append()"]
  DISP -- failure --> SHOT["Auto-screenshot<br/>(failure dir)"]
  RECORD --> DONE["Result dict"]
  SHOT --> DONE

Backend dispatch

flowchart TB
  CMD["Action command name"] --> ROUTE{"prefix?"}
  ROUTE -- "WR_pw_*" --> PW["Playwright backend<br/>(PlaywrightWrapper)"]
  ROUTE -- "WR_pw_element_*" --> PWE["Playwright element<br/>(PlaywrightElementWrapper)"]
  ROUTE -- "WR_appium_*" --> APM["Appium driver"]
  ROUTE -- "WR_http_*" --> HTTP["requests wrapper"]
  ROUTE -- "WR_db_*" --> DB["SQLAlchemy validator"]
  ROUTE -- "WR_pw_a11y_* / WR_a11y_*" --> AXE["axe-core audit"]
  ROUTE -- "WR_pw_throttle / WR_throttle" --> THR["Network throttling<br/>(CDP)"]
  ROUTE -- "WR_pw_route_*" --> ROUTE_MOCK["Playwright route mock"]
  ROUTE -- "WR_*<br/>(default)" --> SEL["Selenium backend<br/>(WebDriverWrapper)"]
  ROUTE -- "WR_element_*<br/>(default)" --> SE["Selenium element<br/>(WebElementWrapper)"]

Module map

Every utils/ subpackage with a one-line summary, grouped into the core engine, the je_web_runner.api facade themes and the rest: docs/reference/utils_index.md (generated by scripts/gen_utils_index.py).

je_web_runner/
├── __init__.py
├── __main__.py                    # CLI: --execute_dir / --watch / --tag / --shard / --migrate ...
├── element/web_element_wrapper.py
├── manager/webrunner_manager.py
├── webdriver/
│   ├── webdriver_wrapper.py             # Selenium core
│   ├── webdriver_with_options.py
│   ├── playwright_wrapper.py            # Playwright sync backend (full)
│   ├── playwright_element_wrapper.py
│   └── playwright_locator.py            # TestObject ↔ Playwright selector
└── utils/
    ├── ab_run/                  # A/B run mode (run_ab + diff_records)
    ├── accessibility/           # axe-core audit
    ├── ai_assist/               # Pluggable LLM scaffold
    ├── api/                     # HTTP API testing commands
    ├── appium_integration/      # Mobile native via Appium
    ├── auth/                    # OAuth2 / OIDC token helpers
    ├── callback/                # Callback executor
    ├── cdp/                     # Raw CDP passthrough
    ├── ci_annotations/          # GitHub Actions ::error::
    ├── cli/                     # CLI parser, watch mode, dispatch
    ├── cloud_grid/              # BrowserStack / Sauce Labs / LambdaTest
    ├── dashboard/               # Live progress HTTP server
    ├── database/                # SQL validation (SQLAlchemy)
    ├── data_driven/             # CSV/JSON dataset + ${ROW.x}
    ├── docs/                    # Auto-generated command reference
    ├── dom_traversal/           # Shadow DOM / iframe helpers
    ├── env_config/              # .env loader + ${ENV.X}
    ├── exception/               # Exception hierarchy
    ├── executor/                # Action executor + retry/screenshot/gate
    ├── extensions/              # Browser extension loaders
    ├── factories/               # Factory pattern helpers
    ├── file_process/            # File utilities
    ├── file_transfer/           # Upload / download helpers
    ├── generate_report/         # HTML/JSON/XML/JUnit/Allure + manifest
    ├── har_diff/                # HAR file diff
    ├── json/                    # JSON I/O + validator (length 1/2/3)
    ├── lighthouse/              # Lighthouse CLI runner
    ├── linter/                  # action_linter + migration
    ├── load_test/               # Locust wrapper
    ├── logging/                 # Rotating file handler
    ├── multi_user/              # Multi-user matrix runner
    ├── network_emulation/       # Throttling presets via CDP
    ├── notifier/                # Slack / generic webhooks
    ├── observability/           # Console+network capture · OTel
    ├── package_manager/         # Dynamic plugin loader
    ├── perf_metrics/            # FCP / LCP / CLS / TTFB
    ├── pom_generator/           # POM skeleton from URL/HTML
    ├── project/                 # Project template generator
    ├── recorder/                # JS-injection recorder + PII mask
    ├── replay_studio/           # HTML timeline studio
    ├── run_ledger/              # ledger · flaky · classifier
    ├── schema/                  # Action JSON Schema export
    ├── scheduler/               # stdlib-sched scheduled runner
    ├── secrets_scanner/         # Hard-coded credential scanner
    ├── security_headers/        # HTTP headers audit
    ├── selenium_utils_wrapper/  # Keys / Capabilities
    ├── self_healing/            # Fallback locator registry
    ├── service_worker/          # SW unregister + cache clear
    ├── sharding/                # Deterministic test sharding
    ├── snapshot/                # Text/DOM snapshot testing
    ├── socket_server/           # TCP server with token + TLS
    ├── storage/                 # localStorage / session / IDB
    ├── test_data/               # Faker integration
    ├── test_filter/             # Tag filter + dependency graph
    ├── test_management/         # JIRA + TestRail
    ├── test_object/             # TestObject + record
    ├── test_record/             # Action recording
    ├── testcontainers_integration/   # Postgres / Redis / generic
    ├── visual_regression/       # Pillow-based image diff
    └── xml/                     # XML utilities

Cookbook

The examples/ directory ships runnable recipes that exercise the new helpers against real Chrome / network. Each is invoked from the repo root:

Example Demonstrates
counting_stars.{py,json} WR_sleep, WR_set_driver with Chrome flags, autoplay-policy override, JS-driven video.play(), skip-ad polling.
google_search.py Consent dismissal, search-box typing, ENTER submit, result heading scrape.
form_submit.py form_autofill.plan_fill_actions + state_diff.capture_state round trip against httpbin/forms/post.
smart_wait_demo.py wait_for_fetch_idle + wait_for_spa_route_stable + memory_leak.detect_growth against a real page.
fanout_demo.py fanout.run_fan_out parallel HTTP preflights.
pii_redact_demo.py pii_scanner.scan_text + redact_text + assert_no_pii (pure logic).
quick_smoke.json Minimal WR_set_driver → WR_sleep → WR_execute_script → WR_quit_all smoke via the executor CLI.

Run a Python example directly:

python examples/google_search.py

Run an action JSON example through the executor:

python -m je_web_runner -e examples/quick_smoke.json

Test Tiers

test/
├── unit_test/         # mock-based unit tests
├── integration_test/  # wired-modules tests with real I/O
└── e2e_test/          # real-browser tests; skips without Selenium Grid

Thematic API Façade

The 80+ utility helpers live under je_web_runner.utils.<area>; for discoverability they are also re-exported under je_web_runner.api:

from je_web_runner.api import (
    authoring,      # action_formatter, md_authoring, templates, sel_to_pw, bootstrap
    debugging,      # cross_browser, pr_comment, extension_harness
    frontend,       # device emulation, geo/locale, multi-tab, shadow pierce, …
    infra,          # driver pin, k8s runner, pipeline, lock, watch_mode, …
    mobile,         # Appium gestures
    networking,     # api_mock, contract_testing, GraphQL, mock services, har_replay
    observability,  # timeline, failure bundle, trace recorder, OTLP, BiDi, cdp_tap
    quality,        # a11y_diff, a11y_trend, perf budgets/drift, trend, failure cluster
    reliability,    # adaptive retry, browser pool, smart wait, throttler, supervisor
    security,       # PII, license, CSP, cookie consent, header tampering
    test_data,      # DB fixtures, fixture record/replay, form auto-fill
)

The original Selenium-flavoured top-level surface (webdriver_wrapper_instance, execute_action, TestObject, …) is unchanged.

Quick Start

Direct API

from je_web_runner import TestObject, get_webdriver_manager, web_element_wrapper

manager = get_webdriver_manager("chrome")
manager.webdriver_wrapper.to_url("https://www.google.com")
manager.webdriver_wrapper.implicitly_wait(2)

search_box = TestObject("q", "name")
manager.webdriver_wrapper.find_element(search_box)
web_element_wrapper.click_element()
web_element_wrapper.input_to_element("WebRunner automation")

manager.quit()

JSON action list (modern aliases)

from je_web_runner import execute_action

actions = [
    ["WR_new_driver", {"webdriver_name": "chrome"}],
    ["WR_to_url", {"url": "https://www.google.com"}],
    ["WR_implicitly_wait", {"time_to_wait": 2}],
    ["WR_save_test_object", {"test_object_name": "q", "object_type": "NAME"}],
    ["WR_find_recorded_element", {"element_name": "q"}],
    ["WR_element_click"],
    ["WR_element_input", {"input_value": "WebRunner automation"}],
    ["WR_quit_all"],
]
execute_action(actions)

The legacy names (WR_get_webdriver_manager, WR_SaveTestObject, WR_quit, WR_input_to_element, …) still work — see Quality & Security for the one-shot migration helper.

Mixed positional + keyword arguments

[
    ["WR_to_url", ["https://example.com"], {"timeout": 30}],
]

The validator accepts length-1, length-2 ([cmd, dict_or_list]), and length-3 ([cmd, [positional], {kwargs}]) actions.

Public API & Deprecation Policy

What is public — these are the only things other code should depend on, and the only things this policy protects:

Surface What it covers
Top-level names Everything listed in je_web_runner.__all__ (from je_web_runner import …)
CLI python -m je_web_runner flags, including the original -e/--execute_file, -d/--execute_dir and --execute_str (Windows double-encoded JSON included)
Action JSON The WR_* command names registered in executor.event_dict, and the webdriver_wrapper / meta top-level keys of an action file
Socket server start_web_runner_socket_server, send_command, read_frame, encode_frame and the length-prefixed framing
Supported module paths je_web_runner.utils.executor.action_executor (executor), je_web_runner.utils.logging.loggin_instance (web_runner_logger, WebRunnerLoggingHandler), je_web_runner.webdriver.webdriver_wrapper (WebDriverWrapper, webdriver_wrapper_instance, the _options_dict / _webdriver_dict / _webdriver_manager_dict patch targets)

The three module paths are supported because other repositories already import them: AutoControlGUI’s WebRunner bridge takes executor, Jeffrey_RPA hooks loggin_instance by that exact (misspelled) name and patches the wrapper dictionaries. They are treated as public rather than asked to move. test/unit_test/test_public_api.py fails if any of them disappears.

What is not public: every other module under je_web_runner.utils., anything named with a leading underscore other than the patch targets above, the shape of report files, and the log format. They can move in any release.

Retiring something public

  1. The replacement ships first, and the old name keeps working as an alias.
  2. The alias raises DeprecationWarning naming the replacement, and the release notes say so.
  3. The alias stays for at least two further releases before it can be removed, and only a release that names the removal may drop it.
  4. A name listed in architecture.md §6 as a cross-project contract is different: it never changes without the consuming repository changing in the same round, and §6 is updated on both sides.

Core API

The original Selenium-flavoured API remains the canonical entry point for programmatic use. Sections preserved from the original README:

Programmatic examples for each surface are kept identical to the previous edition; see the relevant Sphinx pages under docs/source/Eng/doc/ for full code snippets.

Action Executor

The executor maps a string command name to a Python callable. Every backend, integration, and helper registers under event_dict.

Action shapes

["command"]                                    # no args
["command", {"key": "value"}]                  # kwargs
["command", [arg1, arg2]]                      # positional
["command", [arg1], {"key": "value"}]          # positional + kwargs (length 3)

Length-3 example

[
    ["WR_pw_evaluate", ["() => document.title"], {"arg": None}],
]

Pacing actions

WR_sleep blocks the executor thread for a given number of seconds — useful when the page needs settle time, when a JS animation needs to finish, or when an example wants to hold the browser open for the user to watch:

[
    ["WR_to_url", {"url": "https://example.com"}],
    ["WR_sleep", {"seconds": 2.5}],
    ["WR_get_screenshot_as_png"],
]

Negative or non-numeric seconds raise ValueError. For pacing inside JavaScript (e.g. waiting on a custom event from the page) use WR_execute_async_script with a setTimeout-driven callback.

Top-level shapes

[ ...actions... ]                                                  # bare list

{
  "webdriver_wrapper": [ ...actions... ],
  "meta": {"tags": ["smoke", "fast"], "depends_on": ["login"]}     # optional
}

meta.tags and meta.depends_on are picked up by the CLI for filtering and topological execution.

Adding custom commands

from je_web_runner import add_command_to_executor

def my_step(name: str) -> None:
    print(f"hello {name}")

add_command_to_executor({"my_command": my_step})

Retry, screenshots, scripts

from je_web_runner.utils.executor.action_executor import executor

executor.set_retry_policy(retries=2, backoff=0.5)             # global retry
executor.set_failure_screenshot_dir("./failures")              # auto PNG on raise
executor.set_allow_arbitrary_script(False)                     # gate WR_execute_script / WR_pw_evaluate / WR_cdp

Backends

Selenium (default)

Selenium is the original backend. Every legacy command (and its modern alias) routes here unless an explicit WR_pw_* / WR_appium_* prefix is used.

Playwright (full)

The Playwright backend mirrors the operational surface of the Selenium wrapper under WR_pw_*:

Existing scripts can move to Playwright incrementally; TestObject records are translated to Playwright selectors automatically (CSS_SELECTOR → as-is, XPATH → xpath=…, ID → #…, NAME → [name="…"], LINK_TEXT → text=…, PARTIAL_LINK_TEXT → :has-text("…")).

Cloud Grid

from je_web_runner.utils.cloud_grid.cloud_drivers import (
    connect_browserstack,
    build_browserstack_capabilities,
)

connect_browserstack(
    username="...",
    access_key="...",
    capabilities=build_browserstack_capabilities(
        browser_name="chrome",
        browser_version="latest",
        os_name="Windows",
        os_version="11",
        project="WebRunner",
        build="ci-2026-04-26",
    ),
)
# All existing WR_* commands now run against the cloud session.

connect_saucelabs and connect_lambdatest follow the same shape.

Appium (mobile)

from je_web_runner.utils.appium_integration.appium_driver import (
    start_appium_session,
    build_android_caps,
    build_ios_caps,
)

start_appium_session(
    "https://appium.example/wd/hub",
    capabilities=build_android_caps(app="/path/to/app.apk"),
)
# WR_* commands now drive the mobile session.

Reports

from je_web_runner import (
    generate_html_report,
    generate_json_report,
    generate_xml_report,
    generate_junit_xml_report,
    generate_allure_report,
)
from je_web_runner.utils.generate_report.report_manifest import generate_all_reports

# Run every generator + write a manifest binding all outputs:
result = generate_all_reports("run_2026_04_26", allure_dir="allure-results")
print(result["manifest_path"])  # → run_2026_04_26.manifest.json
Format Output shape Spec-driven?
JSON <base>_success.json + <base>_failure.json split
HTML <base>.html single
XML <base>_success.xml + <base>_failure.xml split
JUnit XML <base>_junit.xml single
Allure <allure_dir>/<uuid>-result.json (× N) directory

The manifest captures the actual paths produced — CI globs no longer need to know the per-format conventions.

Observability

from je_web_runner import (
    test_record_instance,
    summarise_run,
    notify_run_summary,
)
from je_web_runner.utils.executor.action_executor import executor
from je_web_runner.utils.observability.otel_tracing import install_executor_tracing
from je_web_runner.utils.dashboard.live_dashboard import start_dashboard
from je_web_runner.utils.replay_studio.replay_studio import export_replay_studio

executor.set_failure_screenshot_dir("./failures")
install_executor_tracing("webrunner")                 # one OTel span per action
start_dashboard("127.0.0.1", 8080)                    # browser-friendly progress UI
test_record_instance.set_record_enable(True)

# … run actions …

export_replay_studio("./run.html", screenshot_dir="./failures")
notify_run_summary("https://hooks.slack.com/services/...")

Failure screenshot, OpenTelemetry tracing, retry policy, and the live dashboard all hook into the same Executor.event_dict so they compose without coupling.

Test Orchestration

# Filter by tag, run in parallel processes, persist a ledger, fail fast on dep breaks.
python -m je_web_runner \
    --execute_dir ./actions \
    --tag smoke,fast \
    --exclude-tag slow \
    --parallel 4 \
    --parallel-mode process \
    --ledger ./.run_ledger.json

# Re-run only the files that failed last time:
python -m je_web_runner --execute_dir ./actions --rerun-failed ./.run_ledger.json

# Watch a directory and re-run on file change:
python -m je_web_runner --execute_dir ./actions --watch ./actions

# Distribute across 4 runners deterministically (per machine):
python -m je_web_runner --execute_dir ./actions --shard 1/4
python -m je_web_runner --execute_dir ./actions --shard 2/4
python -m je_web_runner --execute_dir ./actions --shard 3/4
python -m je_web_runner --execute_dir ./actions --shard 4/4

Companion APIs — WR_run_for_users (multi-user matrix), WR_run_ab (A/B mode), WR_flakiness_stats, WR_classify_failure, WR_schedule + WR_run_scheduler_for.

Quality & Security

Extended Capabilities

Reliability & flake reduction:

Debugging & observability:

Test data & determinism:

API & contract testing:

Security probes:

Browser & locale:

Reporting & CI:

Orchestration & developer experience:

AI assistance:

MCP Server

WebRunner ships a Model Context Protocol server so any MCP-aware client (Claude, IDE plugins, etc.) can drive WebRunner over JSON-RPC stdio.

python -m je_web_runner.mcp_server

The default tool list (22 tools) exposes:

Live browser execution:

Plus the utility tools below (no live browser):

Action JSON authoring & linting:

Code generation:

Quality / triage:

Security & privacy:

Reporting & contract:

Sharding & infra:

from je_web_runner.mcp_server import McpServer, Tool, build_default_tools, serve_stdio

# Or build a custom server
server = McpServer()
for tool in build_default_tools():
    server.register(tool)
server.register(Tool(
    name="my_custom_tool",
    description="…",
    input_schema={"type": "object", "properties": {"x": {"type": "string"}}},
    handler=lambda args: f"hello {args['x']}",
))
serve_stdio(server=server)

The server speaks MCP 2024-11-05: initialize, tools/list, tools/call, resources/list, ping, shutdown.

Action JSON LSP

A standard Language Server Protocol implementation for action JSON files:

python -m je_web_runner.action_lsp

textDocument/completion returns every registered WR_* command; textDocument/publishDiagnostics runs the action linter on didOpen / didChange. Pair with VS Code’s Configure JSON Language Servers or the JetBrains LSP plugin.

More Capabilities

Smaller modules, grouped by what they help with. The larger areas have their own sections above and in Specialized Modules.

CLI & orchestration polish

Frontend / mobile / coverage

Debugging & reproducibility

Authoring / scaffolding

CI reproducibility

Long-running observability

Authoring / formatting

Triage / production observability

Frontend / component

Onboarding / migration

Test authoring

Quality

Performance / orchestration

Browser internals

Reliability & dev-loop

Determinism & offline runs

Quality / privacy

Test orchestration

Specialized Modules

A second wave of utility modules, each in its own subpackage under je_web_runner/utils/, organised by capability area. Each module is fully unit-tested and ships independent of the core executor (import only what you use).

Web Platform APIs

Security & Headers

Performance Budgets

Backend Integration

AI / Workflow

a11y / i18n / Visual

Governance & Reporting

Other Specialised Modules

Modern web platform & runtime APIs

Modules covering newer browser surfaces that are awkward to drive through plain WebDriver:

Modern auth, payments, identity

Mobile-web specific

LLM / AI feature testing

Email & notification delivery

Performance budgets (cont.)

Security & headers (cont.)

Backend integration (cont.)

QA governance & DevX (cont.)

i18n / a11y (cont.)

Emerging-tech device APIs

For per-module reference also see CLAUDE.md, the auto-generated docs/reference/command_reference.md, and the Sphinx chapter under docs/source/Eng/doc/specialized_modules/.

Advanced WebDriverWrapper

The Selenium wrapper is now composed via mixins under je_web_runner/webdriver/_wrapper_mixins/ (lifecycle / element / wait stay in webdriver_wrapper.py; cookies / actions / media / navigation / scripting are the mixin themes). External imports — webdriver_wrapper_instance, WebDriverWrapper, the _options_dict / _webdriver_dict / _webdriver_manager_dict patch targets — are unchanged.

Stealth / anti-bot launch

from je_web_runner import webdriver_wrapper_instance

webdriver_wrapper_instance.set_driver(
    "chrome",
    options=[
        "--disable-blink-features=AutomationControlled",
        f"--user-data-dir={profile_dir}",
    ],
    experimental_options={
        "excludeSwitches": ["enable-automation"],
        "useAutomationExtension": False,
    },
    extension_paths=["/path/to/extension.crx"],  # optional
    enable_bidi=True,                            # for add_console_listener etc.
)
webdriver_wrapper_instance.add_script_to_evaluate_on_new_document(
    "Object.defineProperty(navigator, 'webdriver', {get: () => undefined});"
)

Attach to a manually started Chrome (session reuse)

# Step 1 — user starts Chrome themselves:
#   chrome.exe --remote-debugging-port=9222 --user-data-dir="C:/temp/profile"
webdriver_wrapper_instance.attach_to_existing_browser("127.0.0.1:9222")

CDP shortcuts (emulation, network, downloads)

w = webdriver_wrapper_instance
w.set_timezone("Asia/Tokyo")
w.set_locale("ja-JP")
w.set_device_metrics(390, 844, device_scale_factor=3, mobile=True)
w.set_user_agent("Mozilla/5.0 (custom)")
w.set_extra_http_headers({"X-Test-Run": "ci-123"})
w.set_geolocation(35.68, 139.69, accuracy=50)
w.set_network_conditions(offline=False, latency=200,
                          download_throughput=50_000, upload_throughput=10_000)
w.block_urls(["*.doubleclick.net/*", "*.googletagmanager.com/*"])
w.set_cache_disabled(True)
w.set_download_directory("./downloads")
w.clear_origin_storage("https://example.com")    # cookies + localStorage + IDB + cache

Session persistence

w.to_url("https://example.com/")
# … log in, etc. …
w.save_cookies("./cookies.json")

# Later (after browser restart):
w.to_url("https://example.com/")
added = w.load_cookies("./cookies.json")          # → number of cookies applied

Fetch interception primitives

w.enable_fetch_interception(patterns=["*/api/*"])
# In a Fetch.requestPaused event callback (subscribe via CDPEventListener):
w.fulfill_request(req_id, response_code=200,
                  body=b'{"ok": true}',
                  response_headers={"Content-Type": "application/json"})
# Or:  w.continue_request(req_id, url=rewritten_url)
# Or:  w.fail_request(req_id, error_reason="AccessDenied")

Page metadata + multi-tab navigation

w.get_current_url(); w.get_title(); w.get_page_source()
w.get_window_handles(); w.get_current_window_handle()
w.new_window("tab")
w.switch_to_window_by_url("checkout")    # restores original if no match
w.close_window()                         # vs quit() which terminates the driver
w.reload(ignore_cache=True)              # CDP Page.reload — Ctrl+Shift+R equivalent
w.bring_to_front()
w.save_full_page_screenshot("./shot.png")   # full page, beyond viewport
w.print_page("./page.pdf")

W3C BiDi listeners (Selenium 4.16+)

webdriver_wrapper_instance.set_driver("chrome", enable_bidi=True)
sub_id = webdriver_wrapper_instance.add_console_listener(
    lambda entry: print(entry.text)
)
err_id = webdriver_wrapper_instance.add_js_error_listener(
    lambda err: print("page exception:", err)
)
# …later…
webdriver_wrapper_instance.remove_console_listener(sub_id)
webdriver_wrapper_instance.remove_js_error_listener(err_id)

Background CDP event loop (independent module)

CDPEventListener opens its own CDP WebSocket on a worker thread so commands and events share the same target session — required because Selenium’s execute_cdp_cmd cannot subscribe to events.

from je_web_runner import CDPEventListener

with CDPEventListener.from_driver(driver) as listener:
    listener.on("Fetch.requestPaused", handle_paused)
    listener.send("Fetch.enable", {"patterns": [{"urlPattern": "*"}]})
    # … drive the browser …

Requires pip install websocket-client (lazy-loaded; a clear CDPEventLoopError is raised if missing).

Performance tracing

from je_web_runner import record_trace

record_trace(
    driver, "perf.json",
    categories=["devtools.timeline", "loading"],
    duration=10.0,
)
# Open perf.json in chrome://tracing or DevTools "Performance".

Cross-browser BiDi network (Selenium 4.16+, Firefox-compatible)

from je_web_runner import (
    bidi_add_request_handler,
    bidi_add_response_handler,
    bidi_clear_network_handlers,
)

sub = bidi_add_request_handler(driver, lambda req: print(req.url))
bidi_clear_network_handlers(driver)

Every method above is also reachable from action JSON via WR_* aliases (WR_set_timezone, WR_save_cookies, WR_enable_fetch_interception, …) so the same surface drives the MCP server too.

Browser Internals

from je_web_runner import (
    selenium_cdp,                 # raw CDP
    pw_emulate, pw_set_locale,    # mobile / locale
)
from je_web_runner.utils.storage.browser_storage import (
    selenium_local_storage_set,
    selenium_indexed_db_drop,
)
from je_web_runner.utils.observability.event_capture import (
    start_event_capture,
    assert_no_console_errors,
    assert_no_5xx,
)
from je_web_runner.utils.dom_traversal.shadow_iframe import (
    selenium_query_in_shadow,
    playwright_shadow_selector,
    selenium_switch_iframe_chain,
)
from je_web_runner.utils.file_transfer.file_helpers import (
    selenium_upload_file,
    wait_for_download,
)
from je_web_runner.utils.extensions.extension_loader import (
    selenium_chrome_options_with_extension,
    playwright_extension_launch_args,
)

Service worker / cache control, console + network event capture and assertions, file upload via element + download dir watcher, browser extension loaders for Chromium-family.

Test Data

from je_web_runner import (
    load_env, get_env, expand_in_action,                   # .env + ${ENV.X}
    load_dataset_csv, load_dataset_json, run_with_dataset, # data-driven + ${ROW.x}
)
from je_web_runner.utils.test_data.faker_integration import (
    fake_email, fake_name, fake_credit_card, fake_value,   # faker
)
from je_web_runner.utils.factories.factory import user_factory, order_factory
from je_web_runner.utils.testcontainers_integration.containers import (
    start_postgres,
    start_redis,
    cleanup_all,
)

Every helper is JSON-callable too (WR_load_env, WR_load_dataset_csv, WR_run_with_dataset, WR_faker_email, WR_user_factory, WR_tc_postgres, …).

Auth & APIs

from je_web_runner import (
    http_get, http_post, http_assert_status, http_assert_json_contains,
)
from je_web_runner.utils.auth.oauth import (
    client_credentials_token,
    bearer_header,
)
from je_web_runner.utils.database.db_validate import (
    db_query,
    db_assert_count,
    db_assert_value,
)

token = client_credentials_token(
    "https://idp.example/oauth2/token",
    "client-id", "client-secret",
    cache_key="default",
)
http_get("https://api.example/users/me", headers=bearer_header(token["access_token"]))
http_assert_status(200)
http_assert_json_contains("role", "admin")

db_assert_count(
    "postgresql+psycopg://user:pw@host/db",
    "SELECT 1 FROM orders WHERE user_id = :uid",
    expected=1,
    params={"uid": 42},
)

OAuth2 helpers cache tokens in-process and refresh 30 seconds before expiry.

Recorder

from je_web_runner import (
    recorder_start,
    recorder_stop,
    recorder_save_recording,
)

recorder_start(webdriver_wrapper_instance)
# … user clicks / inputs in the browser …
recorder_save_recording(
    webdriver_wrapper_instance,
    output_path="./recorded.json",
    raw_events_path="./raw.json",  # optional — debugging
)

The recorder injects a static JS listener (no CDP, no eval), so it works on Chrome / Firefox / Edge alike. Sensitive fields are masked by default — type=password, names matching password / card_number / cvv / ssn / secret / token / api_key / otp / passcode, and 13–19-digit numeric values are replaced with ***MASKED***.

CI / Integrations

from je_web_runner.utils.notifier.webhook_notifier import notify_run_summary
from je_web_runner.utils.test_management.jira_client import jira_create_failure_issues
from je_web_runner.utils.test_management.testrail_client import (
    testrail_send_results,
    testrail_results_from_pairs,
)
from je_web_runner.utils.ci_annotations.github_annotations import (
    emit_failure_annotations,
    emit_from_junit_xml,
)

For GitHub Actions inline annotations, run emit_from_junit_xml("run_junit.xml") after generate_junit_xml_report — failed test cases surface as ::error file=…:: lines on the PR diff.

docker/docker-compose.yml ships a Selenium Grid 4 stack (hub + Chrome + Firefox nodes); docker/.env.example exposes the version pin and concurrency settings.

The IDE config examples under docs/ide/ wire VS Code and JetBrains to the action JSON schema produced by WR_export_action_schema.

AI Assistance

from je_web_runner.utils.ai_assist.llm_assist import (
    set_llm_callable,
    suggest_locator,
    generate_actions_from_prompt,
)

# Plug in any callable that returns a string:
def my_llm(prompt: str) -> str:
    # call OpenAI / Anthropic / local Ollama / mock
    ...

set_llm_callable(my_llm)

locator = suggest_locator(html_blob, description="primary submit button")
draft = generate_actions_from_prompt("log in as alice and place an order")

WebRunner intentionally ships no built-in LLM client — the boundary is a single Callable[[str], str] so swapping provider is one line.

CLI Usage

# Original entry points (unchanged):
python -m je_web_runner -e actions.json
python -m je_web_runner -d ./actions/
python -m je_web_runner --execute_str '[["WR_quit_all"]]'

# Newer flags:
python -m je_web_runner -d ./actions --tag smoke --exclude-tag slow
python -m je_web_runner -d ./actions --parallel 4 --parallel-mode process
python -m je_web_runner -d ./actions --ledger ledger.json
python -m je_web_runner -d ./actions --rerun-failed ledger.json
python -m je_web_runner -d ./actions --shard 1/4
python -m je_web_runner -d ./actions --watch ./actions
python -m je_web_runner --report run                          # JSON + HTML + XML + JUnit
python -m je_web_runner --validate ./action_smoke.json
python -m je_web_runner --migrate ./actions --migrate-dry-run

Compose any of the flags above; the dispatcher applies tag filters → ledger / re-run-failed → sharding → dependency-aware ordering before handing files to the runner.

Test Record

from je_web_runner import test_record_instance

test_record_instance.set_record_enable(True)
# … perform automation …
records = test_record_instance.test_record_list
# Each record: {"function_name", "local_param", "time", "program_exception"}
test_record_instance.clean_record()

Exception Handling

WebRunner provides a hierarchy of custom exceptions — every helper raises a domain-specific subclass of WebRunnerException:

Exception Description
WebRunnerException Base
WebRunnerWebDriverNotFoundException WebDriver not found
WebRunnerOptionsWrongTypeException Invalid options type
WebRunnerArgumentWrongTypeException Invalid argument type
WebRunnerWebDriverIsNoneException WebDriver is None
WebRunnerExecuteException Action execution error
WebRunnerJsonException JSON processing error
WebRunnerGenerateJsonReportException JSON / XML / JUnit / Allure report error
WebRunnerHTMLException HTML report error
WebRunnerAddCommandException Custom command registration error
WebRunnerAssertException Assertion failure
XMLException / XMLTypeException XML processing error
CallbackExecutorException Callback execution error
PlaywrightBackendError Playwright backend / element failure
PlaywrightLocatorError TestObject → Playwright selector mapping
RecorderError / VisualRegressionError Recorder / visual regression
HealingError / EnvConfigError / DataDrivenError Self-healing / env / dataset
HttpAssertionError / HttpError HTTP API assertions
AccessibilityError / LighthouseError a11y / Lighthouse
NotifierError / JiraError / TestRailError Notifications / test management
CDPError / StorageError / ServiceWorkerError Browser internals
OAuthError / DatabaseValidationError Auth / DB
NetworkEmulationError / LoadTestError Throttling / Locust
ShardingError / MigrationError / ActionLinterError Orchestration / linting
LLMAssistError / OTelTracingError AI / observability

Logging

WebRunner uses a rotating file handler:

Supported Browsers

Browser Selenium key Playwright
Google Chrome chrome chromium
Chromium chromium chromium
Mozilla Firefox firefox firefox
Microsoft Edge edge chromium
Internet Explorer ie n/a
Apple Safari safari webkit

Supported Platforms

License

This project is licensed under the MIT License.

This site is open source. Improve this page.