WebRunner

CLAUDE.md — WebRunner

Project Overview

WebRunner (je_web_runner) is a cross-platform web automation framework built on Selenium. It supports multi-browser parallel execution, JSON-driven action scripts, report generation, and remote automation via TCP sockets.

Development Commands

# Install dependencies
pip install -r requirements.txt

# Install dev dependencies
pip install -r dev_requirements.txt

# Run tests
python -m pytest test/

# Build package
python -m build

Project Structure

je_web_runner/
├── __init__.py              # Public API exports
├── __main__.py              # CLI entry point
├── element/                 # WebElement interaction (Wrapper pattern)
├── manager/                 # Multi-driver management (Manager pattern)
├── webdriver/               # WebDriver wrapper & options (Facade pattern)
└── utils/
    ├── callback/            # Event-driven callback executor (Observer pattern)
    ├── exception/           # Custom exception hierarchy
    ├── executor/            # Action executor engine (Command pattern)
    ├── generate_report/     # HTML/JSON/XML report generators (Strategy pattern)
    ├── json/                # JSON file operations
    ├── logging/             # Rotating file handler
    ├── package_manager/     # Dynamic package loading (Plugin pattern)
    ├── project/             # Project template generator (Template pattern)
    ├── socket_server/       # TCP socket server for remote control
    ├── test_object/         # Test object & record classes (Value Object pattern)
    ├── test_record/         # Action recording
    ├── xml/                 # XML utilities
    ├── chrome_profile/      # Persistent Chrome profile + stealth + snapshot/sync-back
    ├── failure_triage/      # AI failure root-cause analysis on failure bundles
    ├── flake_detector/      # Time-decayed flake scoring + quarantine registry
    ├── locator_health/      # Project-wide locator audit + upgrade suggestions
    ├── device_cloud/        # Real-device cloud (BrowserStack/Sauce/LambdaTest) connector
    ├── otel_bridge/         # W3C traceparent injection for distributed tracing
    ├── mutation_testing/    # Action JSON mutation testing (kill rate / score)
    ├── otp_interceptor/     # MailHog/Mailpit/IMAP/SMS OTP polling for 2FA flows
    ├── download_verify/     # PDF / CSV / Excel / JSON / SHA256 download assertions
    ├── test_auto_repair/    # LLM-driven test rewrite from failure + git diff
    ├── edge_case_generator/ # LLM edge-case variant generator (complement to mutation_testing)
    ├── openapi_to_e2e/      # OpenAPI/Swagger spec → WR_http_* action JSON
    ├── cross_tab_sync/      # Multi-page BroadcastChannel / storage propagation asserts
    ├── visual_ai/           # aHash/dHash/pHash + SSIM-proxy for canvas/chart diff
    ├── test_scheduler/      # Value-density scheduler under time + cloud budget
    ├── walkthrough_docs/    # AI step-by-step SOP / Confluence doc from recorded runs
    ├── live_dashboard/      # Aggregated web UI: runs + flake + quarantine + locators
    ├── ocr_assert/          # OCR-based text assertion for canvas / WebGL / image content
    ├── email_render/        # Capture outbound mail (MailHog/Mailpit/EML) + multi-viewport screenshots
    ├── backend_log_correlator/ # W3C trace_id → Loki/Elasticsearch/file log fetch into failure bundle
    ├── websocket_assert/    # WebSocket frame recorder + count / payload / pubsub assertions
    ├── console_error_budget/ # JS console / unhandled-rejection budget with ignore patterns
    ├── chaos_hooks/         # Seeded chaos injection (offline / throttle / mid-flow reload)
    ├── pr_risk_score/       # Fuse flake / impact / locator / coverage signals into 0-100 PR risk
    ├── flag_matrix/         # Feature-flag combo matrix with constraints + minimal failing subset
    ├── session_to_test/     # rrweb / generic session events → WR action JSON
    ├── exploratory_ai/      # Agentic exploratory tester (observer/planner protocols + RandomPlanner)
    ├── story_to_actions/    # LLM-driven user story / Figma frame → validated WR action JSON
    ├── db_snapshot/         # Per-test DB savepoint/rollback with pluggable backend
    ├── time_freezer/        # Inject Date/Date.now/performance.now patch via CDP for deterministic time tests
    ├── persona_runner/      # Same suite × N personas (admin/free/enterprise) matrix
    ├── token_leak_detector/ # Scan HAR / logs / responses for leaked JWTs, API keys, session tokens
    ├── consent_audit/       # GDPR/CCPA cookie classification + pre-consent / post-reject violation detection
    ├── pii_in_screenshot/   # OCR + PII regex (Luhn-validated card, SSN, TWID) scanner over screenshots
    ├── pseudo_localization/ # ASCII → look-alike + expansion + brackets; detect hard-coded i18n leaks
    ├── screen_reader_runner/ # Walk a11y tree to simulate NVDA/VoiceOver order + flag a11y violations
    ├── forced_colors_mode/  # dark / reduced-motion / forced-colors / high-contrast matrix verification
    ├── sse_assert/          # Server-Sent Events recorder + count/data/JSON-shape/strict-id assertions
    ├── webrtc_assert/       # PeerConnection state / ICE / track / RTP stats assertions
    ├── view_transitions/    # Instrumentation + duration/CLS/group assertions for View Transitions API
    ├── test_dedup_ai/       # Structural + embedding-based semantic dedupe of action JSON files
    ├── multimodal_qa/       # Send screenshot + question to vision LLM, parse pass/fail/notes envelope
    ├── prompt_drift_monitor/ # Track LLM-feature output drift via embeddings + lexical anchors
    ├── git_bisect_flake/    # Ledger-only or probe-driven bisect to find regression commit
    ├── test_cost_estimator/ # Cloud-minute × rate-card × CO₂ estimate per suite/runner/test
    ├── slack_digest/        # Render Slack Block-Kit / Teams card / plain-text test digest payload
    ├── webtransport_assert/ # HTTP/3 WebTransport datagram + stream frame recorder + assertions
    ├── indexed_db_explorer/ # IndexedDB snapshot harvest + store/key/index/record assertions
    ├── file_system_access/  # Mock showOpenFilePicker/showSaveFilePicker + record writes
    ├── notifications_audit/ # Notification.requestPermission timing + permission/spam policy checks
    ├── mixed_content_audit/ # HTTP-on-HTTPS detection via HAR + console scanner
    ├── clickjacking_audit/  # X-Frame-Options / frame-ancestors header check + iframe probe
    ├── open_redirect_detector/ # Probe ?redirect=/?next= params with attacker-host payloads
    ├── sri_verify/          # Subresource Integrity hash presence + correctness + crossorigin
    ├── coop_coep_audit/     # crossOriginIsolated COOP/COEP + per-resource CORP/CORS check
    ├── inp_tracker/         # Interaction to Next Paint instrumentation + p98 + budget
    ├── hydration_check/     # SSR hydration mismatch detection (DOM diff + console markers)
    ├── bundle_budget/       # Per-asset-kind byte budget from HAR + biggest-assets ranking
    ├── third_party_budget/  # Third-party vendor classification + req/byte/blocking-ms budgets
    ├── long_animation_frame/ # Long Animation Frame API listener + per-script attribution
    ├── grpc_tester/         # gRPC stub call recorder + gRPC-Web framing/trailer helpers
    ├── webhook_receiver/    # Threaded HTTP server for catching app's outbound webhooks
    ├── idempotency_check/   # Run request twice + compare status/body/state/side-effects
    ├── pagination_audit/    # Walk all pages, detect dups/gaps/cursor-loop/sort violations
    ├── failure_narrator/    # LLM natural-language failure summary from failure_bundle
    ├── repro_minimizer/     # Delta-debugging (ddmin) to shrink failing action list to minimum
    ├── locator_hardener/    # Heuristic fragility score + LLM-suggested stable selectors
    ├── test_categorizer/    # Auto-tag tests as smoke / regression / perf / a11y / data / api
    ├── quarantine_age_report/ # Quarantine entries with age + fresh/lingering/stale/abandoned tiers
    ├── test_debt_dashboard/ # Inventory of skip/xfail/TODO/_skip markers with age + CODEOWNERS
    ├── sla_tracker/         # % suites finishing under SLA threshold, weekly/daily bucketing
    ├── bug_repro_stability/ # Repeat probe N times, classify deterministic/flaky/non-reproducible
    ├── test_owners_map/     # CODEOWNERS parser + override layer + unowned-test audit
    ├── popover_assert/      # <dialog>/popover open/close/invoker assertions
    ├── cookie_store_api/    # Async cookieStore API harvest + change-event assertions
    ├── speculation_rules/   # Speculation Rules (prerender/prefetch) verification + no-double-fire
    ├── web_locks/           # Multi-tab Web Locks contention harness + deadlock/serialise assertions
    ├── storage_buckets/     # Storage Buckets API isolation + durability + IDB-isolation checks
    ├── hydration_streaming/ # Streaming SSR per-boundary timing + arrival/interactive assertions
    ├── memory_pressure_emulate/ # CDP memory/CPU pressure emulation profiles + run-under-profile
    ├── third_party_block_test/ # Vendor-by-vendor block-resilience matrix
    ├── bundle_diff_pr/      # PR bundle delta (added/removed/grew) + markdown report + growth gate
    ├── prompt_injection_scanner/ # LLM jailbreak payload library + canary-leak scan
    ├── cors_matrix/         # CORS preflight matrix probe + credentials/origin policy assertions
    ├── oauth_pkce_replay/   # Replay OAuth state/PKCE verifier; confirm server rejects
    ├── cookie_chips_audit/  # CHIPS Partitioned cookie compliance auditor
    ├── sbom_diff/           # CycloneDX SBOM diff (added/removed/upgrade/license/vuln)
    ├── failure_auto_tag/    # Heuristic + LLM failure auto-tagger (flaky-locator/timeout/js-error...)
    ├── test_self_describe/  # Reverse-engineer Gherkin Given/When/Then from action JSON
    ├── pr_title_generator/  # Conventional-Commits PR title from diff + commit history
    ├── action_refactor_suggester/ # Rule-based action-JSON refactor smells (hard sleep / positional xpath...)
    ├── rtl_layout_verify/   # RTL layout direction / logical-property / bidi-isolation audit
    ├── dst_boundary_test/   # DST spring-forward/fall-back gap & overlap detection + scheduled-fire model
    ├── number_currency_locale/ # Number/currency/date locale-format assertion helpers
    ├── wcag22_touch_target/ # WCAG 2.2 SC 2.5.8 target-size auditor with spacing-circle exception
    ├── graphql_n_plus_1/    # N+1 query detector for GraphQL operations
    ├── mq_assert/           # Kafka/RabbitMQ/SQS-style message-queue publish assertions
    ├── grpc_streaming_assert/ # gRPC streaming (unary/server/client/bidi) frame/status/half-close
    ├── webhook_signature_verify/ # GitHub/Stripe/Slack/generic HMAC webhook verifier
    ├── test_roi_scorer/     # Find-rate/cost/coverage/recency-weighted ROI score per test
    ├── pre_merge_gate_dsl/  # Declarative pre-merge gate rules (when/require) over PrFacts
    ├── commit_msg_trigger/  # Parse [skip ci]/[ci e2e]/[ci shard=3/8]/tickets from commit message
    ├── flakiness_graveyard/ # Quarantine/revive/bury ledger with TTL for stale flaky tests
    ├── test_blame_owner/    # CODEOWNERS + git-blame + HEAD + default → test owner chain
    ├── webgpu_pixel_verify/ # WebGPU canvas pixel readback + mean/solid/tile-diff assertions
    ├── webhid_mock/         # WebHID device shim with input/output report harness
    ├── webusb_mock/         # WebUSB device shim with control/bulk transfer capture
    ├── webserial_mock/      # Web Serial UART shim + line write capture
    ├── webcodecs_assert/    # WebCodecs chunk codec/resolution/keyframe/framerate assertions
    ├── speech_api_assert/   # SpeechSynthesis/SpeechRecognition mock + spoke/lang assertions
    ├── webauthn_mock/       # WebAuthn / FIDO2 / Passkey navigator.credentials shim
    ├── credential_management/ # navigator.credentials password/federated autofill mock
    ├── payment_request_assert/ # Payment Request API + Apple/Google Pay sheet validation
    ├── three_d_secure_flow/ # 3DS challenge / frictionless / fallback path assertions
    ├── rag_grounding_assert/ # RAG citation + grounding + hallucination phrase scan
    ├── llm_token_cost_tracker/ # Per-test token/$ ledger + per-model rate card + budget
    ├── streaming_chat_assert/ # TTFT / inter-token gap / UTF-8 / dup-or-OOS chunk assertions
    ├── tool_call_assert/    # LLM tool/function call name+order+arg-schema assertions
    ├── hallucination_probe/ # Ground-truth probe runner + hallucination rate budget
    ├── web_push_assert/     # VAPID subscription + endpoint + userVisibleOnly + showNotification
    ├── background_sync_assert/ # Background Sync register/fire/retry/lastChance assertions
    ├── wake_lock_assert/    # Screen wake lock acquire/release/leak/re-acquire detection
    ├── pip_assert/          # Picture-in-Picture (video + document) enter/exit/size assertions
    ├── web_share_assert/    # navigator.share recorder + payload + fallback assertions
    ├── compression_streams/ # CompressionStream gzip/deflate/brotli round-trip + ratio budget
    ├── compute_pressure/    # Compute Pressure API fake observer + throttle reaction assertions
    ├── touch_gesture/       # tap/swipe/pinch/long-press CDP-frame builder + event assertions
    ├── viewport_audit/      # viewport meta + safe-area-inset + WCAG 1.4.4 scalable audit
    ├── virtual_keyboard/    # visualViewport before/after + keyboard inset + focused-visible
    ├── pull_to_refresh/     # overscroll-behavior + threshold + refresh handler + network refetch
    ├── email_deliverability/ # SPF/DKIM/DMARC + List-Unsubscribe + BCC leak header audit
    ├── inbox_render_outlook/ # Outlook/Gmail/Apple Mail render compatibility pre-flight
    ├── push_delivery/       # FCM/APNs payload size + required fields + PII + collapse + TTL
    ├── lcp_image_audit/     # LCP image preload + no-lazy + fetchpriority=high assertions
    ├── font_loading_strategy/ # @font-face display + size-adjust + FOIT/FOUT/FOFT verification
    ├── resource_hints_audit/ # preload/prefetch/preconnect used vs declared + preload-as audit
    ├── critical_css_audit/  # Inline-CSS-in-head + first-packet budget + preload-blocking-CSS
    ├── lighthouse_regression/ # Lighthouse score regression vs baseline + CWV metric budget
    ├── dom_xss_taint/       # source→sink JS instrumentation + canary-based taint detection
    ├── csp_violation_parser/ # CSP report-uri / report-to parser + recon heuristic
    ├── hsts_preload_audit/  # HSTS preload-list compliance (max-age + includeSubDomains + preload)
    ├── tls_cipher_audit/    # Live TLS handshake + version + cipher allowlist + subject check
    ├── cookie_scope_abuse/  # Session-like cookie scope / HttpOnly / Secure / SameSite audit
    ├── test_dup_dry/        # Structural action-JSON duplicate + prefix-overlap detection
    ├── snapshot_diff_approval/ # Baseline/pending/rejected snapshot register + approval workflow
    ├── failure_cluster_dbscan/ # Failure-message tokeniser + DBSCAN root-cause clustering
    ├── test_naming_lint/    # should_when / given_when_then / camel_subject naming linter
    ├── openapi_drift/       # Live API vs spec drift (undocumented / zombie / status / method)
    ├── api_version_compat/  # Old-client vs new-server backward-compat response/request matrix
    ├── rate_limit_assert/   # 429 / Retry-After / X-RateLimit headers + recovery assertions
    └── har_to_openapi/      # HAR → OpenAPI 3.1 path/method/query/schema reverse engineering

Design Patterns & Architecture

Coding Standards

Software Engineering Principles

Performance Best Practices

Security Requirements

Code Quality

Static Analysis Compliance (SonarQube & Codacy)

All code MUST pass SonarQube and Codacy static analysis without introducing new issues. Follow these rules proactively:

Complexity & Maintainability

Naming & Style (PEP 8 enforced)

Bug Prevention

Security (SonarQube hotspots / Codacy bandit)

Documentation & Typing

Testing Quality

Pre-Commit Verification

Before committing, run the following checks locally and ensure they pass cleanly:

# Lint with project tools (add as needed)
python -m pylint je_web_runner/
python -m flake8 je_web_runner/ --max-line-length=120 --max-complexity=10
python -m bandit -r je_web_runner/ -ll

# Type check
python -m mypy je_web_runner/

# Tests
python -m pytest test/

If SonarQube or Codacy is wired into CI, fix any new issues in the same PR — do not defer them with suppressions unless a justified # noqa: <rule> comment explains why.

Git & Commit Conventions

Testing