# Browser Automation with Playwright and OpenClaw

Canonical: https://clawdocx.com/docs/browser-automation
Author: Sam Okafor
Difficulty: INTERMEDIATE
Published: 2026-02-24
Updated: 2026-09-19

> Learn how to use Playwright MCP skills with OpenClaw to automate browser tasks: web scraping, form filling, screenshot capture, and price monitoring.

## Why Browser Automation Is a Game-Changer for AI Agents

Most AI agents are limited to APIs and text-based tools. They can search the web, read files, and call endpoints, but they cannot interact with websites the way a human does. Browser automation changes that completely.

With Playwright integrated into OpenClaw, your agent can:

- **Navigate any website**: not just API-friendly ones, but any site with a URL
- **Fill out forms**: job applications, contact forms, registration pages, checkout flows
- **Extract data**: scrape product prices, job listings, news articles, competitor information
- **Take screenshots**: capture visual evidence, monitor UI changes, create reports
- **Interact with dynamic content**: handle JavaScript-heavy SPAs, infinite scroll, modal dialogs
- **Automate repetitive web tasks**: anything you do in a browser, your agent can do too

> **Note**
>
> Browser automation is one of the most powerful capabilities you can give your OpenClaw agent. It transforms your agent from a text-only assistant into something that can truly act on your behalf across the entire web.

The combination of an LLM's reasoning ability with a real browser creates workflows that were previously impossible without custom code. Your agent can look at a webpage, understand its structure, decide what to click, and extract exactly the information you need.

---

## Playwright MCP Skill Setup

### What Is the Playwright MCP Skill?

The Playwright MCP skill gives OpenClaw access to a headless browser through the Model Context Protocol. It wraps Microsoft's Playwright library, providing tools for navigation, interaction, data extraction, and screenshot capture.

### Installation

Install the Playwright browser automation skill from ClawHub:

```bash
openclaw skills install @<publisher>/playwright-browser
```

This installs the skill and its dependencies. The first run will download browser binaries (approximately 400 MB for Chromium).

### Browser Selection

Playwright supports three browser engines. The skills config reference puts a skill's settings in the main config file: "Most skills configuration lives under `skills` in `~/.openclaw/openclaw.json`." On an entry under `skills.entries.<key>`, `config` is an "Optional bag for custom per-skill configuration fields", so the field names inside that bag belong to the skill rather than to OpenClaw.

`playwright-browser` is an illustrative skill key, as are those field names and the publisher scope in the install ref above; take the real ones from the skill's ClawHub listing before copying any block below.

```json5
// ~/.openclaw/openclaw.json
{
  skills: {
    entries: {
      "playwright-browser": {
        enabled: true,
        config: {
          browser: {
            engine: "chromium", // chromium, firefox or webkit
            headless: true, // run without a visible window
            timeout: 30000, // default timeout in milliseconds
            viewport: { width: 1920, height: 1080 },
            userAgent: "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36",
          },
        },
      },
    },
  },
}
```

**Choosing a browser engine:**

| Engine | Best For | Notes |
|--------|----------|-------|
| **Chromium** | General use, widest compatibility | Default choice, most sites work perfectly |
| **Firefox** | Sites that block Chromium headless | Different fingerprint, useful for stealth |
| **WebKit** | Safari-specific testing | Smallest footprint, macOS-native rendering |

> **Tip**
>
> Start with Chromium. Only switch to Firefox or WebKit if you encounter sites that specifically block headless Chromium. Most users never need to change the default.

### Verify Installation

There is no `openclaw skills test`. The documented readiness command is `openclaw skills check`, which "reports missing prerequisites independently of agent exclusion", and `openclaw skills info <name>` resolves one skill's metadata:

```bash
openclaw skills check
```

```bash
openclaw skills info playwright-browser
```

A ready skill appears without a **Missing requirements** entry. Browser binaries are fetched by the skill on its first real run, so ask your agent for one screenshot afterwards to confirm it can launch a browser end to end.

---

## Basic Browser Operations

Once installed, you can ask your agent to perform browser tasks in natural language. Behind the scenes, the Playwright skill translates these requests into browser commands.

### Navigating Pages

The simplest operation is visiting a URL:

```
You: "Go to https://news.ycombinator.com and tell me the top 5 stories"
```

Your agent will:
1. Launch a headless browser
2. Navigate to the URL
3. Wait for the page to load
4. Extract the story titles
5. Return them in a formatted list

For programmatic access, the skill exposes these navigation tools:

```yaml
# Navigation tools available
tools:
  - browser_navigate:
      url: "https://example.com"
      waitUntil: "networkidle"    # domcontentloaded, load, or networkidle
  - browser_go_back
  - browser_go_forward
  - browser_reload
```

### Clicking Elements

Your agent can click buttons, links, and any interactive element:

```
You: "Go to GitHub, click on the Explore tab, and tell me the trending repositories"
```

The skill uses multiple selector strategies to find elements:

```yaml
# Selector strategies (in order of preference)
selectors:
  - text: "Sign In"              # By visible text
  - css: "button.submit-btn"     # By CSS selector
  - xpath: "//button[@id='go']"  # By XPath
  - role: "button[name='Submit']" # By ARIA role
  - testid: "login-button"       # By data-testid attribute
```

> **Tip**
>
> Text-based selectors are the most resilient because they match what humans see on screen. CSS selectors break when developers change class names. Prefer text selectors when possible.

### Filling Forms

Form interaction is where browser automation really shines:

```
You: "Go to the contact form on our website and fill in:
  Name: John Smith
  Email: john@example.com
  Message: I'd like to schedule a demo"
```

The skill handles various input types:

```yaml
# Form interaction tools
tools:
  - browser_fill:
      selector: "#email"
      value: "john@example.com"
  - browser_select:              # Dropdowns
      selector: "#country"
      value: "United States"
  - browser_check:               # Checkboxes
      selector: "#agree-terms"
  - browser_upload:              # File uploads
      selector: "#resume"
      file: "/path/to/resume.pdf"
```

### Taking Screenshots

Screenshots are useful for verification, monitoring, and reporting:

```
You: "Take a screenshot of https://example.com and save it to my desktop"
```

Screenshot configuration options:

```yaml
# Screenshot tools
tools:
  - browser_screenshot:
      fullPage: true              # Capture entire scrollable page
      path: "~/Desktop/shot.png"
      type: "png"                 # png or jpeg
      quality: 90                 # JPEG quality (1-100)
  - browser_screenshot_element:
      selector: ".main-content"   # Screenshot specific element
      path: "~/Desktop/content.png"
```

---

## Web Scraping: Extracting Structured Data

Web scraping is one of the most common uses for browser automation. Your agent can extract structured data from virtually any website.

### Basic Data Extraction

Ask your agent to scrape data in natural language:

```
You: "Go to https://books.toscrape.com and extract all book titles and prices
     from the first page into a JSON file"
```

The agent will navigate to the page, identify the data structure, extract each book's title and price, and save the results.

### Handling Pagination

Most real-world scraping involves multiple pages:

```
You: "Scrape all job listings from the first 5 pages of
     https://example-jobs.com/search?q=python+developer
     Save title, company, location, and salary to jobs.csv"
```

### Step 1: Agent navigates to the first page of results

### Step 2: Extracts all listings on the current page into structured data

### Step 3: Identifies the 'Next Page' button or pagination link

### Step 4: Clicks to advance to the next page and waits for load

### Step 5: Repeats extraction for each page up to the specified limit

### Step 6: Compiles all results and saves to the requested CSV format

### Dealing with Dynamic Content

Many modern websites load content dynamically with JavaScript. The Playwright skill handles this automatically:

```yaml
# Dynamic content strategies
strategies:
  wait_for_selector:
    selector: ".results-loaded"
    timeout: 10000

  wait_for_network_idle:
    timeout: 5000

  infinite_scroll:
    scrolls: 10
    delay: 2000                  # Wait 2s between scrolls

  lazy_load:
    scroll_increment: 500        # Scroll 500px at a time
    wait_between: 1000           # Wait 1s for images to load
```

> **Warning**
>
> Always set reasonable timeouts when dealing with dynamic content. A page that never reaches "network idle" will cause your agent to hang until the timeout expires. Use `waitForSelector` targeting a specific element instead of `networkidle` when possible.

### Extracting Tables

HTML tables are a common data source. The skill can extract them directly:

```
You: "Go to https://en.wikipedia.org/wiki/List_of_countries_by_GDP
     and extract the main GDP table into a spreadsheet"
```

The agent will:
1. Navigate to the page
2. Locate the table element
3. Parse header rows and data rows
4. Handle merged cells and footnotes
5. Export as CSV or JSON

---

## Anti-Bot Protection and Stealth

Many websites employ anti-bot measures. Here is how to handle them responsibly.

### Common Anti-Bot Measures

| Measure | Description | Mitigation |
|---------|-------------|------------|
| **User-Agent checks** | Block known headless browser UAs | Set realistic user-agent string |
| **JavaScript challenges** | Fingerprint browser capabilities | Playwright passes most checks natively |
| **Rate limiting** | Throttle rapid requests | Add delays between requests |
| **CAPTCHAs** | Human verification challenges | Slow down; some can be solved with services |
| **IP blocking** | Block IPs with too many requests | Use residential proxies or reduce frequency |
| **Honeypot fields** | Hidden form fields that trap bots | The agent's visual understanding avoids these |

### Stealth Mode Configuration

Configure the Playwright skill for stealth operation in the same `config` bag:

```json5
// ~/.openclaw/openclaw.json
{
  skills: {
    entries: {
      "playwright-browser": {
        config: {
          stealth: {
            enabled: true,
            options: {
              mask_webdriver: true, // hide the navigator.webdriver flag
              mock_plugins: true, // simulate browser plugins
              mock_languages: true, // set realistic language headers
              mock_permissions: true, // simulate the permission API
              random_viewport: false, // slight viewport randomization
            },
          },
        },
      },
    },
  },
}
```

### Rate Limiting and Delays

Always add delays between requests to avoid overwhelming servers:

```json5
// ~/.openclaw/openclaw.json
{
  skills: {
    entries: {
      "playwright-browser": {
        config: {
          rate_limit: {
            min_delay: 2000, // minimum 2 seconds between page loads
            max_delay: 5000, // random delay up to 5 seconds
            concurrent_pages: 1, // one page at a time
            requests_per_minute: 15, // hard cap on requests
            backoff: {
              enabled: true,
              initial: 5000, // 5 second backoff on 429 responses
              multiplier: 2, // double the backoff each retry
              max: 60000, // 60 second ceiling
            },
          },
        },
      },
    },
  },
}
```

> **Warning**
>
> Rate limiting is not just about avoiding detection. It is about being a responsible web citizen. Hammering a server with rapid requests can degrade service for real users and may violate terms of service. Always add reasonable delays.

### User-Agent Rotation

Rotate user agents to reduce fingerprinting:

```yaml
# User-agent rotation
user_agents:
  rotate: true
  pool:
    - "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"
    - "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"
    - "Mozilla/5.0 (X11; Linux x86_64; rv:134.0) Gecko/20100101 Firefox/134.0"
    - "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.2 Safari/605.1.15"
```

---

## Form Filling Automation

Automating form submissions is one of the most practical browser automation tasks. Your agent can handle everything from simple contact forms to complex multi-step applications.

### Single-Page Forms

```
You: "Fill out the feedback form at https://example.com/feedback with:
     - Rating: 5 stars
     - Comment: 'Great product, very intuitive interface'
     - Would recommend: Yes
     Then submit it"
```

### Multi-Step Wizards

Multi-step forms require the agent to navigate between steps:

```
You: "Go to the job application at https://careers.example.com/apply/12345
     and fill it out with my information from ~/Documents/resume-info.json"
```

### Step 1: Agent reads resume-info.json for applicant data

### Step 2: Navigates to the application URL

### Step 3: Step 1: Fills personal information (name, email, phone)

### Step 4: Clicks 'Next' and waits for step 2 to load

### Step 5: Step 2: Fills education history

### Step 6: Clicks 'Next' and waits for step 3 to load

### Step 7: Step 3: Fills work experience

### Step 8: Clicks 'Next' and waits for step 4 to load

### Step 9: Step 4: Uploads resume PDF and cover letter

### Step 10: Reviews all information on summary page

### Step 11: Takes a screenshot of the completed form before submitting

### Step 12: Submits the application and captures the confirmation

> **Tip**
>
> Always ask your agent to take a screenshot of a completed form before submitting. This gives you a visual record and a chance to verify the information is correct. You can configure an approval step where the agent pauses and sends you the screenshot before clicking Submit.

### Approval Workflow for Form Submissions

For high-stakes submissions, configure an approval step:

```yaml
# Form submission approval workflow
form_automation:
  require_approval: true
  approval_channel: "telegram"    # Send screenshot to Telegram for approval
  timeout: 3600                   # Wait up to 1 hour for approval
  on_timeout: "cancel"            # Cancel if not approved in time
```

The agent will fill the form, screenshot it, send it to your Telegram, and wait for your "approve" or "reject" response before submitting.

---

## Screenshot Automation

Screenshots serve many purposes beyond simple capture: monitoring, testing, evidence collection, and reporting.

### Scheduled Website Screenshots

Monitor how a website looks over time:

```
You: "Set up a cron job to screenshot https://example.com every morning at 9 AM
     and save screenshots to ~/Screenshots/daily/ with the date in the filename"
```

```bash
openclaw cron add --schedule "0 9 * * *" --task "Take a full-page screenshot of https://example.com and save to ~/Screenshots/daily/example-$(date +%Y-%m-%d).png"
```

### Visual Regression Testing

Compare screenshots to detect changes:

```
You: "Compare today's screenshot of our landing page with yesterday's.
     Highlight any visual differences and report what changed."
```

The agent will:
1. Take a fresh screenshot of the current page
2. Load the previous day's screenshot
3. Perform a pixel-level comparison
4. Generate a diff image highlighting changes
5. Describe the changes in plain language

### Capturing Evidence

For documentation, compliance, or dispute resolution:

```
You: "Go to https://competitor.com/pricing, take a timestamped screenshot,
     and save it to ~/Evidence/ with today's date. Also save the page source."
```

---

## Practical Example: Price Monitoring

Track prices across e-commerce sites and get alerts when prices drop.

### Setup

```
You: "I want to track these products:
     1. Sony WH-1000XM5 headphones on Amazon
     2. MacBook Air M4 on Best Buy
     3. Standing desk on IKEA
     Check prices daily at 8 AM and alert me on Telegram if any price drops by 5% or more."
```

### Configuration

Create a price monitoring configuration:

```yaml
# ~/price-monitor.yaml
products:
  - name: "Sony WH-1000XM5"
    url: "https://www.amazon.com/dp/B0BX3K9VLM"
    selector: ".a-price-whole"
    currency: "USD"
    alert_threshold: 5            # Alert on 5% drop

  - name: "MacBook Air M4"
    url: "https://www.bestbuy.com/site/macbook-air-m4/..."
    selector: ".priceView-customer-price span"
    currency: "USD"
    alert_threshold: 3            # Alert on 3% drop

  - name: "Standing Desk"
    url: "https://www.ikea.com/us/en/p/..."
    selector: ".pip-price__integer"
    currency: "USD"
    alert_threshold: 10           # Alert on 10% drop

monitoring:
  history_file: "~/price-history.json"
  alert_channel: "telegram"
```

### Cron Setup

```bash
openclaw cron add --schedule "0 8 * * *" --task "Check all products in ~/price-monitor.yaml, compare with historical prices, and alert me on Telegram if any price dropped by more than the threshold. Update price history."
```

### Sample Alert Output

```
Price Drop Alert!

Sony WH-1000XM5 Headphones
  Previous: $348.00 (Feb 22)
  Current:  $298.00 (Feb 24)
  Drop:     $50.00 (14.4%)
  Link:     https://www.amazon.com/dp/B0BX3K9VLM

Historical prices (last 30 days):
  High: $378.00 (Feb 1)
  Low:  $298.00 (today!)
  Avg:  $352.40
```

---

## Practical Example: Job Board Scraping

Aggregate job listings from multiple sites into a single spreadsheet.

### Setup

```
You: "Scrape Python developer jobs from Indeed, LinkedIn, and Glassdoor.
     Filter for remote positions in the US with salary > $120k.
     Save results to a Google Sheet and update daily."
```

### Workflow Configuration

```yaml
# ~/job-search.yaml
search:
  keywords: "Python Developer"
  location: "United States"
  remote: true
  salary_min: 120000
  experience: "mid-senior"

sources:
  - name: "Indeed"
    url: "https://www.indeed.com/jobs?q=python+developer&l=remote&salary=120000"
    max_pages: 5
  - name: "LinkedIn"
    url: "https://www.linkedin.com/jobs/search/?keywords=python+developer&remote=true"
    max_pages: 3
  - name: "Glassdoor"
    url: "https://www.glassdoor.com/Job/python-developer-jobs-SRCH..."
    max_pages: 3

output:
  format: "csv"
  path: "~/job-listings.csv"
  fields:
    - title
    - company
    - location
    - salary_range
    - posted_date
    - url
    - source
  deduplicate_by: ["title", "company"]
```

```bash
openclaw cron add --schedule "0 7 * * 1-5" --task "Run job board scraper using ~/job-search.yaml config, deduplicate against existing listings, and send me a Telegram summary of new listings found"
```

---

## Practical Example: Competitor Tracking

Monitor competitor websites for changes in pricing, features, and content.

### Setup

```
You: "Monitor these competitor pages for changes:
     1. https://competitor-a.com/pricing
     2. https://competitor-b.com/features
     3. https://competitor-c.com/blog
     Check daily, screenshot each page, and notify me of any significant changes."
```

### How It Works

### Step 1: Agent navigates to each competitor URL

### Step 2: Takes a full-page screenshot for visual records

### Step 3: Extracts the page text content

### Step 4: Compares text content with the previous day's snapshot

### Step 5: Generates a diff highlighting additions, removals, and changes

### Step 6: Classifies changes by significance (pricing change vs minor text edit)

### Step 7: Sends a summary to your configured notification channel

### Step 8: Archives screenshots and text snapshots in dated folders

### Sample Change Report

```
Competitor Tracking Report - Feb 24, 2026

Competitor A - Pricing Page
  SIGNIFICANT CHANGE DETECTED
  - Pro plan price increased from $49/mo to $59/mo
  - New "Enterprise" tier added at $199/mo
  - Free tier now limited to 100 requests (was 500)
  Screenshot: ~/competitor-tracking/competitor-a/2026-02-24.png
  Diff: ~/competitor-tracking/competitor-a/2026-02-24-diff.html

Competitor B - Features Page
  MINOR CHANGE
  - Added "AI-powered analytics" to feature list
  - Updated testimonial section
  Screenshot: ~/competitor-tracking/competitor-b/2026-02-24.png

Competitor C - Blog
  NEW CONTENT
  - New post: "How We Scaled to 1M Users"
  - Published: Feb 23, 2026
  Screenshot: ~/competitor-tracking/competitor-c/2026-02-24.png
```

---

## Practical Example: Social Media Monitoring

Track mentions, engagement, and trends across social platforms.

### Setup

```
You: "Monitor social media for mentions of 'OpenClaw' and our competitors.
     Track Twitter, Reddit, and Hacker News.
     Send me a weekly report every Monday morning."
```

### Monitoring Configuration

```yaml
# ~/social-monitor.yaml
keywords:
  brand: ["OpenClaw", "openclaw", "@openclaw"]
  competitors: ["CompetitorX", "CompetitorY"]
  industry: ["AI agent", "personal AI assistant", "autonomous agent"]

sources:
  twitter:
    enabled: true
    search_url: "https://twitter.com/search?q=OpenClaw"
    track_metrics: ["mentions", "sentiment", "engagement"]

  reddit:
    enabled: true
    subreddits: ["selfhosted", "artificial", "MachineLearning", "LocalLLaMA"]
    search_terms: ["OpenClaw", "AI agent"]

  hackernews:
    enabled: true
    search_url: "https://hn.algolia.com/?q=OpenClaw"
```

```bash
openclaw cron add --schedule "0 8 * * 1" --task "Run social media monitoring report using ~/social-monitor.yaml, compile weekly summary with sentiment analysis, and send to Telegram"
```

---

## Data Extraction Workflows

### Saving to CSV

```
You: "Scrape the product catalog from https://example-store.com
     and save all products with name, price, category, and rating to products.csv"
```

The agent outputs clean CSV:

```csv
name,price,category,rating,url
"Widget Pro",29.99,"Electronics",4.5,"https://example-store.com/widget-pro"
"Gadget X",49.99,"Electronics",4.2,"https://example-store.com/gadget-x"
"Super Tool",15.99,"Tools",4.8,"https://example-store.com/super-tool"
```

### Saving to JSON

For structured data that preserves hierarchy:

```
You: "Extract the full product catalog with all details including
     descriptions, images, and specifications. Save as JSON."
```

```json
{
  "scraped_at": "2026-02-24T08:00:00Z",
  "source": "https://example-store.com",
  "total_products": 156,
  "products": [
    {
      "name": "Widget Pro",
      "price": 29.99,
      "category": "Electronics",
      "rating": 4.5,
      "description": "Professional-grade widget with...",
      "images": ["https://example-store.com/img/widget-pro-1.jpg"],
      "specifications": {
        "weight": "0.5 kg",
        "dimensions": "10x5x3 cm",
        "color": "Matte Black"
      }
    }
  ]
}
```

### Piping Data to Other Tools

The real power comes from chaining browser data extraction with other skills:

```
You: "Scrape the latest tech job salaries from levels.fyi,
     analyze the trends, create a chart, and post it to our Slack channel"
```

This chains: Playwright (scrape) -> File Manager (save data) -> Analysis (process) -> Slack (notify).

---

## Scheduling with Cron

### Common Scheduling Patterns

```bash
# Every morning at 8 AM
openclaw cron add --schedule "0 8 * * *" --task "..."

# Every weekday at 9 AM and 5 PM
openclaw cron add --schedule "0 9,17 * * 1-5" --task "..."

# Every 6 hours
openclaw cron add --schedule "0 */6 * * *" --task "..."

# Every Monday at 7 AM
openclaw cron add --schedule "0 7 * * 1" --task "..."

# First day of each month
openclaw cron add --schedule "0 9 1 * *" --task "..."
```

### Managing Scheduled Jobs

```bash
openclaw cron list
```

```
ID  Schedule        Task                                    Status    Last Run
1   0 8 * * *       Price monitor                           Active    Feb 24, 08:00
2   0 7 * * 1-5     Job board scraper                       Active    Feb 24, 07:00
3   0 9 * * *       Competitor tracking                     Active    Feb 24, 09:00
4   0 8 * * 1       Social media report                     Active    Feb 17, 08:00
```

```bash
openclaw cron runs 1 --limit 5
```

View the recent run history for any scheduled task to verify it is running correctly. There is no `cron logs` subcommand and no `--last` flag; run history is `runs` with `--limit`, and the Gateway's own stream is `openclaw logs --follow`.

---

## Error Handling and Resilience

Web scraping is inherently fragile. Sites change their structure, go down, or add new protections. Build resilience into your workflows.

### Retry Logic

```yaml
# Error handling configuration
error_handling:
  retries: 3
  retry_delay: 5000              # 5 seconds between retries
  backoff_multiplier: 2          # Exponential backoff
  retry_on:
    - "timeout"
    - "navigation_failed"
    - "element_not_found"
  fail_on:
    - "blocked"
    - "captcha_required"
```

### Handling Site Changes

When a site changes its structure, selectors break. Build in fallback strategies:

```yaml
# Fallback selectors for price extraction
selectors:
  primary: ".product-price .current"
  fallbacks:
    - ".price-current"
    - "[data-price]"
    - ".price"
    - "span:has-text('$')"
  on_all_fail: "screenshot_and_notify"
```

> **Tip**
>
> When all selectors fail, have your agent take a screenshot and send it to you with a message like "The pricing page structure has changed. Here is what it looks like now. Should I update the selectors?" This turns a failure into a quick manual fix.

### Health Checks

Add health checks to your scraping workflows:

```yaml
# Health check configuration
health_check:
  verify_data:
    min_results: 1               # Expect at least 1 result
    max_results: 10000           # Sanity check upper bound
    required_fields: ["name", "price"]
  on_anomaly:
    action: "notify"
    channel: "telegram"
    message: "Scraping anomaly detected: {details}"
```

---

## Ethics and Legal Considerations

Browser automation is powerful, but it comes with responsibilities. Always operate ethically and within legal boundaries.

### Respect robots.txt

Before scraping any site, check its robots.txt file:

```
You: "Check if https://example.com allows scraping in their robots.txt"
```

> **Warning**
>
> robots.txt is a voluntary standard, not a legal requirement. However, ignoring it can lead to your IP being blocked, legal action from the site owner, or terms of service violations. Always respect it.

### Rate Limiting Is Non-Negotiable

Never send more requests than a human would. Good rules of thumb:

- **Minimum 2-second delay** between page loads
- **Maximum 15-20 pages per minute** for general browsing
- **Reduce frequency** for small sites with limited server capacity
- **Stop immediately** if you receive 429 (Too Many Requests) responses

### Terms of Service

Many websites explicitly prohibit automated access in their terms of service. Before setting up automated scraping:

1. Read the target site's Terms of Service
2. Check their robots.txt
3. Look for an official API (always prefer APIs over scraping)
4. Consider reaching out to ask permission for large-scale data collection

### Data Privacy

Be careful with scraped data:

- **Do not scrape personal information** without consent or legal basis
- **GDPR and CCPA apply** to data you collect, even through scraping
- **Do not store sensitive data** (emails, phone numbers, addresses) without a legitimate purpose
- **Delete data** when it is no longer needed

### Best Practices Checklist

### Step 1: Check robots.txt before scraping any site

### Step 2: Read the site's Terms of Service for automation restrictions

### Step 3: Prefer official APIs over scraping when available

### Step 4: Always add 2+ second delays between requests

### Step 5: Set a reasonable maximum requests-per-minute cap

### Step 6: Never scrape personal or sensitive data without legal basis

### Step 7: Store only the data you actually need

### Step 8: Monitor your scraping jobs for errors and anomalies

### Step 9: Be prepared to stop if the site owner requests it

### Step 10: Document your scraping activities for compliance purposes

---

## Complete Browser Automation Config Example

Here is a full configuration bringing together everything covered in this guide, as one entry in `~/.openclaw/openclaw.json`:

```json5
// ~/.openclaw/openclaw.json
{
  skills: {
    entries: {
      "playwright-browser": {
        enabled: true,
        config: {
          browser: {
            engine: "chromium",
            headless: true,
            timeout: 30000,
            viewport: { width: 1920, height: 1080 },
          },
          stealth: {
            enabled: true,
            options: {
              mask_webdriver: true,
              mock_plugins: true,
              mock_languages: true,
            },
          },
          user_agents: {
            rotate: true,
            pool: [
              "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36",
              "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36",
              "Mozilla/5.0 (X11; Linux x86_64; rv:134.0) Gecko/20100101 Firefox/134.0",
            ],
          },
          rate_limit: {
            min_delay: 2000,
            max_delay: 5000,
            concurrent_pages: 1,
            requests_per_minute: 15,
            backoff: { enabled: true, initial: 5000, multiplier: 2, max: 60000 },
          },
          error_handling: {
            retries: 3,
            retry_delay: 5000,
            backoff_multiplier: 2,
            screenshots_on_error: true,
            error_screenshot_dir: "~/browser-errors/",
          },
          output: {
            default_format: "json",
            default_dir: "~/browser-data/",
            timestamp_files: true,
            compress_old: true,
            retention_days: 90,
          },
        },
      },
    },
  },
}
```

---

## Troubleshooting

### Common Issues

| Issue | Cause | Solution |
|-------|-------|----------|
| "Browser not found" | Chromium not downloaded | Reinstall with `openclaw skills install @<publisher>/playwright-browser --force`, which "Overwrites an existing workspace skill folder for the same slug." |
| Timeout on page load | Slow site or heavy JS | Increase `timeout` in config, use `domcontentloaded` instead of `networkidle` |
| Elements not found | Site structure changed | Update selectors, use text-based selectors for resilience |
| Screenshots are blank | Page not fully rendered | Add `waitForSelector` before screenshot |
| "Blocked by Cloudflare" | Bot detection triggered | Enable stealth mode, add delays, try Firefox engine |
| Memory issues | Too many pages open | Ensure pages are closed after use, set `concurrent_pages: 1` |

### Getting Help

If you encounter issues not covered here, read the Gateway log. `openclaw logs` tails "Gateway file logs over RPC", and OpenClaw documents no per-skill log command:

```bash
openclaw logs --follow
```

Run the failing task again while that is attached, and the browser errors and stack traces come through the stream. Most issues are related to selector changes on target websites or network timeouts.

---

## Next Steps

Now that you have browser automation set up, consider:

- **Combining with email automation**: scrape data, then email reports automatically
- **Building data pipelines**: scrape, process, store, visualize
- **Creating custom monitoring dashboards**: aggregate scraped data into a single view
- **Exploring the Playwright MCP skill's advanced features**: PDF generation, video recording, network interception

Browser automation is one of the most versatile tools in your OpenClaw toolkit. Start with simple tasks and build up to complex multi-site workflows as you gain confidence.

## Frequently asked questions

**Where do a browser skill's settings live?**

In the main config file, not in a file of the skill's own. OpenClaw's skills config reference states that most skills configuration lives under skills in ~/.openclaw/openclaw.json, and a skill's own options sit on its entry at skills.entries.<key>.config, which the same page calls an optional bag for custom per-skill configuration fields.

**How do I check that the skill installed correctly?**

Run openclaw skills check, which the skills CLI documents as reporting missing prerequisites, and openclaw skills info <name> for one skill's resolved metadata. There is no openclaw skills test subcommand; the documented set is search, install, update, verify, list, info, check, library, curator and workshop.

**Where are the logs when a browser run fails?**

In the Gateway log. The openclaw logs command tails Gateway file logs over RPC and takes --follow to watch a run live. OpenClaw documents no per-skill log command, so run the failing task again while openclaw logs --follow is attached.

## Sources

- [OpenClaw: Skills config](https://docs.openclaw.ai/tools/skills-config)
- [OpenClaw: Configuration](https://docs.openclaw.ai/gateway/configuration)
- [OpenClaw: Skills CLI](https://docs.openclaw.ai/cli/skills)
- [OpenClaw: Logs](https://docs.openclaw.ai/cli/logs)
- [OpenClaw: Automations (cron)](https://docs.openclaw.ai/cli/cron)