AI Interview Questions

A computer use agent is an AI system that can directly interact with computer interfaces - clicking, typing, navigating menus, and using applications - just like a human user. It can control the mouse and keyboard to perform tasks across any software, not just through APIs.

Computer use agents operate by: 1) Observing the screen (via screenshots or accessibility data) to understand the current state. 2) Reasoning about what actions to take next. 3) Executing actions through system commands: mouse movements, clicks, keyboard input, shortcuts. 4) Observing results and iterating. They can use computer vision to identify buttons, fields, and UI elements, or leverage accessibility APIs for structured information. Tasks include: filling forms, navigating websites, using desktop applications, copying files, and even playing games.

Computer use agents represent a leap beyond API-based automation. They can interact with any software, even legacy systems without APIs. This enables automation of tasks that previously required human GUI interaction: data entry across multiple systems, software testing, robotic process automation (RPA), and personal assistance (e.g., 'book that restaurant on OpenTable'). They make AI truly act in the digital world.

User: 'Download the latest sales report from the company portal, extract the Q3 numbers, and add them to this spreadsheet.' Computer use agent: 1) Opens browser, navigates to portal. 2) Logs in (enters credentials). 3) Clicks through menus to reports section. 4) Clicks download button. 5) Opens Excel, opens target spreadsheet. 6) Copies Q3 numbers from downloaded report. 7) Pastes into correct cells. 8) Saves file. All done through GUI interaction, no APIs needed. This is computer use in action.

Anthropic's Computer Use feature (introduced in Claude 3.5 Sonnet) enables the model to interact with computer interfaces by viewing screenshots and outputting mouse and keyboard actions. It's a pioneering capability that allows Claude to use software like a human.

The system works in a loop: 1) Claude receives a screenshot of the current computer screen. 2) Based on the user's goal, it reasons about what action to take. 3) It outputs structured commands: mouse_move(x,y), left_click, type(text), key_press(key), scroll(direction), etc. 4) The system executes these commands, updating the screen. 5) New screenshot is taken and fed back to Claude. This continues until task completion. Claude can see pixel-level information, so it can identify buttons, text fields, and UI elements visually. It's trained to understand GUI layouts and common interaction patterns.

Anthropic's Computer Use is a breakthrough in agent capabilities. Instead of relying on APIs or structured data, Claude can use any software a human can, making it universally applicable. It opens possibilities for automating legacy systems, testing, personal assistance, and accessibility. While still experimental (it can be slow and error-prone), it points to a future where AI can truly use computers.

User asks Claude to 'find the latest research paper on quantum computing and save it to my Downloads folder.' Claude: 1) Sees desktop, opens browser. 2) Navigates to arXiv.org. 3) Types 'quantum computing' in search. 4) Clicks on most recent paper. 5) Clicks 'Download PDF' button. 6) Confirms save location. All through mouse and keyboard simulation. This works on any website, no API needed.

A browser agent is a specialized computer use agent that operates within a web browser, automating web navigation and interaction. It can understand web page structure, fill forms, click links, and extract information, enabling web automation at scale.

Browser agents interact with web pages through multiple methods: 1) DOM access - directly reading page structure (HTML) and manipulating elements via JavaScript. This is fast and reliable but requires access to page internals. 2) Vision-based - taking screenshots and using computer vision to identify elements, like human would. Works on any page but slower and less precise. 3) Hybrid - using accessibility tree or structured data when available, falling back to vision. Agents can navigate by: finding elements by text, XPath, or visual location; clicking, typing, selecting; waiting for page loads; handling pop-ups and authentication.

Web automation is one of the most valuable agent applications. Browser agents can: test web applications, scrape data, automate repetitive tasks (filling forms), monitor websites, and assist users with complex web tasks. They make the entire web programmable. Unlike traditional web scraping, they can handle JavaScript-heavy sites and complex workflows.

Price monitoring agent: 1) Navigates to Amazon product page. 2) Extracts price and availability. 3) Navigates to competitor site. 4) Compares prices. 5) If price drops below threshold, adds to cart. 6) Proceeds to checkout. This workflow, previously requiring custom code for each site, can be handled by a general browser agent that adapts to site structure.

Playwright is a browser automation library that provides a high-level API to control Chromium, Firefox, and WebKit. It's widely used as the underlying engine for browser agents, handling the low-level details of page interaction, waiting, and navigation.

Playwright enables: 1) Launching and controlling browsers headlessly or with UI. 2) Navigating to URLs. 3) Finding elements via selectors (CSS, XPath, text). 4) Interacting: clicking, typing, selecting, dragging. 5) Waiting for conditions (page load, element visible). 6) Extracting data from pages. 7) Handling multiple tabs/pages. 8) Taking screenshots. For agents, Playwright provides the 'hands' that execute actions. The agent decides what to do; Playwright does it reliably. It handles the complexities of browser quirks, network conditions, and timing.

Building browser automation from scratch is complex - dealing with asynchronous loading, element visibility, race conditions, and cross-browser differences. Playwright abstracts this, providing a reliable foundation. Agents built on Playwright can focus on high-level reasoning while Playwright ensures actions succeed. It's the standard for modern browser automation.

Browser agent using Playwright: Agent decides to click the Add to Cart button. Playwright: finds button by text Add to Cart, waits for it to be visible and enabled, clicks, waits for navigation/cart update, returns success/failure to agent. Agent doesn't need to know about element selectors or waiting logic. This separation of concerns makes agent development faster and more reliable.

Browser agents are AI-powered systems that can understand and navigate websites dynamically, making decisions based on page content and user goals. Web scraping tools are programs that extract data from websites using predefined rules and selectors. The key difference is adaptability and intelligence.

Web scraping tools: require manual configuration - you specify URLs, CSS selectors, and extraction rules. They break when site structure changes. They can't handle complex workflows or make decisions. Browser agents: use LLMs to understand page content and goals. They can adapt to different site layouts, handle pop-ups, make choices (which link to click), and complete multi-step tasks. They don't need pre-programmed selectors - they 'see' the page and reason about what to do.

Web scraping tools are brittle and require constant maintenance. Browser agents are flexible and can handle novel situations. For tasks like 'monitor competitor prices', a scraper needs custom code per site and breaks when sites redesign. An agent can adapt, reading the page visually or via DOM and understanding what elements mean. This robustness makes agents suitable for large-scale, dynamic web automation.

Task: 'Find the cheapest flight from NYC to London next Friday.' Web scraper: needs predefined selectors for each airline site's search form, results page, price display. If any site changes, scraper breaks. Browser agent: navigates to each site, visually identifies search fields (departure, destination, date), fills them, clicks search, reads results, compares prices. Works even if sites redesign, because agent understands the goal, not just fixed selectors.

LLMs can perceive web pages through different representations: screenshots (visual), DOM (structured HTML), or accessibility tree (semantic structure). Each has trade-offs in information richness, processing efficiency, and model compatibility.

1. Screenshot: page image passed to vision-language model. Model sees layout, colors, visual relationships. Works for any page, but high token cost, can't see hidden elements, limited text recognition. 2) DOM: raw HTML structure passed as text. Contains all elements, attributes, text. Can be huge (entire page HTML), requires parsing, but complete information. 3) Accessibility tree: simplified structure designed for screen readers - elements with roles, names, values. Smaller than DOM, semantic, designed for machine consumption. Often best balance of completeness and size. Some agents use multiple representations: accessibility tree for structure, screenshots for visual context.

Representation choice affects agent capability and cost. Screenshot works universally but expensive. DOM complete but noisy. Accessibility tree optimal when available (modern browsers expose it). For production agents, using accessibility tree with fallback to screenshot for visual-only sites is common. Understanding trade-offs helps design efficient, capable agents.

Login page. Accessibility tree: contains elements with roles 'textbox' for username/password fields, 'button' for login, with accessible names. Agent easily identifies what to do. Screenshot: model must visually locate fields, may misidentify. DOM: contains all HTML, including hidden elements, scripts - much larger, harder to parse. Accessibility tree provides clean, semantic representation perfect for agent decision-making.

Building reliable browser agents is challenging due to web complexity: dynamic content, varying layouts, authentication, pop-ups, and the inherent ambiguity of visual understanding. Agents must handle these robustly to be useful in production.

Key challenges: 1) Dynamic content - pages change after load (AJAX, single-page apps). Agent must wait for content to appear. 2) Layout variation - same action (e.g., 'click login') may require different coordinates/selectors on different sites. 3) Authentication - logins, CAPTCHAs, 2FA block automation. 4) Pop-ups and overlays - cookie notices, modals that block interaction. 5) Element identification - finding the right button among many visually similar. 6) Page complexity - large pages overwhelm context. 7) Site changes - redesigns break learned patterns. 8) Rate limiting - sites may block automated access.

These challenges make browser agents hard to productionize. A 90% reliable agent may still fail on critical paths. Solving them requires: robust waiting strategies, fallback mechanisms, human-in-loop for authentication, and continuous adaptation. For enterprise use, reliability often requires combining multiple approaches and extensive testing.

Agent booking flight on airline site. Challenges: 1) Date picker is custom JavaScript - agent must figure out interaction. 2) After search, results load dynamically - need to wait correctly. 3) Pop-up offers seat upgrade - agent must close or handle. 4) Site redesign happens - previously working agent breaks. Each challenge requires specific handling. Without robust design, agent fails frequently, frustrating users.

A GUI agent interacts with desktop applications through the graphical user interface, simulating mouse and keyboard input to control software just like a human. Unlike browser agents, they work with native apps (Excel, Photoshop, custom enterprise software) that lack web interfaces.

GUI agents use: 1) Screen capture - take screenshots of application windows. 2) Computer vision - identify UI elements (buttons, fields, menus). 3) Accessibility APIs - on Windows (UI Automation), macOS (Accessibility), to get structured element information. 4) Input simulation - send mouse clicks, keyboard input via system APIs. They can launch applications, navigate menus, fill forms, and extract data. Common frameworks: PyAutoGUI, SikuliX, WinAppDriver. More advanced agents use LLMs to reason about screenshots and decide actions.

Many business processes rely on desktop applications that lack APIs - legacy systems, specialized software, or tools not designed for automation. GUI agents can automate these, enabling RPA (robotic process automation) for tasks like data entry across multiple applications, report generation, and software testing.

Accounting task: agent needs to extract data from emailed PDF, enter into Excel, then upload to legacy accounting software. GUI agent: 1) Opens email client, finds attachment. 2) Opens PDF, extracts numbers (OCR if needed). 3) Opens Excel, navigates to correct cells, enters data. 4) Opens accounting software, logs in, navigates to data entry screen. 5) Fills form with extracted data. 6) Saves. All through GUI interaction, no APIs. This automates a previously manual task.

Giving AI agents browser control introduces significant security risks: they could visit malicious sites, enter credentials into phishing pages, download malware, or perform unauthorized actions on legitimate sites (e.g., posting, purchasing). These risks must be carefully mitigated.

Key risks: 1) Phishing - agent could be directed to fake login page and enter real credentials. 2) Malicious actions - agent could be instructed to post harmful content, delete data, make purchases. 3) Data exfiltration - agent could read sensitive data and transmit it. 4) Drive-by downloads - visiting malicious sites could download malware. 5) Session hijacking - if agent logged into sites, attacker could use session. 6) Prompt injection - web content could contain instructions that manipulate agent. 7) Resource abuse - agent could be used for DDoS, ad fraud.

Browser agents have real power - they can act as the user. Compromised agent could cause significant harm: financial loss, data breach, reputational damage. Security must be designed in: sandboxed environments, allowlists of permitted sites, human approval for sensitive actions, read-only modes, and strict input validation. For production, security often limits agent autonomy.

Agent asked to 'find information about competitor pricing'. It searches, clicks links. One link leads to phishing site designed to look like login page. Agent, thinking it needs to log in to see pricing, enters user's stored credentials. Attacker captures them. Mitigation: agent only allowed on pre-approved domains, never enters credentials without human approval, uses sandboxed browser isolated from real credentials. These controls prevent compromise.

CAPTCHAs and anti-bot measures are designed to distinguish humans from automated agents, posing a significant challenge for browser agents. Handling them requires a combination of avoidance, solving services, and human fallback.

Strategies: 1) Avoidance - design agent behavior to appear human-like: random delays, mouse movements, variable speeds. Respect robots.txt, rate limits. 2) CAPTCHA solving services - use third-party services (2Captcha, Anti-Captcha) that employ humans or AI to solve CAPTCHAs. Cost per solve. 3) Machine learning - train models to solve specific CAPTCHA types (less common now). 4) Session reuse - maintain authenticated sessions to avoid repeated CAPTCHAs. 5) Human fallback - when CAPTCHA detected, pause and ask human to solve. 6) Alternative paths - if site blocks, try different site or method.

CAPTCHAs are increasingly sophisticated and common. A browser agent that can't handle them is useless on many sites. However, solving CAPTCHAs programmatically may violate terms of service. The approach must balance effectiveness with legality. For enterprise use, often combine human fallback for occasional CAPTCHAs with careful behavior to avoid triggering them.

Price monitoring agent encounters CAPTCHA on retail site. Strategy: 1) Agent detects CAPTCHA image. 2) Pauses automation. 3) Sends CAPTCHA to solving service (cost $0.01). 4) Receives solution, enters it, continues. If solving fails or site blocks, fallback to human: alert operator to solve manually once, then resume. This keeps automation running while managing costs and compliance.

Grounding in computer use agents refers to connecting the agent's abstract goals (e.g., 'click the login button') to concrete, executable actions in the specific interface (e.g., moving mouse to coordinates (x,y) and clicking). It's the bridge between reasoning and physical interaction.

Grounding involves: 1) Perceptual grounding - identifying UI elements from visual or structural input. 'Where is the login button?' This requires understanding element roles, labels, positions. 2) Action grounding - mapping intentions to specific actions. 'Click' becomes mouse movement to element coordinates + click. 3) Context grounding - understanding current state (page loaded? element enabled?). 4) Feedback grounding - interpreting results (did click work? page changed?). Grounding failures cause agents to click wrong places, misread state, or take inappropriate actions.

Poor grounding is a major source of agent errors. An agent may correctly reason 'need to click login' but click the signup button instead because grounding failed. Reliable grounding requires accurate perception (vision/accessibility) and precise action execution. It's what makes agent decisions actually work in the real interface.

Agent on login page. Goal: 'log in'. Grounding steps: 1) Perceptual: find username field (role='textbox', name='username'), password field, login button. 2) Action: type username into field (needs field coordinates), type password, click button. 3) Context: verify fields enabled, button clickable. 4) Feedback: after click, page changes to dashboard - grounding successful. If button identified incorrectly, agent clicks wrong place - grounding failure. Good grounding makes all this work.

Evaluating browser agents requires measuring both task completion and execution quality: did the agent achieve the goal? How efficiently? Did it follow desired paths? Did it handle errors gracefully? Evaluation must account for web variability and agent non-determinism.

Evaluation methods: 1) Task success - binary, did agent complete goal? Verify final state (e.g., item in cart, data extracted). 2) Step efficiency - number of actions vs optimal path. 3) Time to completion - total duration. 4) Error rate - how many failed actions, retries. 5) Path correctness - did agent take appropriate steps? (e.g., logged in before purchasing). 6) Robustness - success across different sites, layouts, conditions. 7) Cost - tokens used, CAPTCHA solves, API calls. 8) Human evaluation - rate interaction quality, naturalness.

Success alone isn't enough. An agent that succeeds but takes 10x steps, clicks randomly, or triggers CAPTCHAs is not production-ready. Comprehensive evaluation reveals these issues, guiding improvement. For benchmarking, standardized environments (WebArena, MiniWoB++) provide reproducible tasks and metrics.

Flight booking agent evaluated on 100 tasks. Success rate: 85% (good). But average steps: 45 vs optimal 15 - inefficient. Error rate: 30% of actions fail (retries). Path analysis: agent often searches multiple times unnecessarily. Cost: $0.80 per booking vs $0.30 benchmark. This evaluation shows need for efficiency improvements despite decent success rate. Without these metrics, would miss optimization opportunities.

Pixel-based interaction uses computer vision on screenshots to identify UI elements by their visual appearance. DOM-based interaction uses the underlying HTML structure to find elements via selectors, IDs, or accessibility attributes. Each has distinct advantages and limitations.

Pixel-based: agent takes screenshot, uses vision-language model or CV to locate elements (e.g., 'find the blue button with Submit'). Works on any visual interface, including images, videos, and canvas elements. But slower, less precise, can't see hidden elements, affected by visual changes. DOM-based: agent accesses page's HTML or accessibility tree, finds elements by role, text, or attributes. Fast, precise, sees everything, but requires structured access (not available in all contexts, e.g., images).

Choice affects reliability and capability. DOM-based is generally more reliable for web automation where structure available. Pixel-based essential for: images, video, canvas, or when DOM access restricted. Many agents use hybrid: DOM first, fallback to pixel when needed. Understanding trade-offs helps design robust agents.

Finding 'Add to Cart' button. DOM-based: find element with role='button' and text='Add to Cart' - fast, reliable. Pixel-based: scan screenshot, identify button visually - slower, may mistake similar buttons. But if button is an image without text, DOM-based fails, pixel-based may succeed. Hybrid: try DOM first, if not found (button is image), use pixel. Best of both.

Dynamic and JavaScript-heavy pages load content asynchronously, update without navigation, and have elements that appear/disappear. Browser agents must handle this complexity to interact reliably, requiring sophisticated waiting strategies and state management.

Strategies: 1) Intelligent waiting - not just fixed delays, but waiting for specific conditions: element visible, enabled, text present, network idle. 2) Mutation observation - detect DOM changes to know when content loaded. 3) Retry logic - if action fails, retry after short delay (page may still loading). 4) State tracking - maintain understanding of page state (e.g., 'after clicking search, waiting for results'). 5) Handling SPAs - single-page apps require different navigation model (no page loads). 6) Fallback to visual - if DOM unreliable, use screenshot to verify state.

Modern web is dynamic. An agent that assumes static pages will fail constantly - clicking elements before they exist, misinterpreting loading states, getting stuck. Robust handling of dynamics is essential for real-world performance. It's often the difference between a demo and production-ready agent.

E-commerce search: agent clicks 'Search', page shows loading spinner, then results load dynamically. Without proper waiting, agent might try to click first result before it exists (fails), or assume no results (wrong). Good agent: after click, waits for spinner to appear and disappear, or waits for results container to have children, then proceeds. This handles variable load times and ensures reliable interaction.

Browser agents fail in predictable ways: element not found, wrong element clicked, timing issues, navigation failures, and unexpected page states. Understanding these failure modes helps design robust agents and effective error handling.

Common failures: 1) Element not found - agent looks for element that doesn't exist (page changed, misidentified). 2) Wrong element - clicks similar-looking but wrong button. 3) Timing - acts before page ready, element still loading. 4) Navigation failure - click doesn't trigger expected navigation. 5) State confusion - agent misinterprets page state (e.g., thinks logged in when not). 6) Pop-up interference - unexpected modal blocks interaction. 7) Authentication issues - login fails, session expires. 8) Site changes - redesign breaks previously working flows. 9) Rate limiting - site blocks automation.

Understanding failure modes enables targeted fixes. If agent frequently clicks wrong elements, improve element identification (use multiple selectors, add verification). If timing issues, improve waiting strategies. For production, build error handling that can recover from common failures - retry, alternative paths, human escalation.

Agent booking flight: fails because date picker is custom calendar. Failure mode: element not found (agent looked for standard date input, but site uses JavaScript calendar). Fix: add specialized handler for date pickers - detect calendar widget, use arrow keys to navigate. Another failure: after search, results page has 'Sort by' dropdown that agent clicks, but dropdown options appear slowly - timing failure. Fix: wait for dropdown options visible before clicking. Each failure mode addressed makes agent more robust.

Selenium, Playwright, and Puppeteer are browser automation libraries that provide APIs to control browsers. They differ in architecture, language support, and features, affecting their suitability for agent development.

Selenium: oldest, supports multiple browsers (Chrome, Firefox, Safari), uses WebDriver protocol. Broad language support, large community. Can be slower, more prone to flakiness. Playwright: modern, by Microsoft, supports Chromium, Firefox, WebKit. Auto-waiting, reliable selectors, network control, mobile emulation. Excellent for agents. Puppeteer: by Google, focused on Chromium. Fast, powerful, but Chrome-only. Playwright evolved from Puppeteer with multi-browser support.

For agent development, Playwright is generally preferred due to its auto-waiting (reduces timing failures), comprehensive API, and cross-browser support. Selenium still used in legacy systems. Puppeteer good for Chrome-only scenarios. Choice affects agent reliability and development speed. Playwright's auto-waiting alone reduces many common agent failures.

Agent needs to click button that appears after dynamic load. With Selenium: must write explicit wait code. With Playwright: page.click('button') automatically waits for element to be visible and enabled. This reduces agent complexity and failures. For multi-browser testing (Chrome, Firefox, Safari), Playwright's unified API invaluable. This is why many agent frameworks build on Playwright.

A web agent benchmark is a standardized evaluation suite for browser agents, consisting of tasks on simulated or real websites, with clear success criteria and metrics. It enables objective comparison of different agent approaches and tracking progress over time.

Structure: 1) Task set - dozens to hundreds of web navigation tasks (e.g., 'book flight on Expedia', 'find price of iPhone on Amazon'). Tasks vary in complexity. 2) Environment - simulated websites (MiniWoB, WebShop) or live sites with controlled conditions (WebArena). 3) Success criteria - defined outcomes (e.g., item in cart, correct data extracted). 4) Metrics - success rate, steps, time, cost. 5) Evaluation protocol - how agents interact (via DOM, screenshots), number of runs. 6) Leaderboard - rankings of different agents. Examples: MiniWoB++ (simplified web tasks), WebArena (realistic e-commerce, social, forum sites), Mind2Web (cross-domain tasks).

Benchmarks drive progress by providing common yardsticks. They reveal what works and where challenges remain. For researchers, they enable systematic experimentation. For practitioners, they help select agent frameworks and identify areas needing improvement.

WebArena benchmark includes 100+ tasks on simulated versions of Amazon, Reddit, GitLab, etc. Agent must complete tasks like 'post a comment on Reddit', 'create a GitLab issue'. Success rate across all tasks reported. If Agent A scores 45%, Agent B 38%, A is better. Breakdown by task type reveals strengths: Agent A good at e-commerce but poor at social sites. This granular insight guides research.

Logging and replay for browser agents capture the full interaction sequence - screenshots, actions, page state - enabling post-mortem debugging of failures. Given agent non-determinism, this is essential for understanding why failures occurred and how to fix them.

Implementation: 1) Action logging - record every agent action (thought, tool call, mouse movement, click) with timestamp. 2) State capture - take screenshot after each action, save page HTML/DOM snapshot. 3) Metadata - record URLs, page titles, error messages. 4) Trace storage - save all data structured for replay. 5) Replay tool - can replay step-by-step, showing screenshots and actions, allowing inspection. 6) Search - ability to find sessions by failure type, query. 7) Integration with monitoring - automatic logging of all production sessions.

When agent fails, you need to know why. Did it misidentify element? Timeout? Wrong reasoning? Logging captures everything, enabling root cause analysis. Without logs, you're guessing. Replay lets you step through failure like a video, seeing exactly what agent saw and did. This is essential for debugging and improvement.

Agent fails to book flight. Logs show: step1: saw homepage, clicked 'Flights' (good). step2: entered dates (good). step3: clicked 'Search' but screenshot shows search button was actually 'Search Hotels' - agent misidentified. Replay shows exact visual. Fix: improve element identification for similar-looking buttons. Without logs, would have vague idea of failure; with logs, precise fix.

Computer use agents are most suitable for tasks that are repetitive, rules-based, and time-consuming for humans, but where APIs are unavailable or would be too complex to build. They excel at GUI automation across web and desktop applications.

Suitable use cases: 1) Data entry across multiple systems - copying data from one app to another. 2) Web scraping at scale - monitoring prices, extracting leads. 3) Software testing - automated UI testing across browsers. 4) RPA (robotic process automation) - automating legacy enterprise software. 5) Personal assistance - repetitive online tasks (bill payment, form filling). 6) Research - gathering information from multiple sites. 7) Accessibility - helping users with disabilities navigate computers. 8) Training - demonstrating software workflows.

Computer use agents are still emerging; they're not yet reliable for all tasks. Focusing on suitable use cases where they excel - structured, predictable workflows - yields highest value. For tasks requiring high reliability or complex reasoning, human oversight still needed. Understanding sweet spots helps prioritize investment.

Insurance company uses computer use agent to process claims: agent logs into multiple systems (policy database, claims system, payment portal), extracts data, enters into forms, updates records. This previously took human 20 minutes per claim; agent does in 3 minutes with 95% success. High-value, structured task ideal for automation. Contrast with creative task like 'design a marketing campaign' - not suitable. Match use case to capability.

A guardrail system for computer use agents is a multi-layered safety framework that prevents agents from taking harmful actions: visiting dangerous sites, entering credentials in wrong places, making unauthorized purchases, or deleting data. It's essential for safe deployment.

Guardrail layers: 1) URL allowlist/blocklist - agent can only navigate to approved domains. 2) Action approval - high-risk actions (purchases, data deletion) require human approval. 3) Input validation - prevent agent from entering credentials into non-approved fields. 4) Rate limiting - limit actions per minute to prevent abuse. 5) Read-only mode - for sensitive systems, agent can only read, not write. 6) Session isolation - agent runs in sandboxed environment, separate from user's real sessions. 7) Content filtering - block access to known malicious sites. 8) Monitoring and alerting - detect suspicious behavior patterns. 9) Human oversight - random sampling of agent actions for review.

Computer use agents have real power to act in the digital world. Without guardrails, a misstep or malicious prompt could cause significant harm. Guardrails provide safety boundaries, ensuring agent operates within intended limits. For production, they're not optional - they're essential for risk management and user trust.

E-commerce agent with guardrails: 1) URL allowlist: only amazon.com, bestbuy.com - prevents visiting phishing sites. 2) Purchase > $100 requires human approval via text. 3) Never stores or enters credit card info - uses saved payment method with confirmation. 4) Rate limit: max 10 actions/minute prevents rapid-fire abuse. 5) Sandboxed browser has no access to user's other tabs/data. These guardrails allow agent to be useful while preventing common failure modes. Without them, one mistake could cost real money.