Agent Flows
All about blocks
Web Scraper
Web Scraper

Usage

The web scraper block is used to scrape a website and extract the content. Currently the web-scraper will return the parsed TEXT content of the page - not the HTML.

The purpose of this block is to allow you to scrape a website and extract the content you need as if you were accessing the website directly. If you are looking for more programmatic access to the HTML, you should use the API call block.

Input Variables

  • URL to scrape: The URL of the website to scrape.
  • Result Variable: The variable to store the result of the web scraping.