# Web Crawling

> The automated process of browsing the web to collect information about websites and their pages, often used by search engines to index web content.

Source: https://dagster.io/glossary/web-crawling