web crawler

noun
UK: /ˈweb ˌkrɔː.lər/
US: /ˈweb ˌkrɑː.lɚ/

  1. A computer program that automatically browses the World Wide Web to collect data and index websites.

    1. The web crawler indexes sites.
    2. Our web crawler systematically explores the internet, gathering information for search engines.

  2. A type of bot used by search engines to discover and add new web pages to their index.

    1. A web crawler checks for updates.
    2. The search engine utilizes a sophisticated web crawler to maintain an up-to-date index of the web.

Synonyms

spider bot crawler

Frequently Asked Questions

The word "web crawler" in English means: A computer program that automatically browses the World Wide Web to collect data and index websites., A type of bot used by search engines to discover and add new web pages to their index..

The phonetic transcription of "web crawler" is /ˈweb ˌkrɔː.lər/ in British English and /ˈweb ˌkrɑː.lɚ/ in American English. Click the 🔊 button to hear both pronunciations.

Synonyms for "web crawler": spider, bot, crawler.

Example usage of "web crawler": "The web crawler indexes sites.". More examples on the page.