{"article":{"id":49002658519569,"url":"https://qwant.zendesk.com/api/v2/help_center/en-us/articles/49002658519569.json","html_url":"https://help.qwant.com/hc/en-us/articles/49002658519569-How-Qwant-Indexes-the-Web","author_id":46426822342289,"comments_disabled":true,"draft":false,"promoted":false,"position":0,"vote_sum":0,"vote_count":0,"upvote_count":0,"downvote_count":0,"section_id":47119696763537,"created_at":"2026-07-21T16:07:50Z","updated_at":"2026-10-16T17:32:38Z","name":"How Qwant Indexes the Web","title":"How Qwant Indexes the Web","source_locale":"fr-fr","locale":"en-us","outdated":false,"outdated_locales":[],"edited_at":"2026-07-21T16:09:21Z","user_segment_id":null,"permission_group_id":32021775980433,"content_tag_ids":[],"label_names":[],"body":"

To provide you with results, Qwant maintains a continuously updated index of public web pages. Here’s how it works.

Crawlers

Qwant uses crawlers (robots d'exploration) that automatically browse the web. QwantBot — Qwant’s main crawler — is identifiable in your server logs.

The process happens in two steps:

Crawling

    \n
  1. The crawler discovers new pages by following links from already known pages or via a sitemap file.
    2. \n
  2. It respects the directives in each domain’s robots.txt file.
    3. \n
  3. It downloads the HTML content of accessible pages (HTTP code 200, without blocking by captcha or error 403).
    4. \n

Indexing

    \n
  1. Each crawled page is evaluated according to quality criteria.
    2. \n
  2. Pages that pass this evaluation are indexed and become candidates for display in the results.
    3. \n

Crawl Frequency

The frequency with which our crawlers visit a site depends on:

What Is Indexed

Qwant indexes publicly accessible HTML pages, as well as several file formats: PDF, office documents (Word, Excel, PowerPoint, OpenDocument), plain text.

What Is Not Indexed

Tips for Publishers

To optimize your site’s indexing by Qwant:

","user_segment_ids":[]}}