Any cache between the client and this server can answer a request without the server observing it. Such a request is not recorded as cached; it is not recorded at all. The study therefore configures its delivery path to disable intermediate caching, and verifies from outside that requests are reaching the origin rather than assuming it.
If this server failed to send version fingerprints, no client could revalidate and every crawler would appear not to cache. The measurement would then describe a server misconfiguration while appearing to describe crawler behaviour. Responses here always carry both an entity tag and a modification date, and visits where a fingerprint was absent are excluded from all conclusions.
A client that does not re-fetch may have retained the page, or may have reused a previous answer without consulting the page at all. Server logs alone cannot separate these. The study separates them by varying the question asked while holding the requested page constant: a reused answer cannot survive a changed question, whereas a retained page can.
Results describe the crawlers that visited these pages during the measurement window. Behaviour observed through a provider's programmatic interface does not necessarily match the behaviour of its consumer product, and the two are reported separately where both are measured.
Part of an open study of AI crawler cache behaviour. Study reference RCP-2026-08. These pages are served by the study's own logging server; requests to them are recorded for the purpose described in the method. Content is stable by design and is not edited during a measurement run.