search engine crawler
A program run by a search engine that downloads pages and reads their structure to index what they are about.
In more detail
Many crawlers now run JavaScript, but content present in the initial HTML is still found most reliably. Crawlers use semantic elements and headings to understand which content matters, so good structure helps both accessibility and search.
Related terms
- semantic HTML — Using HTML elements for their meaning, such as <button> for actions and <nav> for navigation, rather than generic <div>s.
- heading hierarchy — The outline formed by headings h1 to h6, which should go down one level at a time so the structure of the page is clear.
Taught in
- Semantic HTML Elements: Landmarks And Structure — Frontend Architecture & Web Platform