HTML tokenization
The parsing step that splits a stream of characters into meaningful pieces called tokens, such as a start tag, an end tag or a run of text.
In more detail
The HTML tokenizer is a state machine: each character moves it between states such as reading a tag name or reading an attribute value, and it emits a token whenever one is complete. The tree construction stage then turns those tokens into DOM nodes.
Related terms
- HTML parser — The part of the browser that reads HTML text and builds the DOM tree from it, one token at a time.
- state machine — A model in which something is always in exactly one named state, such as idle, loading, success or error, and only defined events move it to another.
- tree construction — The parsing stage that takes tokens from the tokenizer and assembles them into the DOM tree, tracking which elements are still open.
Taught in
- HTML Parsing, Tokenization And The Document Object Model (DOM) — Frontend Architecture & Web Platform