HTML tokenization

The parsing step that splits a stream of characters into meaningful pieces called tokens, such as a start tag, an end tag or a run of text.

In more detail

The HTML tokenizer is a state machine: each character moves it between states such as reading a tag name or reading an attribute value, and it emits a token whenever one is complete. The tree construction stage then turns those tokens into DOM nodes.

Related terms

Taught in

References