HTML Entities: Guide to Character References

Syntax and Parser Mechanics

Tim Berners-Lee specified the original four character entities in the 1991 HTML design document to solve an unambiguous syntactic collision: the characters < and > define tags, " delimits attributes, and & initiates escape sequences. If an HTML parser encounters a raw < inside character data, it shifts state from the data state into the tag open state, assuming the subsequent letters form a tag name. An ampersand forces the parser into the character reference state, where it consumes characters until it either encounters a terminating semicolon or hits a delimiter rule.

Character references come in three distinct formats:

• Named character references: Mnemonic representations such as &, ©, or €. These map directly to predefined strings within the WHATWG HTML Living Standard entity table.
• Decimal numeric references: Base-10 references in the format &#D;, where D represents the integer code point assigned to that glyph in the ISO/IEC 10646 / Unicode standard (for example, © for ©).
• Hexadecimal numeric references: Base-16 references written as &#xH; or &#XH;, where H is the case-insensitive hexadecimal value of the Unicode code point (such as © or €).

A frequent point of confusion concerns the numeric base: decimal references never contain letters. Writing &#A9; is a syntax error because the lack of an x causes the parser to expect base-10 digits. The browser parses up to the invalid character A, fails to find a valid integer, and either outputs the literal text or leaves the DOM tree corrupted depending on the document mode.

The Five Predefined Entities and Attribute Contexts

The XML 1.0 specification hardcoded exactly five predefined entities: ", &, ', <, and >. HTML carries these forward, but applies subtle context rules depending on whether text sits between tags or inside attribute values.

Character	Named Entity	Decimal	Hexadecimal	Primary Context Requiring Escaping
&	&	&	&	Everywhere in text nodes and attribute values.
<	&lt;	&#60;	&#x3C;	Everywhere in text nodes and attributes.
>	&gt;	&#62;	&#x3E;	Following a tag or inside legacy XML contexts.
"	&quot;	&#34;	&#x22;	Inside double-quoted attribute values.
'	&apos;	&#39;	&#x27;	Inside single-quoted attribute values.

The entity ' was standard in XML from 1998, but the W3C did not officially adopt it into HTML until HTML5. Internet Explorer 8 and earlier versions failed to render ' when parsing standard text/html documents, treating it as unknown plain text. For backwards compatibility across legacy browsers, developers historically relied on numeric entity ' or ' instead.

Attribute quoting directly alters whether a quote character requires an entity. In <input value="O'Reilly">, the apostrophe requires no escape because the attribute itself uses double quotation marks. Conversely, writing <input value="He said "Hello""> breaks parsing immediately; the second quotation mark terminates the attribute string, leaving Hello"" as orphaned, invalid attributes. Here, " is mandatory.

The Legacy Semantics of Missing Semicolons

Netscape Navigator 1.1 introduced an error-tolerant entity parser in 1995 that accepted named entities without trailing semicolons if the following character was not alphanumeric. This decision created quirks that persist in modern web engines. Under the WHATWG standard, a subset of named character references are classified as "legacy named character references that do not require a semicolon" when used outside attributes.

Consider the query string inside a hyperlink: <a href="index.php?order=desc&copy=1">. Because © is a valid reference for the copyright symbol (©), early parsers saw &copy followed by an equals sign, recognized the legacy entity, and transformed the URL into index.php?order=desc©=1. The server then received a parameter named © instead of copy.

HTML5 resolved this ambiguity by establishing strict boundary conditions inside attribute values. If an entity lacks a semicolon inside an attribute and the next character is an equals sign (=), an alphanumeric character (letters from printable ASCII), or an underscore, the HTML parser does not resolve the reference. It leaves the characters intact as raw text. Even with this safeguard in place, modern linters treat naked ampersands inside URL parameters as markup errors. The valid, bulletproof representation is always <a href="index.php?order=desc&amp;copy=1">.

Unicode, Windows-1252, and Numeric Reference Re-mapping

Numeric character references in HTML always refer to Unicode code points, not to bytes in the document's declared character sets. Even if an HTML document is served with the header Content-Type: text/html; charset=ISO 8859-1, the numeric entity € resolves to the Euro sign (€, code point U+20AC), not whatever byte exists at position 0xAC in ISO-8859-1.

A historical conflict occurred around the range 0x80 through 0x9F (128 to 159). In the ISO-8859 family, these code points are non-printable C1 control characters, such as PAD (Padding) or HOP (High Octane Preset). However, Microsoft's Windows-1252 character set assigned popular punctuation symbols and typographical marks to these byte values, including the smart quotes (“, ”), the em-dash, and the trade mark sign (™).

Authors frequently pasted text from desktop word processors into web forms, resulting in numeric references like “ or ™. Strictly interpreted against Unicode, code point 147 (U+0093) is the control character "Set Transmit State", which produces no visible glyph. HTML5 formalized browser error-recovery behavior by requiring all user agents to parse numeric character references in the range 0x80 to 0x9F through a mapping table that converts them to their intended Windows-1252 Unicode counterparts:

• € (0x80) maps to U+20AC (€)
• … (0x85) maps to U+2026 (…)
• ‘ (0x91) maps to U+2018 (‘)
• ’ (0x92) maps to U+2019 (’)
• “ (0x93) maps to U+201C (“)
• ” (0x94) maps to U+201D (”)
• ™ (0x99) maps to U+2122 (™)

While browsers repair these numbers automatically, writing them violates the HTML specification and causes validation failures. The proper approach is to use the accurate Unicode code point: write ™ instead of ™, or declare UTF-8 in the document header and place the character directly into the file. Using raw bytes incorrectly can lead to corrupted display, requiring steps for fixing mojibake in downstream storage.

When to Use Entities Instead of Native UTF-8

RFC 3986, RFC 7230, and the W3C have converged on UTF-8 as the mandatory encoding standard for modern web development. When a document begins with <meta charset="UTF-8">, authors can type Japanese kanji, Arabic script, mathematical operators, and graphical icons directly into the document. Over-using entities creates bloated markup and makes source code difficult to read.

There are specific situations where character references remain practical:

1. Syntax Escapes: As long as HTML remains a markup language based on tags, delimiters like <, >, &, and " will remain mandatory when presenting code snippets or data within attributes.
2. Invisible Whitespace Disambiguation: Characters like the non-breaking space ( , U+00A0), zero-width space (​), or narrow no-break space ( ) look identical to an ordinary space bar press inside code editors. If a developer uses a raw non-breaking space in source code, another programmer might format the file and accidentally convert it into an ordinary space, breaking table column layouts or typography. Writing   communicates intent unambiguously.
3. Font and Symbol Catalogs: When compiling html symbols or testing system fallbacks for characters outside common fonts, writing numeric entities like ✔ allows inspectors to read the code point directly without needing a hex editor. You can convert values easily using a dedicated HTML entity tool.
4. Encoding-Constrained Environments: Automated build pipelines, legacy email generators, and SMS gateways often strip or corrupt multi-byte UTF-8 sequences. Using numeric references like é instead of a raw é ensures the character survives unchanged through 7-bit ASCII distribution channels. Reviewing how UTF-8 works reveals why encoding mismatches cause raw non-ASCII bytes to drop silently during serialization.

Security Considerations: Cross-Site Scripting (XSS)

Insufficient or context-insensitive character escaping is a primary root cause of Cross-Site Scripting (XSS) vulnerabilities. Developers often assume that running a string through a basic function that replaces &, <, and > makes user input safe everywhere. In practice, escaping requirements depend entirely on the document context.

Consider an unquoted HTML attribute: <input value="USER_DATA"> versus <input value=USER_DATA>. If an attribute has no surrounding quotes, an attacker does not need to use < or > to escape the tag. Submitting x onfocus=alert(1) injects a new attribute directly into the element because spaces delimit attributes in unquoted contexts. Running an HTML entity escaper that only targets tags leaves the space characters untouched, allowing full script execution.

Another dangerous pattern occurs when injecting dynamic data inside inline script blocks: <script>let name = "USER_DATA";</script>. If an author relies on standard HTML entities here, sending "; alert(1); // will not work directly, but sending </script><script>alert(1)</script> immediately breaks out of the context. An HTML parser gives precedence to the closing </script> tag even if it sits inside a JavaScript string literal. Entities like " are not evaluated as code by JavaScript engines; they remain literal strings inside the DOM, which can lead to double-escaping errors or script execution depending on how the data is loaded into the page.

Context-aware output encoding requires distinct rules:

• HTML Text Content: Replace & with &, < with <, and > with >.
• Quoted Attributes: Replace & with &, " with " (for double-quoted attributes), and ' with ' (for single-quoted attributes).
• Unquoted Attributes: Do not use unquoted attributes with untrusted input; no standard entity set cleanly protects against attribute space-injection.
• JavaScript Variables: Use JSON serialization (such as json_encode() or JSON.stringify()) combined with Unicode escaping (such as \u003C for <), rather than HTML character references.

Modern browser DOM APIs handle text node escaping natively. Setting element.textContent = untrustedInput safely converts characters into raw text without interpreting tags or entities, bypassing parser vulnerabilities entirely. Reserve manual entity substitution for server-side template engines and serialization workflows where raw string concatenation cannot be avoided.