UTF-8
UTF-8 encodes a Unicode codepoint into one to four bytes. The first byte’s own leading bits announce how many bytes the whole sequence has (0 for one byte, 110 for two, 1110 for three, 11110 for four); every byte after the first starts with the fixed marker 10, leaving six payload bits each. The codepoint’s own bits are distributed across whatever payload positions are available, most significant bits first.
When it is the right choice
The encoding essentially every modern text format and network protocol uses for Unicode, specifically because it is backward-compatible with plain ASCII (every ASCII byte is already a valid one-byte UTF-8 sequence) and self-synchronising (a decoder can tell from any single byte, on its own, whether that byte starts a sequence or continues one).
How it works
- Decide the byte count from the codepoint’s own numeric range: up to U+007F is one byte, up to U+07FF is two, up to U+FFFF is three, otherwise four.
- Write the fixed header bits onto the first byte, then fill its remaining bits with the codepoint’s own most-significant bits.
- Write the fixed 10 marker onto every later byte, then fill its remaining six bits with the next six bits of the codepoint, most significant first.
- To decode, strip each byte’s header or continuation marker and concatenate what remains, in the same order.
Worth noticing
- The byte count is decided entirely by the codepoint’s numeric range, not by how visually complex the character looks — an emoji needs four bytes because its codepoint is simply a large number.
- Every continuation byte looks the same (starts 10) regardless of which character it belongs to — the structure, not the content, is what a decoder reads first.
- An astral character (anything above U+FFFF, including most emoji) is stored by JavaScript strings as a two-unit UTF-16 surrogate pair, but is still exactly one Unicode scalar value and one UTF-8 sequence.
Cost
- Best
- O(1)
- Average
- O(1)
- Worst
- O(1)
- Extra space
- O(1)