TextKit

Text Encoding Explained: ASCII, UTF-8, and Unicode

Every character you type is stored as a number. The encoding standard defines which number maps to which character. ASCII (American Standard Code for Information Interchange, 1963) defined 128 characters using 7 bits — the English alphabet, digits, punctuation, and control codes. It worked fine for English but left no room for accented characters, non-Latin scripts, or emoji.

Unicode was created in 1991 to solve this by assigning a unique "code point" to every character in every writing system. Unicode currently defines over 149,000 characters across 161 scripts, plus emoji, mathematical symbols, and historical writing systems like Linear B and Egyptian hieroglyphics. Unicode code points are written as U+XXXX — so "A" is U+0041, "€" is U+20AC, and "😀" is U+1F600.

UTF-8 is the most common encoding for Unicode on the web (used by 98% of websites). It's a variable-width encoding: ASCII characters (U+0000 to U+007F) use 1 byte, Latin extended and common symbols use 2 bytes, most other scripts use 3 bytes, and emoji use 4 bytes. UTF-8's genius is backward compatibility — a valid ASCII file is already valid UTF-8. Our URL Encoder converts special characters to their percent-encoded UTF-8 byte sequences.

Why encoding errors happen: "Mojibake" (garbled text like "ÃÆ'©" instead of "é") occurs when text encoded in one format is decoded with a different format. The most common culprit: a file saved in Windows-1252 (a Western European encoding) being opened as UTF-8. Modern text systems almost universally use UTF-8, but legacy files and some database systems still use older encodings.

Base64 Encoding: What It Is and When You Need It

Base64 is not a compression format or an encryption format — it's a way to represent binary data using only printable ASCII characters. It takes 3 bytes of binary data and encodes them as 4 ASCII characters from a 64-character alphabet (A-Z, a-z, 0-9, +, /). The output is about 33% larger than the input.

Base64 exists because some systems — particularly email (SMTP), HTML attributes, and early internet protocols — were designed to handle text only. To embed binary data (images, files, certificates) in these text contexts, the binary must be converted to printable characters. This is why email attachments, CSS data URIs (background-image: url('data:image/png;base64,...')), and JWT tokens use base64 encoding.

Base64 is trivially reversible — it provides zero security. Anyone who sees a base64 string can decode it instantly. Our Base64 Encoder/Decoder converts text and files to and from base64 format for debugging, data URI creation, or API testing.

Hash Functions Explained: SHA-256, MD5, and Why They're One-Way

A cryptographic hash function takes any input (a word, a file, a database) and produces a fixed-length output (the hash or digest) with three key properties: deterministic (same input always → same output), avalanche effect (changing one bit of input changes ~50% of output bits), and one-way (computing the hash is fast; reversing it is computationally infeasible).

MD5 produces 128-bit (32 hex character) hashes and is fast — but cryptographically broken since 2004. Researchers found MD5 collision attacks (two different inputs producing the same hash). MD5 should never be used for security-sensitive purposes, but remains useful for data integrity checksums where collision attacks are not a concern.

SHA-256 produces 256-bit (64 hex character) hashes and remains cryptographically secure as of 2026. It's used in TLS/HTTPS certificate verification, Bitcoin's proof-of-work algorithm, code signing, and password storage (typically as PBKDF2 or bcrypt, which add salt and stretching on top of SHA-256). Our Hash Generator computes MD5, SHA-1, SHA-256, and SHA-512 hashes for any text input.