How a Single Frame Gets Compressed: Blocks, Frequencies, Quantization
I, P and B frames cover the space between frames; this is the story inside one frame. The encoder’s four-step pipeline squeezes a picture to a few percent of its raw size — and it is also where mosaic blocks and color bands are born.
Step one: split into blocks
The encoder divides the frame into small blocks (macroblocks, typically 16×16, subdividing to 4×4 or smaller) and processes each independently. The point is local treatment of local complexity: a flat sky block can be crushed, a detailed face block handled with care.
The block boundaries are also the side effect — the "mosaic" or blocky artifacts you see in broken quality are the grid showing itself when differences between blocks get amplified. Every blocky symptom traces back to this step’s structure.
Step two: frequency transform (DCT)
Each block’s pixel values are transformed into frequency coefficients (discrete cosine transform): low frequencies describe flat areas and main contours, high frequencies describe edges, texture and fine detail. Human eyes are far less sensitive to high frequencies — that is the whole lever of lossy compression.
The transform itself is lossless — it merely changes how the data is viewed: pixel values become frequency coefficients. Image compression (JPEG) and intra-frame video compression are fully the same animal here: an I frame is, in essence, a JPEG living inside a video.
Step three: quantization — the quality switch
Quantization is where loss happens: high-frequency coefficients get merged, small ones zeroed outright, low frequencies kept precise. The harsher the quantization, the more high frequencies die, the smaller the file, the heavier the artifacts. Sky gradients collapsing into bands and fuzzy block edges are the fingerprints of over-quantization.
CRF is, at its core, a quantization-strength control: lower numbers mean gentler quantization, higher quality, bigger files. It does not set the size directly — it sets "how much to throw away", and size is the result. Once that lands, compression parameters stop being folklore.
Step four: entropy coding, and the frame completes
The quantized coefficients get one more lossless pass (entropy coding), giving common patterns shorter codes — the same idea as zipping a file, applied to video coefficients. At this point one frame sits at a few percent of its raw size.
I frames run all four steps in full; P and B frames predict from neighbors first, then run the same pipeline on the residual — and when the residual is nearly zero, quantization and entropy coding have almost nothing to chew on, which is why P/B frames are so cheap. The time dimension (between frames) and the space dimension (within a frame) are the two legs of all modern video compression.