The panel does not read all pixels of a large photo. It computes a stride — every 4th pixel at 80,000 total (20,000 samples), every 7th at 153,600 (21,943) — so the statistics see roughly 20,000 samples, not the whole frame. Each sample is a triple of 0–255 values; the average rounds each channel to an integer. The histogram throws away the low six bits (value >> 2), which is why a photo with a single flat colour shows one 4-value-wide spike instead of a single bar.
Median cut is the palette step. It starts with one box holding every sample, then repeatedly takes the box with the widest channel range, sorts that box along the channel, and splits it in half. After six boxes exist, each box contributes its mean colour. That is why the palette above returns the four quadrant colours intact — each cluster is tight enough that no split turns two of them into one mixed swatch, and no further split has a range left to cut. On a real photograph the same process mixes gradients into "average of the sky" swatches, which is the point: six representatives of a continuous distribution.