Fundamentals of Multimedia Second Edition

If you build anything with media — apps, games, websites — this book explains the machinery underneath your tools. It suits students in multimedia or computer science courses, plus working developers who never took the theory. The math is real but always tied to a practical format you already use, like JPEG or MP3, so nothing floats free.

1. 1Ch. 3 (Digital Images), Ch. 5 (Color)

   A picture is a grid of numbers

   Open any photo on your phone and zoom in far enough: the smooth face dissolves into a rectangle of tiny colored squares. The book starts here, with the digitization of images — sampling a continuous scene into discrete pixels, each one just a number for brightness. Then color complicates things: human vision responds to three kinds of receptors, so the book introduces tristimulus values and shows how displays fake every hue by mixing three primaries. What looked like a rich photograph is revealed as arithmetic — millions of little numbers, each pretending to be light.

   The idea

   Digitization is a bargain: you trade an infinite, continuous world for a finite list of numbers, and the sampling and quantization steps decide how good the trade is. The surprise is how little you need — because human vision has limits, a modest grid of coarse color values can look seamless. Once you see an image as data, everything else in the book — compression, enhancement, transmission — becomes a question of manipulating those numbers efficiently.

   Try this: Zoom into a photo on your screen until you can see individual pixels, and notice how a smooth edge is really a staircase of colored squares.

2. 2Ch. 5 (Color)

   Three numbers to fake every color

   The book pauses on a deceptively simple question: how can a screen show millions of colors using only red, green, and blue light? The answer comes from color science — the human eye has three types of cones, so any stimulus can be matched by adjusting three primary lights. This gives the RGB color model, and its cousin CMY for printing, where inks subtract light instead of adding it. The chapter walks through these models and the conversions between them, showing why a color that glows on your monitor can look dull when printed on paper.

   The idea

   Color models are not descriptions of light — they are descriptions of human vision. RGB works because your eye reduces every wavelength mixture to three signals, so three numbers suffice to fool it. The gap between additive and subtractive color explains a daily frustration: screens and printers speak different color languages, and converting between them loses something. Understanding this makes color management feel less like vendor nonsense and more like translation between species of device.

   Try this: Open an image editor and drag a photo's saturation slider to zero — watch three color channels collapse into one gray value.

3. 3Ch. 7 (Lossless), Ch. 8 (Lossy)

   Compression: throwing away what you can't see

   A raw image is enormous, so the book turns to compression. First the lossless kind: run-length coding, dictionary methods, and Huffman coding, which assigns short codes to common symbols and long codes to rare ones — all reversible, nothing lost. Then the leap to lossy compression, where the goal changes: discard information your eyes won't miss. JPEG does this by transforming blocks of pixels into frequency coefficients, quantizing the high frequencies away, and entropy-coding what remains. The result is a file a fraction of the size that still looks fine — until you push it too far and blocky artifacts appear.

   The idea

   Lossy compression is a psychological trick as much as a mathematical one: it exploits the limits of human perception to decide what is safe to destroy. The DCT concentrates an image's energy into a few low-frequency coefficients, and quantization simply drops the ones your eye weighs least. The surprise is how aggressive you can be — most of the data in a typical image is perceptually redundant. The art is knowing where the cliff is, past which artifacts like blocking and ringing become visible.

   Try this: Save a photo as JPEG at maximum quality, then at quality 10, and compare file sizes and artifacts side by side.

4. 4Ch. 6 (Audio concepts), Ch. 13 (MPEG audio)

   Sound as a stream of samples

   The book shifts from pixels to pressure waves. Sound is continuous variation in air pressure; digitizing it means sampling that wave thousands of times per second and storing each sample as a number. The sampling rate and bit depth set the quality ceiling — CD audio uses 44,100 samples per second at 16 bits. From there the book explores how audio is stored, filtered, and compressed, including perceptual coding that hides quantization noise beneath louder sounds. What your ear hears as a smooth violin note is, inside the machine, a long list of integers arriving in strict order.

   The idea

   Audio digitization is simpler than image digitization in one dimension, but harsher in another: the ear is far more sensitive to errors than the eye. That is why audio compression leans so heavily on psychoacoustics — masking effects let codecs discard data that would be glaring in an image. Once you grasp that a sound file is just samples plus a header, formats stop being magic and become packaging choices.

   Try this: Record a voice memo, then listen to it at half the sample rate — hear how high frequencies vanish first.

5. 5Ch. 9-10 (Video concepts, MPEG video)

   Video: images with a clock attached

   Video, the book explains, is just a sequence of still images shown fast enough to fool the eye into seeing motion — but storing every frame separately would be absurd. So video compression adds a temporal dimension: instead of coding each frame fully, the codec predicts the next frame from the previous one and codes only the differences. MPEG standards formalize this with I-frames (complete pictures), P-frames (differences from earlier frames), and B-frames (bidirectional predictions). The chapter walks through motion estimation, the search for how blocks moved between frames, which makes prediction accurate enough to work.

   The idea

   Video compression succeeds because most frames barely change: the camera pans, a few pixels move, and the rest of the scene stays put. Motion compensation turns that redundancy into savings — you transmit where a block went, not what it looks like. The I/P/B frame structure is a scheduling problem: B-frames need future frames to decode, so the order of storage differs from the order of display. Once you see video as delta-coding plus motion search, streaming bitrates stop seeming arbitrary.

   Try this: Pause a video during a fast action scene and step frame by frame — watch how much (or little) changes between consecutive frames.

6. 6Ch. 11-12 (Networks, streaming media)

   Streaming: media that plays before it arrives

   The book's final stretch tackles delivery: how do you watch a video that hasn't finished downloading? Streaming media protocols send data over networks in real time, with buffering to absorb network jitter — the player holds a few seconds of video in reserve so brief delays go unnoticed. The chapters cover network protocols, RTP for real-time transport, and the challenges of variable bandwidth. The stakes are practical: a streaming system must balance quality, latency, and robustness, deciding whether to drop frames, reduce resolution, or pause and buffer when the network sags.

   The idea

   Streaming is an engineering compromise between three enemies: delay, jitter, and bandwidth. Buffering is the central weapon — trading a few seconds of startup wait for smooth playback — and adaptive strategies adjust quality to match the pipe. What surprises newcomers is how much of streaming design is about graceful failure: the system assumes the network will misbehave and plans for it. Media delivery is less about clever coding and more about managing expectations on an unreliable channel.

   Try this: Watch a video on a throttled connection and notice the player drop to a lower resolution before it stutters.

Students in multimedia, computer science, or digital media courses, and developers who work with images, audio, or video and want to understand what their libraries actually do. Also valuable for anyone curious about how familiar formats like JPEG and MPEG earn their keep.

FAQ