119 TB
frame-by-frame
8 fps
sustained, peaks to 12
<1 hour
for a typical 50-100 GB upload
AI video labeling - 119 TB of footage, frame-by-frame
A global events company had a decade of footage - main-stage cameras, on-site interviews, drone passes, b-roll - that nobody could search through. We fine-tuned a vision-language model on a Mac Studio M2 Ultra 512 GB; ran the historical pass on a Mac-Mini-M4-Pro processing rig connected over 10 GbE to a Synology NAS; and put the GDPR posture on the right side of the line for crowd footage where the vast majority of attendees never explicitly consented to facial processing. The result: the archive answers prompts like 'a scene where a specific speaker is on stage and not wearing his black leather jacket' in seconds.
119 TB
frame-by-frame
8 fps
sustained, peaks to 12
<1 hour
for a typical 50-100 GB upload
Last updated:
A decade of footage, effectively un-searchable
The agency runs large industry events worldwide. Across a decade they had accumulated main-stage cameras (multiple angles, hours per event), on-site interview footage (tens of speakers per event), drone passes, B-roll, breakout sessions, attendee crowd shots, and marketing edit-cuts. Around 119 TB of video, sitting on cold storage, effectively un-searchable. Finding a specific moment ("when did so-and-so first say X on the panel?") meant a researcher hours-deep in scrubbing timelines. Re-using footage for marketing meant guessing. Two compounding problems:
- Search by content, not metadata. Files were named by event and date; that was it. A query like "keynote moment where Person A is alone on stage and not wearing a leather jacket" required a human to know which event, which keynote, which camera, and roughly when.
- GDPR on historical footage. The vast majority of attendees in the older crowd footage never gave explicit consent to facial processing. Building a labelled, searchable archive that included those people was a legal non-starter unless the pipeline carefully separated the consent boundary.
Apple Silicon at 8 fps, frame-by-frame
Hardware
Fine-tune training rig
A Mac Studio with M2 Ultra and 512 GB unified memory. The 512 GB unified-memory architecture was the right call for training a vision-language fine-tune at this domain scale - full model + working set resident, no PCIe / disk-spill thrashing.
Production processing rig - Synology DS2422+ NAS
AMD Ryzen V1500B (4 cores @ 2.2 GHz), 12 GB RAM, 10 GbE (Synology E10G18-T1). Role: storage of the 119 TB archive, frontend hosting, and the database holding per-frame metadata.
Production processing rig - Mac Mini M4 Pro (2024)
12 cores (8 performance + 4 efficiency), 24 GB RAM, 16-core Neural Engine, 10 GbE. Role: dedicated video-processing engine for both the historical pass and the real-time ingestion.
Network
10 Gigabit Ethernet between the NAS and the Mac Mini. Theoretical 1.25 GB/s; sustained ~800-1000 MB/s with overhead. The network was never the bottleneck.
Models
Domain-fine-tuned vision-language model
Trained on a labelled corpus drawn from the agency's flagship events - speakers, stage layouts, event-specific iconography, recurring sponsors and venues. The fine-tune lifted recognition accuracy on recurring people and grounding on agency-specific scenery (the same conference branding, year over year) far above what a generic off-the-shelf VLM produced.
Face-recognition stack
Detection model + face-embedding model running per frame against a 250-face registry of recurring speakers and crew. Detection at 40-60 fps; embedding generation at 30-40 fps; comparison against the 250-face registry adds negligible overhead.
Combined pipeline throughput
~8-12 frames per second sustained, including database writes and error handling. The Neural Engine on the M4 Pro handles the inference; the CPU + 10 GbE handle the I/O.
Each frame at 1 fps gets labelled with
Each frame at the 1-fps sampling rate gets labelled with:
The hard architectural decision
Data-in-transit pattern
The agency's lawyers signed off on a data-in-transit pattern: full per-frame analysis runs in memory; only metadata that does not re-identify people without consent is persisted.
Crowd-footage search boundaries
Crowd footage is searchable by clothing, posture, density, scene type - never by face - unless the person is on the agency's consent register.
Consent register as first-class store
The consent register is a first-class data store, read before any PII-bearing metadata is written.
Six weeks of fine-tune training, then 28 days of historical processing
- Shipped01Weeks 1-6 · Fine-tune training
Built the labelled training corpus (a small labelling assistant proposed labels; a human approved). Trained the vision-language fine-tune on the Mac Studio M2 Ultra. Model recognised the agency's recurring speakers, their typical stage outfits, the agency's branding boards, and a small set of recurring venues without confusion.
- Shipped02~4 weeks · Historical pass
With the fine-tune in hand, the production rig (NAS + Mac Mini M4 Pro) processed the 119 TB archive frame-by-frame at 1 fps. The 10 GbE link kept storage I/O ahead of the Neural Engine; the rig ran 24/7 for the duration.
- Shipped03Real-time channel
New uploads (typical event delivery 50-100 GB per session) flow through the same pipeline automatically. Network transfer at 800-1000 MB/s puts a 100 GB upload on the NAS in under two minutes; per-frame processing at 8 fps puts the same 100 GB through inference in ~30-40 minutes. End-to-end upload-to-indexed: under an hour for a typical event session, single-digit minutes for short interview cuts.
- Shipped04Search interface
Plain-English query box backed by semantic search over the per-frame metadata. The query "drone shot over the venue at sunset on the second day" and the query "Person A speaking at a panel without his usual leather jacket" both work; the second is the one the marketing team uses most often.
Friction we hit
- The fine-tune dataset was the careful work. Labelling hundreds of stills well takes a person; labelling thousands needs a system. The labelling-assistant pattern accelerated that work roughly 5× and was the single highest-leverage thing we built in Phase 1.
- GDPR boundary as architecture, not as a filter. Treating compliance as "filter the output" was a non-starter for the lawyers. Treating it as "branch the pipeline at the consent boundary so PII never leaves memory unless there's a consent record" was the architecture they signed off on.
- NAS RAM was the constraint, not the compute. The DS2422+ has 12 GB RAM; the database working set tuned to fit. Caching strategy, query planning, and per-event index sharding mattered more than they would have on a beefier server.
The math behind 119 TB in 28 days
A media archive you cannot search?
The vision-language pipeline pattern - domain-fine-tuned model, GDPR-safe data flow, real-time and historical processing on commodity Apple-Silicon hardware - works on any large-volume media library. A 30-minute diagnostic walks through your archive shape, your compliance posture, and what a hybrid pipeline would cost.