01 · InnovaTech
Security video analytics for government sites
Rebuilt the video pipeline and the AI verification stage: no more dropped video, a fraction of the CPU, and false alarms cut by more than half.
- Cameras
- Ingest (a stage I owned)
- Detection
- Verification (a stage I owned)
- Alert
How I did it
Rebuilt the video pipeline
- Replaced a per-camera setup (a separate Redis container and buffer container for every camera) with one shared Redis and a single buffer service. Far fewer containers to run, monitor and restart.
- Each camera was opened twice, once by the recorder and once by the buffer. I replaced both with a single GStreamer pipeline per camera that decodes and records from one connection. Packet loss went from up to 4.7% to zero.
- Moved video decoding and encoding onto the GPU: ingest dropped from about one CPU core per camera to about 1.4 cores for a whole site.
Fixed the AI verification stage
- A vision-language model checks every alert before it reaches an operator. In production each check took about 11 seconds, alerts queued up in busy periods, and the model server kept crashing.
- Stabilised it first (GPU memory headroom, concurrency, context length, a vLLM scheduler bug), then treated the backlog as a throughput problem. I rejected dropping old alerts; a late alert beats a missed one.
- Built a labelled evaluation set from real footage and benchmarked open models. MiniCPM-V 4.5 replaced Qwen3-VL-8B: slightly more real incidents caught, false alarms from 56% to 21%, about 8× faster. Shipped within three days after re-testing on whole clips, as production sends them.
- While building a motion pre-filter for fight detection, found a third of the "fight" labels were wrong and relabelled them.
Stack. Python, GStreamer, vLLM, Redis, Docker, GPU pipelines
