Case Study

AI Vibe Audit

Law Enforcement

AI Vibe Audit Case Studies

How assessment-led engineering turns vibe-coded prototypes into safer, scalable production systems.

This first case study is based on a client-facing PDF covering a law-enforcement-oriented AI video analytics platform built through heavy AI-assisted development.

Read Case Study Discuss Your Project

Overview

Stabilizing and hardening an AI-powered CCTV analytics platform built with heavy vibe coding.

The initial platform moved quickly from concept to functionality, but the audit exposed production-level weaknesses in runtime efficiency, security posture, infrastructure design, and cost control.

Sector: Public Safety

Engagement: Audit

Stack: Flask / Next.js / AWS

Case Summary

A law-enforcement-oriented AI video analytics platform was delivered rapidly using a vibe-coded workflow. That speed helped early feature delivery, but the resulting system carried meaningful weaknesses across performance, architecture, security, and operating cost.

The most material issues included repeated object re-instantiation inside loops, redundant LLM and inference lookups, disk-heavy processing patterns, and an AWS footprint that did not align well with actual workload behavior.

The engagement demonstrates how an assessment-first approach can convert a promising prototype into a safer, more scalable, and more commercially credible platform.

Why the client needed the vibe code audit

Operational Context

The platform was designed to analyze CCTV and other video inputs for public safety and investigative workflows, combining computer vision with AI-assisted interpretation so operators could review persons, vehicles, and scene activity more efficiently.

The implementation used Python Flask for backend services, Next.js for the front end, AWS for infrastructure, MongoDB for operational data, SQS for workload decoupling, YOLOv9 for detection, and Qwen 2.5 for downstream inference and interpretation.

The audit found that the product reached functional usefulness quickly, but prototype speed had outpaced production engineering discipline.

Why The Client Needed An Audit

Application behavior under load was inconsistent, with avoidable memory pressure and inefficient runtime patterns.
Security issues existed across application and infrastructure layers and required remediation before broader deployment.
Infrastructure sizing and service usage were not aligned to real workload characteristics, increasing AWS operating costs.
Code organization and data structures reduced maintainability and raised the cost of future enhancements.
Redundant LLM and inference lookups created avoidable latency, storage churn, and unnecessary usage costs.

Engagement Profile

Engagement: Architecture, code, security, and cost audit.

Primary concerns: Performance bottlenecks, security issues, non-optimal infrastructure, poor data structures, and redundant inference calls.

Document type: Client-facing use case for US buyers and partners.

Findings

The audit surfaced issues that directly affected reliability, cost, and deployment readiness.

Audit Findings Matrix

Area	What The Audit Found	Operational / Business Risk	Recommended Direction
Runtime efficiency	Objects and processing components were being re-instantiated inside loops instead of being reused appropriately.	Higher memory pressure, unstable throughput, and starvation risk during sustained processing.	Refactor lifecycle management so heavy resources are initialized once and reused safely.
Inference pipeline	Multiple LLM and inference lookups were executed within the same workflow where consolidation or caching should have been applied.	Higher latency and materially higher model and compute costs.	Reduce duplicate calls, introduce request discipline, and cache where correctness allows.
Storage behavior	Disk-heavy intermediate processing and repeated artifact generation increased I/O pressure and storage usage.	Slower end-to-end execution and inflated storage-related cloud spend.	Tighten artifact retention, reduce unnecessary writes, and move to a cleaner storage lifecycle.
Data structures	Core code paths used weak or poorly chosen structures, increasing complexity and runtime overhead.	Lower maintainability and harder troubleshooting during incident response.	Redesign critical data flows around simpler, predictable, production-grade structures.
Infrastructure	AWS services and workload placement were not aligned to traffic, inference, and retention patterns.	Unnecessarily high operating cost and less predictable scaling behavior.	Right-size services, separate hot and cold paths, and implement cost-aware workload design.
Security posture	The audit identified code and platform security issues requiring hardening before operational expansion.	Elevated exposure for a system intended for sensitive law-enforcement workflows.	Apply secure controls, least-privilege access, secrets discipline, hardening, and validation testing.

Why These Findings Matter

Operational reliability. Repeated initialization patterns and redundant inference work can look acceptable in demos but break down under real traffic, burst workloads, or longer-running jobs.

Budget control. When AI calls, storage writes, and AWS services are not tightly governed, a prototype can become expensive to operate long before it becomes truly production-ready.

Security readiness. For sensitive environments, code issues combined with non-optimal infrastructure controls create avoidable risk and slow procurement or expansion discussions.

Maintainability. A fast-moving vibe-coded codebase can leave teams with unclear ownership, inconsistent patterns, and rising effort for every new change or incident.

Roadmap

A four-phase remediation path from fragile prototype to production-ready platform.

Phase 1: Stabilize Runtime Behavior

Move heavyweight object creation and model setup out of hot loops.
Profile memory, CPU, and I/O across representative workloads.
Eliminate redundant inference paths and enforce clearer request boundaries.

Phase 2: Harden The Application And Platform

Address audit findings in code and infrastructure before expanding operational scope.
Apply least-privilege IAM, stronger secrets management, dependency review, and environment hardening.
Introduce structured code review, security testing, and deployment gates.

Phase 3: Re-Architect For Cost-Efficient Scale

Separate real-time, near-real-time, and batch workloads where appropriate.
Review SQS queueing, worker sizing, storage retention, and model-serving topology.
Align compute placement and storage patterns with actual processing intensity and retention requirements.

Phase 4: Establish Governance For AI-Assisted Development

Treat AI-generated code as an accelerant, not as a substitute for architecture ownership.
Require engineering review for critical data paths, security-sensitive code, and scaling decisions.
Implement documented standards for testing, observability, dependency control, and release readiness.

Achievements

Achievements Delivered Through the Modernization

The modernization work converted key issues discovered in the vibe-coded implementation into practical improvements across runtime behavior, AI processing, cloud operations, security, and maintainability.

Technical Achievements Delivered

Reduced unnecessary object re-initialization, helping lower memory pressure and produce more stable, predictable service behavior under sustained processing.
Consolidated redundant LLM and inference activity, creating cleaner inference orchestration and reducing avoidable processing overhead.
Improved processing efficiency by reducing unnecessary intermediate artifacts, storage writes, and disk-heavy workflow behavior.
Strengthened critical data flows and code structures, making the platform easier to troubleshoot, maintain, and extend.
Introduced a clearer foundation for workload separation, service right-sizing, observability, and cost-aware scaling.

Business and Operational Achievements

Reduced avoidable AWS, storage, compute, and model-usage expenditure by aligning resource consumption more closely with actual workload needs.
Improved security readiness through stronger controls around access, secrets, infrastructure hardening, and validation practices.
Increased confidence in the platform for sensitive public-safety and law-enforcement use cases.
Improved the platform’s credibility during stakeholder review, procurement discussions, and future scaling decisions.
Moved the solution away from a fragile prototype model toward a more disciplined, production-oriented platform with a clearer path for ongoing feature delivery.