AYAZ AHMED / PORTFOLIO
← ALL PROJECTSSELECTED WORK / 04

DRISHTI / COMPUTER VISION

FACETRACE

An ONNX-based camera pipeline combining detection/NMS, greedy IoU tracking, normalised face embeddings, temporal voting and WebSocket frame delivery.

Recorded Drishti walkthrough · interface tourSCROLL TO EXPAND ↓
01 / THE PROJECT

Multi-model inference with bounded frame age.

FaceTrace integrates model inference with the scheduling and state management needed for a live camera application. Capture threads publish the latest frame independently of per-camera processing. Detection boxes, track IDs, pose-derived action rules and identity votes feed event snapshots and alerts. Binary JPEG frames and JSON metadata connect the backend to Drishti’s monitoring interface without requiring every captured frame to pass through inference.

640²

Detector input

Configured spatial resolution before coordinate remapping.

10 frames

Re-identification interval

Default cadence; camera and inference speed determine elapsed time.

5 frames

Action smoothing window

Default temporal majority window before persistent events.

Vectorised identity matching

e^=e∥e∥2,s=e^ E^⊤,i∗=arg⁡max⁡isi\hat e=\frac{e}{\lVert e\rVert_2},\qquad s=\hat e\,\hat E^\top,\qquad i^*=\arg\max_i s_i

Each row of Ê is a normalised enrolled embedding. recognition_utils computes the similarity vector in one matrix operation, selects its maximum and accepts the identity only when the configured threshold is met. Normalisation makes the dot product equivalent to cosine similarity.

Track association

IoU⁡(A,B)=∣A∩B∣∣A∪B∣\operatorname{IoU}(A,B)=\frac{|A\cap B|}{|A\cup B|}

SimpleTracker greedily selects the best unmatched box above an IoU threshold of 0.3. Geometric association maintains a track between processed frames; identity recognition is a separate embedding-based step.

02 / THROUGH THE SYSTEM

INPUT → OUTPUT
  1. 01

    Capture

    Camera threads update the latest available frame independently of inference.

  2. 02

    Detect

    ONNX inference and non-maximum suppression produce object and face detections.

  3. 03

    Track

    IoU-based association maintains track identities between processed frames.

  4. 04

    Recognise

    Periodic face-embedding comparisons and repeated votes stabilise identity matches.

  5. 05

    Deliver

    WebSocket JPEG frames, metadata, MJPEG streams and alert records connect the backend to the interface.

01 / ENGINEERING

Capture-inference decoupling

A latest-frame handoff separates camera acquisition from inference cadence. The processing loop prioritizes the newest available image to keep latency bounded and prevent frame lag. Per-camera processors manage tracks, identity state and recent events. Stream delivery projects recent detections across skipped intervals, maintaining fluid playback while optimizing model inference cycles.

02 / ENGINEERING

ONNX tensors, NMS and geometric association

The detector path resizes input to 640 × 640, runs ONNX inference, remaps detections to frame coordinates and suppresses overlapping boxes with NMS. A custom SimpleTracker associates boxes through spatial intersection-over-union matching. Pose and proximity rules contribute action labels, which are smoothed over a configured temporal window before event generation.

03 / ENGINEERING

Normalised embeddings and identity persistence

Face embeddings are L2-normalised. A cached matrix of enrolled embeddings allows all candidate similarities to be computed in one vector-matrix product; the highest score is compared with the configured match threshold. The pipeline can use the optional InsightFace SCRFD/glintr100 path. Re-identification runs periodically and identity persistence/voting reduces frame-to-frame flicker.

04 / ENGINEERING

Event delivery and alert throttling

WebSockets carry binary JPEG frames and associated JSON state, with MJPEG available as a secondary stream transport. MongoDB stores identity records and alert history. A bounded event timeline and configurable alert throttle consolidate repeated per-frame detections into structured application-level security events.

04 / BUILT WITH

TOOLS & THEIR PLACE
  • 01Python 3.11Pipeline orchestration & services
  • 02ONNX RuntimePortable model inference
  • 03SCRFD / glintr100Optional InsightFace path
  • 04NumPy / BLASVectorised cosine matching
  • 05OpenCVCapture & tensor preparation
  • 06IoU / NMSAssociation & suppression
  • 07WebSocket / MJPEGJPEG stream transports
  • 08MongoDBIdentity & alert persistence
AYAZ AHMED / PORTFOLIOBACK TO COLLECTION ↑