Skip to content
all projects

01 · SECURITY / NETWORKING / DISTRIBUTED SYSTEMS

Enterprise AI Security Gateway

A 19-service appliance that intercepts, inspects and enforces policy on enterprise AI traffic — built at Defenxee.

GoTPROXY / nftablesICAPMITM TLSONNX RuntimeKubernetesPrometheus
129ms
redaction p95 (from 705ms)
62ms
redaction p50 (from 264ms)
21s → 7s
80-request batch @ c=8
1,307
Go tests

01 Context

Employer-owned enterprise AI-security appliance at Defenxee. Customers route traffic through it to detect and control sensitive data flowing to and from AI services. Details here are limited to what is on my resume — no customer names, endpoints or proprietary internals.

02 Engineering challenge

Inspect traffic deeply (text, images, sensitive records) without becoming the slowest hop in every request, and fit into very different enterprise network topologies without maintaining separate inspection implementations.

03 My responsibilities

  • Architecture, delivery and reliability for 19 services
  • Managing a 7-person engineering team
  • Proxy modes: forward, transparent, reverse, chaining, egress
  • Inspection performance and model correctness
  • Verification strategy and test suite

04 Architecture overview

Traffic enters through one of several proxy modes (TPROXY/nftables routing, ICAP, HTTP/3/QUIC, MITM TLS with certificate issuance), is passed to 10 unified detection engines (OCR, Exact Data Match, 104 custom PII recognizers), and a policy decision produces an allow, redact, or block verdict — failing closed. Identity integration, offline licensing, audit forwarding, clustering and diagnostics support enterprise deployment.

05 Key technical decisions

  • →Instrument first: per-detector and gateway metrics before touching performance.
  • →Fix ONNX thread oversubscription and validate INT8 / per-channel MatMul quantization with zero label flips on the evaluated corpus.
  • →Build gates that reject collapsed model outputs or label flips — end-to-end checks alone missed one.
  • →Score-separability analysis over threshold tweaking for enforcement gates.
  • →A 50ms regex budget so a single detector can't stall the pipeline.

06 Measurable outcomes

  • 129ms redaction p95 (from 705ms)
  • 62ms redaction p50 (from 264ms)
  • 21s → 7s 80-request batch @ c=8
  • 1,307 Go tests

07 What I learned

Most latency wins came from measuring precisely where time went, not from clever rewrites. And a model that silently stops finding things is worse than one that crashes loudly — so correctness needs its own gates.

next project

Production Reliability & Database Architecture