Lenskart, 2026.
Built end to end.
Competitor creative intelligence and its eval layer
- Context
Lenskart is an eyewear company that runs large ad campaigns. Its leadership wanted a reliable read on what competitor brands were running.
- Users
Lenskart leadership, through an internal analyst.
- Problem
The earlier pipeline was measured making up facts in 27% of sampled rows. Leadership could not act on numbers it could not trust.
- Built
A pipeline that collects competitor posts and ads from Instagram, TikTok, Meta Ads and YouTube, tags each one on 24 structured dimensions with Gemini, and shows the results in a dashboard with a chat assistant. It runs on a schedule across a set of competitor brands. A deterministic guardrail checks every number the assistant writes against the source data.
- Decision
Tagging calls ran one at a time at first and hit the model provider's free-tier limit of 20 requests a day. I moved them to one combined batch job per cycle across all brands. The tradeoff is freshness: new ads get tagged at the next cycle, not instantly, so that the job stays inside the quota.
A judge metric measures how often an automated grader agrees with human labels. It shows how far to trust the grader. It does not show how often the agent is right.
- Measured
The LLM judge agreed with human labels on 54.0% of 50 held-out real user turns in an early version, and on 88.9% of 27 unseen turns in a later batch, after rounds of calibration. The two figures come from different batches.