Two pieces of work on POIZON’s Shopping Agent: building Gift Agent, a sub-agent for context-aware gift decisions, from 0 to 1; and improving the quality of the Shopping Agent’s web search through evaluation and iteration.
Defined the intent taxonomy, decision factors, dialogue strategies, and retrieval logic that let Gift Agent help people decide what to give in a given context.
Launched Gift Agent to 15K daily UV, lifting Shopping Agent’s conversations per user by 4.60% and products browsed per user by 74.42%.
Defined 7 eval metrics based on unique product value & output quality, and built 200-case golden sets to measure Gift Agent’s quality.
Diagnosed issues from online evaluation results, LLM execution traces, and 50+ pre-release user feedback, then refined prompting and multi-turn context management, improving Gift Agent performance by 12.4%.
Led quality improvement for the query decomposition and web search nodes in Shopping Agent’s LangGraph pipeline: defined 12 node-level metrics and real-case eval sets via SQL, diagnosed failure patterns from execution traces, and partnered with engineering to prioritize fixes.