Extended customer context

What is it? A TypeScript backend service that maintains an always-current, structured "memory" for every customer relationship on a property-management platform, extracting information from multiple isolated data sources. It ingests each party's communication and activity history across many channels, as well as their current state in other data sources, and distills it into a concise, queryable summary. It includes any initial promises, key interactions, user preferences, any issues, and milestones in the relationship between Belong and them, so that internal tools and agents always have context on demand.

Approximate scale: over 80K entities each carry a living summary, and the pipeline has processed 2M+ summarization jobs to date (thousands per day). Since each entity is summarized once and then incrementally re-summarized as new activity arrives, that job count reflects continuous upkeep over the service's lifetime — on the order of dozens of refreshes per entity — not 2M one-off runs.

What makes it interesting:

• Incremental, checkpointed summarization. Rather than re-reading an entity's entire history every time, it tracks a watermark of what's already been analyzed and folds only new activity into the previous summary. Long histories are processed in token-budget-aware batches, and each batch is checkpointed so a failed run resumes where it left off instead of starting over
• Structured, self-healing output. The model returns strictly-typed structured summaries validated against a schema, with layered self-correcting parsers that repair malformed output automatically.
• Cost optimization. Tiered model routing tied to the prompt architecture of the pipeline: a cheap model for the bulk work, a stronger one only for retries. The prompts are deliberately designed to maximize provider-side prefix caching, and the pipeline implements adaptive batching to stay within context limits.
• Event-driven and lazily consistent. Work is decoupled through a message queue; reads return instantly from cache and trigger a background refresh only when the data is actually stale, so the expensive LLM work never sits on a user's critical path.
• Production resilience. Bounded retries with failure classification, graceful shutdown draining, garbage collection, dead-letter queues for poison messages, and full metrics/observability.

Something I personally like: I designed and implemented the prompt architecture based on the SOLID prinples of object-oriented programming, which allowed me to reuse code across the different stages of the summarization pipeline, minimizing errors and simplifying maintenance.

Stack: TypeScript with Express.js for the backend, LangChain for LLM abstraction and parametrization, a custom orchestration harness, MongoDB for summary and execution persistence, RabbitMQ through AWS MQ, deployed as a container in AWS ECS, with distributed tracing/metrics throughout.