2022 — 2023Software Engineer · Emblem Technologies
High-Traffic Commerce & Analytics
Async order processing and real-time dashboards over streamed events
- development time via shared component modules
- −30%
- analytics over Kafka-streamed events
- Real-time
- order processing under peak load
- Decoupled
The problem
Order processing sat on the request path, so traffic spikes turned into timeouts at exactly the moment orders mattered most. Meanwhile every product team was rebuilding the same Angular components slightly differently, and stakeholders were asking for insight the reporting layer could not deliver live.
The approach
Order work moved behind RabbitMQ so the checkout request returns as soon as the order is durable. A shared, typed Angular component library replaced the per-team reimplementations. Analytics dashboards were built on Kafka-streamed events, so what stakeholders saw reflected the system rather than last night's batch.
Architecture
- RabbitMQ queues absorb order processing and background jobs; the API acknowledges once the work is durably enqueued rather than once it is complete.
- NestJS services over MongoDB and PostgreSQL with TypeORM, each store chosen for its access pattern rather than by default.
- A shared Angular component library built on TypeScript, RxJS and NgRx, consumed across products.
- Angular and D3.js dashboards over REST APIs and Kafka-streamed events for live business insight.
- Redis for session caching and pub/sub behind the real-time surfaces.
The queue was the fix, not a bigger box
Under load the instinct is to scale the thing that is slow. But the checkout request did not need the order to be fully processed — it needed the order to be safely recorded. Once that distinction was explicit, the fix was a queue rather than more instances, and the platform stopped degrading in the way that cost money.
A component library only works if adoption is cheap
The 30% saving did not come from writing good components; it came from making them the path of least resistance. Typed props, sensible defaults and drop-in parity with what teams already had meant migrating was faster than not migrating. A library nobody adopts is a second implementation, not a shared one.
Dashboards over events, not over the production database
Reporting queries against the transactional database is how analytics work becomes an availability incident. Consuming Kafka events into a purpose-shaped read model kept the dashboards live without letting a stakeholder's date range take the store down.