Build an in-memory pub-sub with topics, many subscribers per topic and at-least-once delivery. How does a message get redelivered?
An in-memory pub-sub with at-least-once delivery stores each topic as an append-only log and gives every subscriber its own cursor and in-flight set. A message stays in flight until acknowledged and is redelivered after an ack timeout, so a slow subscriber only delays itself and every handler must tolerate duplicates.
Answer
Short answer
Give each subscriber its own position. An in-memory pub-sub with at-least-once delivery stores every topic as an append-only log, and each subscription keeps a cursor plus a set of in-flight offsets with ack deadlines. A message leaves the in-flight set only when the subscriber acknowledges it; if the deadline passes first, it is delivered again. That design isolates slow subscribers, keeps publish order for first deliveries, and means handlers must be idempotent because duplicates are part of the contract.
Was this answer helpful?
Likely follow-ups
- A handler takes longer than the ack timeout. What does the subscriber see, and how should it cope?
- Two subscribers on one topic want to share the work instead of each receiving every message. What changes in your classes?
- A subscriber has been stalled for an hour and memory is climbing. Which choice keeps at-least-once, and which breaks it?
- How would you guarantee order per customer id without forcing the whole topic through one message at a time?
Related questions
- Design an in-memory job scheduler that runs one-off, delayed and recurring jobs. Write the core classes.hardAlso on machine-coding and concurrency6 min
- Design a movie ticket booking system with shows, seats and a temporary hold that expires if the user does not pay. Write the classes.hardAlso on machine-coding and concurrency6 min
- Design a rate limiter that allows N requests per client per minute. Write the class.mediumAlso on concurrency and machine-coding4 min
- Design a food-ordering system: customers place orders with restaurants, and the system assigns a delivery partner. Walk me through the classes.mediumAlso on machine-coding5 min