Skip to content
academia.sh
tr
Log in

Course Intermediate

Caching, Queues and Asynchronous Processing

By the end of this course

Start course

01

Caching

Preventing the same response from being reproduced: cache layers stretching from the client to the database, cache-aside and write strategies, time-based, event-based, and version-based invalidation, key design carrying namespace and tenant separation, preventing the stampede effect with single-flight, HTTP validators, and measuring the hit ratio.

  1. 01 Cache Layers Measuring a read path that produces the same response over and over, and setting up client, edge, server, and database caches as a chain: how many requests each layer stops, the split between shared and private caches, and lowering the rate of requests that reach the origin.
  2. 02 Cache Strategies Comparing cache-aside, write-through, and write-behind on the same workload: the number of reads and writes reaching the origin, the hit ratio, the durability trade-off of write-behind, and the race between a miss and a write in cache-aside.
  3. 03 Invalidation Measuring time-based, event-based, and version-based invalidation on the same workload: the trade-off a lifetime sets up between origin reads and stale reads, the delay of an event consumer fed by an outbox, and the orphaned-entry cost of putting the version in the key.
  4. 04 Cache Key Design Demonstrating, by running them, two flaws of a hand-concatenated key — collision and tenant leak — and writing a function that generates a key from namespace, schema version, generation, tenant, and canonical criteria; measuring the effect of key cardinality on the hit ratio and on correctness.
  5. 05 Hot Key and Stampede Effect Measuring how a hundred concurrent requests pile onto the origin when a hot key's entry drops, and collapsing that to a single production with a single-flight lock; stale-while-revalidate zeroing out the wait, jittering the lifetime to prevent simultaneous expiry, and spreading a hot key across shards.
  6. 06 HTTP Caching The response carrying its own cache rules: ETag and Last-Modified validators producing a 304 on conditional requests, the body transfer dropping to zero, the validator changing when the resource changes, and choosing the public, private, max-age, and no-store directives, along with stale-while-revalidate, by resource type.
  7. 07 Cache Metrics Calculating the hit ratio and its effect on average response time: the curve the hit ratio traces as capacity grows, what the eviction count means, the per-namespace breakdown a single total ratio hides, the difference between byte hit ratio and request hit ratio, and the measured gain from removing low-repeat data from the cache.

02

Messaging

Removing work from the request and carrying it between two ends as a message: the distinction between queue, publish–subscribe, and stream models, measuring delivery semantics, the trade-off between ordering and partitioning, and dead-letter queue and retry strategies.

  1. 01 Rationale for Asynchronous Processing Shortening the request path by removing work from it: measuring step costs and success probability, separating work that must stay in the request from work that can be deferred, the changed meaning of the response, and the durability difference between handing off work in memory and writing it to the outbox.
  2. 02 Message Queues Building the queue in its smallest form: write, receive, acknowledge and visibility timeout, measuring the redelivery of an unacknowledged message, competing consumers giving a single message to a single side, and how queue depth behaves according to production and consumption rate.
  3. 03 Publish–Subscribe Delivering a single message to more than one interested party: the distinction between topic and subscription, counting how many deliveries a single publish turns into depending on subscriber count, isolating a slow subscriber's backlog from the others, measuring copies dropped by a filter, and a subscription created after publication being unable to see the past.
  4. 04 Streams versus Queues Separating consuming from deleting: an append-only log, a per-consumer offset, comparing a single copy against per-subscription copies in bytes, measuring why replaying is impossible on a queue and cheap on a stream, and what retention means for a consumer that has fallen behind.
  5. 05 Delivery Semantics The three modes determined by the order of acknowledgement and effect: measuring the loss and repeated-effect counts across at-most-once, at-least-once, and deduplicated runs of the same workload, a deduplication guard bringing the effect count rather than the delivery count down to one, and no ordering preventing both when the effect sits outside the database.
  6. 06 Ordering and Partitioning The trade-off between ordering guarantees and concurrency: measuring, as an inversion count, competing consumers finishing the same key's events out of order; partitioning by key preserving in-key order while giving up global order; the effect of partition count on concurrency; and a hot key unbalancing partition load.
  7. 07 Dead-Letter Queues Isolating messages that will never succeed: unbounded retry letting the wasted-attempt count grow without limit, an attempt counter and threshold draining the queue, the fields a dead-letter record must carry, redriving messages once their cause has been fixed, and the trade-off between the threshold value and wrongly isolating a transient failure.
  8. 08 Poison Messages and Retry Isolating the message that brings down the consumer, and calculating wait durations: the necessity of incrementing the attempt counter at delivery time, comparing immediate, fixed, and exponential schedules by total wait and retry budget, jitter lowering the peak of the retry wave, and reconciling backoff with the visibility timeout.

03

Background Jobs

The worker process as a unit of scaling and supervision of a crashed worker, the recurring task definition and the lock that prevents overlap, progress reporting and cancellation in a long-running job, backpressure where production rate exceeds consumption rate, and load shedding that enables controlled rejection under overload.

  1. 01 Worker Processes Treating the job as a unit of scaling separate from the message: setting up a worker pool, measuring how worker count affects throughput, replacing a crashed worker under supervision, and graceful shutdown.
  2. 02 Scheduled Tasks The definition and hazards of calendar-triggered jobs: computing the recurrence rule, the end-of-month and time-zone traps, the fixed-rate versus fixed-delay distinction, and a measured solution that prevents two runs from overlapping with a lease lock.
  3. 03 Long-Running Jobs Making a background job that runs for minutes manageable: breaking the job into chunks, writing progress to a durable status record, building cooperative cancellation with an abort signal, and resuming from where cancellation left off.
  4. 04 Backpressure The consequences of production rate exceeding consumption rate: unbounded growth of wait time in an unbounded queue, a numerical simulation comparing the bounded queue's three policies, and backpressure measured through streams and propagated to the producer.
  5. 05 Load Shedding Controlled rejection under overload: admission control based on a concurrency threshold, measuring the latency gap between accepted and rejected requests, the HTTP shape of a rejection, and preventing a retry storm.

04

Real-Time Data

The request cost of short and long polling, server-sent events and reconnection, the WebSocket server built with its own handshake and connection lifecycle, channel distribution in a multi-instance deployment, and responses produced piece by piece.

  1. 01 Polling Approaches The simplest way for a client to learn about a change on the server: the request and empty-response cost of short polling, the connection-holding trade-off of long polling, measuring both against the same event, and closing the event gap with a version number.
  2. 02 Server-Sent Events Events written one after another onto a single open response body: the line format of the event stream, the header conventions of a streaming response, reconnection by event ID, and the function of the heartbeat line against intermediaries.
  3. 03 WebSocket Server Building a two-way connection from scratch: the upgrade handshake over HTTP and generating the accept key, writing and parsing the frame format, the ping–pong exchange and the closing handshake, and how connection state affects horizontal scaling.
  4. 04 Broadcast and Channel Management Getting a message to reach every subscriber in a multi-instance deployment: the channel concept, measuring how local delivery falls short, building an inter-instance broadcast channel, the cost of fan-out, and subscription accounting.
  5. 05 Streaming Responses Producing the response itself in pieces: the framing of chunked transfer encoding, measuring the time to first byte, the limits of reporting an error after headers have been sent, and backpressure in a streaming response.

Start typing to search.

↑↓ ↵ Esc navigate · open · close