Debouncing typeahead requests and handling out-of-order responses: an interactive timeline

A search box that suggests results as you type looks simple, but two client-side details decide whether it feels right and what it costs the server. Requests sent per keystroke can come back out of order, so the box must ignore stale responses. And most of those requests are for text the person has already typed past, so debouncing, waiting for a short pause before asking, saves load at the cost of a little delay.

The playground has 500 people type “system design” at realistic speeds against a server that usually answers in under 120 ms but sometimes takes up to half a second. The timeline shows one of them.

What the timeline shows

• Showing whatever arrives lets a slow response for older text overwrite newer results. It happens about 1.5 times per query, and 17% of people end up looking at suggestions for text they have already typed past until they type again.
• Dropping out-of-date responses, by tagging each request and rendering only the newest, removes the flicker at no cost in speed: final results about 90 ms after the last keystroke for half of the people. Cancelling the superseded request saves the client bandwidth too.
• Debouncing by 150 ms skips the requests sent mid-word: 8 per query instead of 13, 38% less load on the suggestion service. The final suggestions arrive about 150 ms later, 240 ms after the last keystroke for half of the people.

In the browser

let timer, latest = 0;
input.addEventListener('input', () => {
  clearTimeout(timer);
  timer = setTimeout(async () => {          // debounce: wait for a 150 ms pause
    const id = ++latest;
    controller?.abort();                    // cancel the request still in flight
    controller = new AbortController();
    const res = await fetch('/suggest?q=' + encodeURIComponent(input.value), { signal: controller.signal });
    const items = await res.json();
    if (id === latest) render(items);       // never show results for older text
  }, 150);
});

Servers help from their side: suggestion responses are small and cacheable by prefix, popular prefixes can be precomputed, and a short cache in the client means deleting a character shows the earlier results instantly without asking again.

In a system design interview

When you design typeahead, cover the client as well as the trie: “The client debounces keystrokes by about 150 ms, cancels requests that have been superseded and only renders the response for the current text. Responses are cached per prefix in the browser and at the CDN. That cuts the request rate the suggestion service has to handle by roughly a third and keeps results from flickering.” Expect the follow-ups: how many requests per second the service sees at peak, how to personalise suggestions without losing cacheability, and how the latency budget splits between network and lookup.

Read the guide to tries and autocomplete or mobile system design, see trie typeahead and HTTP caching, or watch a strong candidate design typeahead.

Questions people ask