Read on Twitter
I explained this a few times today, and I think it's important for every dev to know: timeouts are not necessarily what they seem.
A timeout happens when an operation doesn't complete fast enough. But, what does that mean? What are the details of that check? Well...
A timeout happens when an operation doesn't complete fast enough. But, what does that mean? What are the details of that check? Well...
You almost never check timeouts live, unless doing so is VERY critical and worth being a primary CPU consumer. That's very expensive. Usually, what happens is one of 2 things:
- A timeout event is queued, and if it goes off first: boom.
or:
- Timeouts are checked on an interval.
- A timeout event is queued, and if it goes off first: boom.
or:
- Timeouts are checked on an interval.
If you check timeouts "live", well...nothing's "live". It's some period of time, even if it's 1ms.
To queue a timeout with the event and you're checking both: that means another item to track and schedule, which means cost, that scales.
On an interval, you sacrifice fidelity.
To queue a timeout with the event and you're checking both: that means another item to track and schedule, which means cost, that scales.
On an interval, you sacrifice fidelity.
In the queue case, you're waiting on the thing that processes the queue. If the delay is because you're overloaded, that may be a while. But later it'll go "oh, yeah that's past due" **when it checks**. Not instantly, when it gets to it.
So under load, is it worth it? Rarely.
So under load, is it worth it? Rarely.
What's more common is the interval case - every so often, we check the things that should have been done by now. If they aren't: trigger a timeout (usually an exception). This could be on any interval. The tradeoff here is that...
...your fidelity is that of the interval (and potentially higher if that check is blocked by load).
For example, if we check every 1,000ms and your query has a timeout of 2,000ms, we'll likely throw a timeout somewhere between 2,000-3,000ms, importantly: **when we check**.
For example, if we check every 1,000ms and your query has a timeout of 2,000ms, we'll likely throw a timeout somewhere between 2,000-3,000ms, importantly: **when we check**.
I would just like devs to understand that timeouts are not "something took longer than I specified", it's "when we checked, this thing was overdue". They're similar concepts, but not quite the same thing and the delay before the latter triggers can be quite large, under load.
This of it as sending someone to the store. You give them 30 minutes and you have a stopwatch.
Do you watch it every moment? You can, but that's expensive. More likely, you check every so often and go "oh, that's past 30, they're late" the first time you checked after the limit.
Do you watch it every moment? You can, but that's expensive. More likely, you check every so often and go "oh, that's past 30, they're late" the first time you checked after the limit.
