LESSWRONG
LW

Rapid Coordination Field Manual
Dynamical systemsSlackSystems ThinkingPracticalWorld Modeling
Frontpage

43

Queuing theory: Benefits of operating at 60% capacity

by ampdot
1st Dec 2023
1 min read
4

43

This is a linkpost for https://less.works/less/principles/queueing_theory
Dynamical systemsSlackSystems ThinkingPracticalWorld Modeling
Frontpage

43

Previous:
Slack
74 comments292 karma
Next:
One Day Sooner
8 comments125 karma
Log in to save where you left off
Queuing theory: Benefits of operating at 60% capacity
9Sune
11Dagon
6Viliam
2MondSemmel
New Comment
4 comments, sorted by
top scoring
Click to highlight new comments since: Today at 7:58 PM
[-]Sune2y92

You don’t really need the producers to be “idle”, you just have to ensure that if something important shows up, they are ready to work on that. Instead of having idle producers, you can just have them work on lower priority tasks. Has this also been modelled in queueing theory?

Reply
[-]Dagon2y113

Definitely, along with switching costs (if you drop a low-priority to work on a high-priority item, there's some delay and some waste involved).  In many systems, the switching delay/cost is high enough that it's best to just leave some nodes idle.  In others, the low-priority things can be arranged such that dropping/setting-aside is pretty painless.

Reply
[-]Viliam2y62

You need to make sure that when something important shows up, (1) the system will clearly recognize that this happened, and (2) the producers will actually be able to abandon the lower-priority task quickly.

I have seen companies trying to implement this, but what often actually happens is that the manager responsible for the lower-priority task just keeps assigning work to the employees anyway. The underlying cause is that the manager's incentives are misaligned with the company goals -- his bonus depends on getting the lower-priority task done. (How would you set up his incentives?)

Reply
[-]MondSemmel2y20

Related reading: IIRC the topic of queuing theory features prominently in the great book Algorithms to Live By.

Reply
Moderation Log
Curated and popular this week
4Comments

Related to Slack. Related to Lean Manufacturing, aka JIT Manufacturing.

TL;DR A successful task-based system should sometimes be idle, like 40% of worker ants.

Doing tasks quickly is essential for producing value in many systems. For software teams, delivering a feature gives valuable insight into user needs, which can improve future feature quality. For supply chains, faster delivery releases capital for reinvestment. However, the relationship between capacity utilized and service time is exponential, as shown by the diagram below.

queueing-6.png
Stochastic arrivals mean it's better to sometimes have idle capacity, so that the system is quickly ready for when tasks arrive in droves.

A heuristic we can derive from queuing theory is that the optimal balance between efficiency and capacity typically occurs when the system is around 30-40% idle. For a single producer system, being X% idle is that producer being idle X% of the time. For a multi-producer system, being X% idle is X% of those producers being idle on average. This heuristic applies best to systems involving lots of discrete, oddly-shaped tasks.

The linked post explains this theory in more detail, and gives examples of where queues appear in the real world. See also the Wikipedia article:

Queueing theory is the mathematical study of waiting lines, or queues.[1] A queueing model is constructed so that queue lengths and waiting time can be predicted.[1] Queueing theory is generally considered a branch of operations research because the results are often used when making business decisions about the resources needed to provide a service.

Mentioned in
29Throughput vs. Latency