A Full Connection Pool Is an Admission Decision You Did Not Make
TL;DR
A connection pool answers a narrow resource question: how many database connections may this process hold at once? It does not answer the application question that arrives under load: which work should wait, which work should be rejected, and which work has stopped being worth doing?
When every connection is busy, many pools make callers wait until a connection returns or a checkout timeout expires. That is useful protection for the database. It also means the pool has become a scheduler—usually one that cannot see a request's deadline, tenant, route cost, retry class, or business priority.
Put a deliberate admission boundary before database checkout. Give expensive work a bounded queue or an immediate rejection, reserve capacity for work that must remain available, propagate a real deadline, and observe waiting separately from query execution. The pool should enforce the connection budget, not accidentally define the product's overload policy.
The invisible queue is still a queue
An API has a pool of 20 PostgreSQL connections. A burst of report exports starts 20 slow queries. Then a customer tries to cancel one of those exports, an operator opens the incident dashboard, and the normal order API receives a request to create an order.
All three requests can be healthy application code. If each reaches pool.query() and the pool is exhausted, they can be placed in the same undifferentiated waiting room:
report export ─┐
report export ─┼─> connection-pool waiters ─> any returned connection
cancel export ─┤
order create ─┤
incident read ─┘
The database is protected from a forty-first connection, which matters. PostgreSQL has a finite max_connections setting, and increasing it also increases resources such as shared memory allocated by the server. PostgreSQL's connection settings make that limit explicit.
But the service has made several policy choices without writing them down:
- an export and a cancellation share the same waiting policy;
- a request that will miss its user-visible deadline may occupy a place in line anyway;
- a noisy tenant can occupy the waiters that an unrelated tenant needs; and
- a retry can create more waiting work even when the system has no capacity to make it useful.
That is admission control by accident. The pool's queue is real even if it does not appear in a product design.
Node-postgres is a concrete example, not an exception. Its Pool has a maximum client count, and its documentation warns that unreleased clients can leave future checkouts waiting forever. Its configuration separately exposes a connection timeout for making a new client. node-postgres pooling Pool API JDBC pools commonly have a similar shape: HikariCP's maximumPoolSize caps its total connections and getConnection() waits up to connectionTimeout once the pool is full. HikariCP configuration
Those are sensible library semantics. They are not an API priority policy.
A pool knows connections, not intent
The tempting response to saturation is to increase max. That sometimes helps: a pool can be too small for a database that still has useful capacity. It is not a general cure.
In a multi-instance service, the database sees the sum of every process's pool maximum, plus workers, migrations, operator sessions, and other services. The node-postgres sizing guide specifically cautions that a fixed per-process maximum must be considered across instances and leaves headroom for management access and scaling. node-postgres pool sizing
More importantly, a larger pool does not tell a system whether it should run another export while orders are delayed. It can turn application queueing into more concurrent database work, which may increase lock contention, I/O pressure, or tail latency. A connection is a transport resource; it is not a unit of business importance.
The missing information is usually available before checkout:
| Work class | What the application knows | A reasonable overload policy |
|---|---|---|
| Checkout or payment write | A user is waiting and a duplicate or delayed response has a cost. | Reserve a small, bounded share; reject early when it cannot meet a deadline. |
| Interactive read | The response may be useful after a short wait, but not after several seconds. | Small queue with a short deadline; degrade to a cached or partial response when honest. |
| Report export | It is expensive and can continue asynchronously. | Put it in a durable job queue with per-tenant limits; return an operation ID. |
| Cancellation or status lookup | It may be needed to stop or understand the work that caused load. | Keep a separate, small budget so it is not trapped behind the work it controls. |
There is no universal partition or set of numbers. The point is to make the choice where the request still has its meaning. Once it has become an anonymous pool waiter, the pool cannot recover that context.
Rejecting before checkout is often kinder
Suppose an HTTP endpoint has a 1.5-second response objective. It spends 900 ms waiting for a connection, acquires one, and starts a query that normally takes 800 ms. Even if the client disconnects at 1.5 seconds, the query may already be using database capacity for a response nobody will receive. Whether a particular driver cancels that query depends on its cancellation and disconnect behavior; a pool timeout alone does not make that decision.
The useful policy is to decide before asking for a connection whether this request still has a realistic budget. Here is illustrative TypeScript, deliberately separate from any pool library:
type Permit = { release(): void };
class CapacityGate {
constructor(private available: number) {}
tryEnter(): Permit | null {
if (this.available === 0) return null;
this.available -= 1;
let released = false;
return {
release: () => {
if (!released) {
released = true;
this.available += 1;
}
},
};
}
}
const interactiveDb = new CapacityGate(12);
const exportsDb = new CapacityGate(2);
async function getOrder(orderId: string, deadlineMs: number) {
const permit = interactiveDb.tryEnter();
if (!permit) return { status: 503, retryAfterSeconds: 1 };
try {
if (Date.now() >= deadlineMs) return { status: 504 };
// Acquire from the shared pool and run a query with the remaining deadline.
return { status: 200, body: await readOrder(orderId, deadlineMs) };
} finally {
permit.release();
}
}
This is not a recommendation to build an in-memory semaphore and call the problem solved. With multiple replicas, each gate needs a deliberately chosen local share, and important global quotas may need a queue or coordinator with durable state. The example shows the placement of the decision: it happens before an expensive request becomes an interchangeable connection waiter.
For an asynchronous export, the better boundary is often earlier still. Accept a job only after its tenant has an available queued or running-work allowance; otherwise return a documented limit response. A worker should acquire its database budget when it is actually ready to execute, rather than making thousands of accepted jobs all wait on the same database pool.
Keep the three limits separate
It helps to name three limits that are frequently compressed into one pool setting:
- Connection limit: how many server connections this process may hold. This is the pool's job.
- Execution limit: how many database-bound operations of a particular class may run at once. This protects latency and contention.
- Admission limit: how much waiting work the system is willing to promise. This protects memory, deadlines, and user expectations.
These values may coincide in a small service, but they express different promises. A pool of 20 does not imply that 20 reports should run at once, that 500 waiting requests are acceptable, or that every route should receive equal access to those connections.
The same distinction explains why a healthy-looking pool metric can be misleading. active = 20 might mean the database is producing useful work at its chosen concurrency. Or it might mean every connection is stuck behind one slow statement while a queue of urgent cancellations grows upstream. Track at least:
- pool checkout wait duration and timeout count;
- query or transaction duration after checkout;
- rejected and queued work, labelled by route or work class;
- queue age and deadline expiry; and
- capacity consumed per tenant or priority class when those partitions matter.
Pool exhaustion says that a scarce resource is busy. It does not identify whether the problem is a slow query, a leaked checkout, a hot tenant, a request spike, or a concurrency policy that has no business shape. The health-endpoint discussion makes a related point: one Boolean should not silently drive unrelated recovery actions. One pool metric should not silently choose unrelated work's fate.
Do not overcorrect into elaborate scheduling
Many applications do not need a fleet-wide fair scheduler. A small internal service with a handful of equally important requests can use one modest pool and a short checkout timeout quite successfully. Adding route-specific queues can create starvation bugs, surprising failure modes, and operational work that exceeds the value of the distinction.
Likewise, a connection-pool wait can be an entirely reasonable backpressure signal when the caller owns a bounded amount of work and treats failure as a cue to slow down. The problem begins when an unbounded or user-facing request stream treats that wait as an invisible holding area.
Start with the narrowest explicit policy that protects a real promise. Reserve a path for cancellation if exports can consume the pool. Make reporting asynchronous before it competes with checkout. Bound a queue before a retry storm fills it. Give interactive requests a deadline that includes waiting, not only query execution.
Test overload as a routing decision
Most pool tests assert that a checked-out client is returned. Keep that test, then add the questions the pool cannot answer:
Given: all export permits and database connections are occupied
When: a cancellation request arrives
Then: it uses its reserved path or receives a fast, explicit failure
Given: a request has 200 ms of deadline remaining
When: the database budget is unavailable
Then: it is rejected before starting work that cannot finish usefully
Given: tenant A fills its report allowance
When: tenant B requests an interactive read
Then: B's request is not queued behind A's reports
Given: a pool checkout is still waiting when an HTTP client disconnects
When: the connection later becomes available
Then: the application proves whether it starts, cancels, or deliberately reconciles the work
The last case matters because it exposes a missing ownership decision. A client disappearing does not tell the server whether the operation should stop; What “Cancel” Means After an Agent Starts Work explores that boundary in more detail. But a service should at least be able to say who decides—not leave it to a queue inside a resource helper.
References
- PostgreSQL, Connections and Authentication
- node-postgres, Pooling and Pool API
- node-postgres, Pool Sizing
- HikariCP, Configuration
- This repository, A Health Endpoint Is Not a Dependency Dashboard
- This repository, What “Cancel” Means After an Agent Starts Work
Final Take
A connection pool is a resource guard, not a fair scheduler and not an overload policy.
If a full pool is the first place the system decides who must wait, it has already discarded the information needed to make a good decision. Admit, bound, prioritize, or reject work while it still has a deadline and a purpose; let the pool do the smaller job it is actually built for.
Changelog
- 2026-11-12: Initial publication on connection pools and explicit admission policy.