In BigQuery, sharing slots is not possible except for baseline slots or committed slots. Auto-scaling enabled slots can be temporarily used, but they may scale down and become unavailable. This means that only these types of slots can be shared.
When ignore_idle_slots is set to false, it is possible to reserve zero slots and continue to access unused slots. For best practices, we recommend turning off ignore_idle_slots for default reservations and assigning a project or folder to the reservation. In this case, only idle slots will be used.
The ML_EXTERNAL type of allocation is an exception. Slots used by BigQuery ML external model creation jobs are non-preemptible. Both ML_EXTERNAL and QUERY types can be included in a reservation's slots, but they can only be used by other query jobs when the ML_EXTERNAL job is not occupying any slots. These jobs cannot use idle slots from other reservations.
Slot Allocation within Reservations
BigQuery uses an algorithm called fair scheduling to allocate slot capacity to a single reservation.
The BigQuery scheduler allocates slots evenly across projects and multiple jobs within a project, automatically adjusting the allocation as needed. The scheduler ensures fairness by preventing excessive consumption of resources. In rare cases, some jobs may receive more slots than others for a short period, but the scheduler will eventually correct this imbalance.
The goal of fair scheduling is to balance situations where tasks are executed excessively (resulting in wasted slot time) and situations where jobs consume an excessive amount of resources (resulting in imbalanced slot allocation).
BigQuery Fair Scheduling
In BigQuery, slots are allocated evenly across projects and then within each project. This means that all queries can access all available slots at any given time. Additionally, the capacity required by each query may vary, causing the scheduler to dynamically adjust slot allocation between active queries.
When a new query is submitted, the scheduler automatically re-allocates capacity between active queries. Each work unit has the flexibility to use more capacity if needed, and will be temporarily suspended or put in a queue as necessary.
Upon completion of a query, its consumed capacity becomes immediately available for all other queries. If a query's dynamic DAG changes, BigQuery recalculates the required capacity for that query and all others, adjusting slot allocation and suspension as needed to ensure fairness.
BigQuery's fair scheduling ensures that all slots are utilized dynamically, taking into account query complexity and size. This allows for efficient resource utilization and prevents excessive consumption of resources.