Resource Management
Resource management in Flavius allows administrators to control and allocate system resources effectively. This document covers the key concepts and commands related to resource management.
ResourceGroup
NOTE: Currently, flavius has two buildin resource groups: query_resource_group and import_resource_group, where regular queries are submitted to query_resource_group and importing load jobs and backgroud compaction tasks are submitted to import_resource_group.
Currently we have the following configurations for resource groups.
| Key | Description | Value Type | Default Value |
|---|---|---|---|
| memory_limit_bytes | Memory limit of this resource group for each flavius compute node | Integer | |
| cpu_core_limit | For each flavius compute, how many cpu cores are allocated for this resource group’s compute threads | Integer | |
| io_core_limit | For each flavius compute, how many cpu cores are allocated for this resource group’s io threads | Integer | |
| concurrency_limit | Limit the number of concurrent running queries/jobs on this resource group | Integer |
If query queue enabled, the fowlloing configurations are used for manage the behavior of query queue.
| Key | Description | Value Type | Default Value |
|---|---|---|---|
| query_queue_enabled | If query queuing is enabled | Boolean | False |
| query_queue_timeout_second | How long the query stay in the queue before timeout by the queue. 0 means unlimited. | Integer | 0 |
| query_queue_concurrency_limit | If current running queries exceed this limit, new queries are pushed to the queue. 0 means unlimited. | Integer | 0 |
| query_queue_memory_percent_limit | If current flavius compute nodes memory useage exceed this limit, new queries will be pushed to the queue. 0 means unlimited. Max value 100. | Integer | 0 |
| query_queue_cpu_used_permille_limit | If current flavius compute node cpu usage exceed this limit, new queries will be pushed to the queue. cpu_usec_permille = folly::ThreadPool::usedCpuTime() / 1000000 / NUM_THREAD. 0 means unlimited. Max value 1000. | Integer | 0 |
| query_queue_thread_pool_pending_task_count_limit | If current flavius compute node thread pool pending task count exceed this limit, new queries will be pushed to the queue. 0 means unlimited | Integer | 0 |
List Resource Group
LIST RESOURCE GROUPDescribe Resource Group
DESCRIBE RESOURCE GROUP <group_name>NOTE: resource group names are case sensitive.
Example Show the configurations of Query Resource group.
DESCRIBE RESOURCE GROUP QueryAlter Resource Group
Update the resource group configurations.
ALTER RESOURCE GROUP <group_name> WITH (
<key> = <val>,
<key> = <val>,
...
)Example Update enable query queuing and update query queue concurreny limit
ALTER RESOURCE GROUP Query WITH (
query_queue_enabled = true,
query_queue_concurrency_limit = 100,
)