Notes
Search…
Elasticsearch

1. API

Elasticsearch 的 API 在官方文档上有很详细的介绍,我这里仅列举出平时用到的较多的 API,方便查询。

滚动升级集群

1
# 首先关闭 shard 的分配
2
PUT _cluster/settings
3
{
4
"persistent": {
5
"cluster.routing.allocation.enable": "none"
6
}
7
}
8
9
# 然后写入硬盘
10
POST _flush/synced
11
12
# 升级完成后打开
13
PUT _cluster/settings
14
{
15
"persistent": {
16
"cluster.routing.allocation.enable": null
17
}
18
}
Copied!

Query API

1
# 时间范围查询
2
GET element-quantity-2/_search
3
{
4
"query": {
5
"range": {
6
"timestamp": {
7
"lte": "2018.11.13",
8
"format": "yyyy.MM.dd"
9
}
10
}
11
}
12
}
13
14
# 值的种类查询
15
GET element-quantity-2/_search
16
{
17
"size": 0,
18
"query": {
19
"match_all": {}
20
},
21
"aggs": {
22
"count": {
23
"cardinality": {
24
"field": "databag_id"
25
}
26
}
27
}
28
}
Copied!

Cat API

1
# 通用参数
2
## 显示标题
3
?v
4
5
## 选择显示哪些列
6
?h=A,B,C
7
8
## 排序
9
?s=A:desc,B
10
11
# 获取素有的节点
12
GET _cat/nodes
13
14
# 获取所有的 index
15
GET _cat/indices?v
16
17
# 获取符合某个 pattern 的 index 的 shard
18
GET _cat/shards/${pattern}
19
20
# 获取所有的 shard,按照 state 排序
21
GET _cat/shards?v&s=state
22
23
# 获取所有的 shard,显示指定列
24
GET _cat/shards?h=index,shard,prirep,state,unassigned.reason
25
26
# 获取特定类型的 thread_pool
27
GET /_cat/thread_pool/bulk,flush,refresh?v&h=node_name,name,active,queue,rejected,completed
28
29
# 获取所有的 thread_pool
30
GET /_cat/thread_pool?v&h=node_name,name,active,queue,rejected,completed
Copied!

Index API

1
# 关闭 Index
2
POST logstash-bfaccesslog-2018.12/_close
3
4
# 删除 Index,匹配格式
5
DELETE .monitoring-*
6
7
# 统一修改所有 index 的配置
8
PUT _settings
9
{
10
"index": {
11
"blocks": {
12
"read_only_allow_delete": "false"
13
}
14
}
15
}
16
17
# 修改某个 index 的配置
18
PUT portal/_settings
19
{
20
"index": {
21
"number_of_replicas":1
22
}
23
}
24
25
# 获取所有 index 的配置
26
GET _settings
27
28
# 获取所有 template
29
GET _template
30
31
# 增加自己的 template
32
PUT _template/my_logstash
33
{
34
"index_patterns": ["logstash-*"],
35
"order": 1,
36
"settings": {
37
"number_of_shards": 1
38
}
39
}
40
41
# 当 node 丢失的时候,shard 重新分配延迟时间
42
PUT _all/_settings
43
{
44
"settings": {
45
"index.unassigned.node_left.delayed_timeout": "5m"
46
}
47
}
Copied!

Cluster API

1
# 获取所有关闭的 index
2
GET _cluster/state/blocks?pretty
3
4
# 获取集群的配置
5
GET /_cluster/settings
6
7
# 分析 Shard 分配问题
8
GET /_cluster/allocation/explain
9
10
# 获取各个节点上的 task
11
GET _tasks
12
13
# 获取所有数据节点和存储空间
14
GET _cat/allocation?v
Copied!

Shard Allocation

  • cluster.routing.allocation.enable
    • all - (default) Allows shard allocation for all kinds of shards.
    • primaries - Allows shard allocation only for primary shards.
    • new_primaries - Allows shard allocation only for primary shards for new indices.
    • none - No shard allocations of any kind are allowed for any indices.
  • cluster.routing.rebalance.enable
    • all - (default) Allows shard balancing for all kinds of shards.
    • primaries - Allows shard balancing only for primary shards.
    • replicas - Allows shard balancing only for replica shards.
    • none - No shard balancing of any kind are allowed for any indices.
  • cluster.routing.allocation.allow_rebalance
    • always - Always allow rebalancing.
    • indices_primaries_active - Only when all primaries in the cluster are allocated.
    • indices_all_active - (default) Only when all shards (primaries and replicas) in the cluster are allocated.
  • cluster.routing.allocation.disk.threshold_enabled
    • true - default
    • false
  • index.unassigned.node_left.delayed_timeout
    • 见 Index API。
    • 注意:此配置是 index 级别的,所以就算配置的时候指定为_all新建的 index 也不会有这个配置。可以用 template 的方式增加此配置。

2. Tuning

  • index.refresh_interval:由 Buffer 写入 Segment(在 OS cache 中) 的频率,默认 1s。
  • index.merge.scheduler.max_thread_count:merge 操作的最大线程,默认有个公式,详情参考官方
  • index.translog.durability:默认 request,表示同步写入,参考官方文档
  • index.translog.sync_interval:默认 5s,表示 translog 由 OS cache 写入硬盘的频率。需要 close index 才能修改。
对于已存在的 index:
1
PUT .monitoring-*/_settings
2
{
3
"index": {
4
"refresh_interval": "30s",
5
"translog.durability": "async",
6
"merge.scheduler.max_thread_count": 1
7
}
8
}
Copied!
对于以后创建的 index:
  • order:数值越大越后执行,会覆盖数值小的。
1
PUT _template/my_config
2
{
3
"index_patterns": ["*"],
4
"order" : 1,
5
"settings": {
6
"index": {
7
"refresh_interval": "30s",
8
"translog.durability": "async",
9
"translog.sync_interval": "100s",
10
"merge.scheduler.max_thread_count": 1
11
}
12
}
13
}
Copied!
Last modified 2yr ago