Elasticsearch

​本地调试 Elasticsearch​
​选主流程分析​
1. API
Elasticsearch 的 API 在官方文档上有很详细的介绍，我这里仅列举出平时用到的较多的 API，方便查询。
滚动升级集群
# 首先关闭 shard 的分配
PUT _cluster/settings
{
  "persistent": {
    "cluster.routing.allocation.enable": "none"
  }
}
​
# 然后写入硬盘
POST _flush/synced
​
# 升级完成后打开
PUT _cluster/settings
{
  "persistent": {
    "cluster.routing.allocation.enable": null
  }
}
Query API
# 时间范围查询
GET element-quantity-2/_search
{
  "query": {
    "range": {
      "timestamp": {
       "lte": "2018.11.13",
       "format": "yyyy.MM.dd"
      }
    }
  }
}
​
# 值的种类查询
GET element-quantity-2/_search
{
  "size": 0, 
  "query": {
    "match_all": {}
  },
  "aggs": {
    "count": {
      "cardinality": {
        "field": "databag_id"
      }
    }
  }
}
Cat API
# 通用参数
## 显示标题
?v
​
## 选择显示哪些列
?h=A,B,C
​
## 排序
?s=A:desc,B
​
# 获取素有的节点
GET _cat/nodes
​
# 获取所有的 index
GET _cat/indices?v
​
# 获取符合某个 pattern 的 index 的 shard
GET _cat/shards/${pattern}
​
# 获取所有的 shard，按照 state 排序
GET _cat/shards?v&s=state
​
# 获取所有的 shard，显示指定列
GET _cat/shards?h=index,shard,prirep,state,unassigned.reason
​
# 获取特定类型的 thread_pool
GET /_cat/thread_pool/bulk,flush,refresh?v&h=node_name,name,active,queue,rejected,completed
​
# 获取所有的 thread_pool
GET /_cat/thread_pool?v&h=node_name,name,active,queue,rejected,completed
Index API
# 关闭 Index
POST logstash-bfaccesslog-2018.12/_close
​
# 删除 Index，匹配格式
DELETE .monitoring-*
​
# 统一修改所有 index 的配置
PUT _settings
{
  "index": {
    "blocks": {
      "read_only_allow_delete": "false"
    }
  }
}
​
# 修改某个 index 的配置
PUT portal/_settings
{
  "index": {
    "number_of_replicas":1
  }
}
​
# 获取所有 index 的配置
GET _settings
​
# 获取所有 template
GET _template
​
# 增加自己的 template
PUT _template/my_logstash
{
  "index_patterns": ["logstash-*"],
  "order": 1,
  "settings": {
    "number_of_shards": 1
  }
}
​
# 当 node 丢失的时候，shard 重新分配延迟时间
PUT _all/_settings
{
  "settings": {
    "index.unassigned.node_left.delayed_timeout": "5m"
  }
}
Cluster API
# 获取所有关闭的 index
GET _cluster/state/blocks?pretty
​
# 获取集群的配置
GET /_cluster/settings
​
# 分析 Shard 分配问题
GET /_cluster/allocation/explain
​
# 获取各个节点上的 task
GET _tasks
​
# 获取所有数据节点和存储空间
GET _cat/allocation?v
Shard Allocation
参考 Cluster level shard allocation、Disk-based shard allocation。
cluster.routing.allocation.enable
all - (default) Allows shard allocation for all kinds of shards.
primaries - Allows shard allocation only for primary shards.
new_primaries - Allows shard allocation only for primary shards for new indices.
none - No shard allocations of any kind are allowed for any indices.
cluster.routing.rebalance.enable
all - (default) Allows shard balancing for all kinds of shards.
primaries - Allows shard balancing only for primary shards.
replicas - Allows shard balancing only for replica shards.
none - No shard balancing of any kind are allowed for any indices.
cluster.routing.allocation.allow_rebalance
always - Always allow rebalancing.
indices_primaries_active - Only when all primaries in the cluster are allocated.
indices_all_active - (default) Only when all shards (primaries and replicas) in the cluster are allocated.
cluster.routing.allocation.disk.threshold_enabled
true - default
false
index.unassigned.node_left.delayed_timeout
见 Index API。
注意：此配置是 index 级别的，所以就算配置的时候指定为_all，新建的 index 也不会有这个配置。可以用 template 的方式增加此配置。
2. Tuning
index.refresh_interval：由 Buffer 写入 Segment（在 OS cache 中） 的频率，默认 1s。
index.merge.scheduler.max_thread_count：merge 操作的最大线程，默认有个公式，详情参考官方。
index.translog.durability：默认 request，表示同步写入，参考官方文档。
index.translog.sync_interval：默认 5s，表示 translog 由 OS cache 写入硬盘的频率。需要 close index 才能修改。
对于已存在的 index：
PUT .monitoring-*/_settings
{
  "index": {
  	"refresh_interval": "30s",
    "translog.durability": "async",
    "merge.scheduler.max_thread_count": 1
  }
}
对于以后创建的 index：
order：数值越大越后执行，会覆盖数值小的。
PUT _template/my_config
{
  "index_patterns": ["*"],
  "order" : 1, 
  "settings": {
    "index": {
        "refresh_interval": "30s",
        "translog.durability": "async",
        "translog.sync_interval": "100s",
        "merge.scheduler.max_thread_count": 1
      }
  }
}
Last modified 3yr ago