Elasticsearch-学习笔记

发表于 | 更新于: | 分类于 | 阅读次数:
字数统计: 1.8k

_cat APIs

通过访问elasticsearch集群的/_cat可以获取很多有用的信息。

使用curl -X GET "http://elasticsearch:9200/_cat"可以单独列出所有可用的命令

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
curl -X GET "http://elasticsearch-1:9200/_cat/"
=^.^=
/_cat/allocation
/_cat/shards
/_cat/shards/{index}
/_cat/master
/_cat/nodes
/_cat/tasks
/_cat/indices
/_cat/indices/{index}
/_cat/segments
/_cat/segments/{index}
/_cat/count
/_cat/count/{index}
/_cat/recovery
/_cat/recovery/{index}
/_cat/health
/_cat/pending_tasks
/_cat/aliases
/_cat/aliases/{alias}
/_cat/thread_pool
/_cat/thread_pool/{thread_pools}
/_cat/plugins
/_cat/fielddata
/_cat/fielddata/{fields}
/_cat/nodeattrs
/_cat/repositories
/_cat/snapshots/{repository}
/_cat/templates

获取_cat API帮助

  • 可以在/_cat/COMMAND后面加上参数help，获取对应的帮助信息
1
2
3
4
5
6
7
8
9
10
curl -XGET "http://elasticsearch-1:9200/_cat/allocation?help"
shards       | s              | number of shards on node      
disk.indices | di,diskIndices | disk used by ES indices       
disk.used    | du,diskUsed    | disk used (total, not just ES)
disk.avail   | da,diskAvail   | disk available                
disk.total   | dt,diskTotal   | total capacity of all volumes 
disk.percent | dp,diskPercent | percent disk used             
host         | h              | host of node                  
ip           |                | ip of node                    
node         | n              | name of node

获取当前master信息

1
2
3
curl -XGET "http://elasticsearch-1:9200/_cat/master?v"
id                     host          ip            node
JLyKyWH_QnSiXXuRNCj_3g 172.16.80.201 172.16.80.201 elasticsearch-1

获取节点信息

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
curl -XGET "http://elasticsearch-1:9200/_cat/nodes?format=json&pretty"
[
  {
    "ip" : "172.16.80.201",
    "heap.percent" : "7",
    "ram.percent" : "85",
    "cpu" : "1",
    "load_1m" : "0.00",
    "load_5m" : "0.08",
    "load_15m" : "0.20",
    "node.role" : "mdi",
    "master" : "*",
    "name" : "elasticsearch-1"
  }
]

获取索引信息

  • 以json格式返回结果
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
curl -XGET -H "Accept: application/json" "http://elasticsearch-1:9200/_cat/indices?format=json&pretty"
[
  {
    "health" : "yellow",
    "status" : "open",
    "index" : ".kibana",
    "uuid" : "V_t2TnJKQ9WV8E1C_uzIfA",
    "pri" : "1",
    "rep" : "1",
    "docs.count" : "6",
    "docs.deleted" : "0",
    "store.size" : "37.3kb",
    "pri.store.size" : "37.3kb"
  },
  {
    "health" : "yellow",
    "status" : "open",
    "index" : "bank",
    "uuid" : "3OEfeSwsTyKyC2zKHbuzJQ",
    "pri" : "5",
    "rep" : "1",
    "docs.count" : "1000",
    "docs.deleted" : "0",
    "store.size" : "640.8kb",
    "pri.store.size" : "640.8kb"
  },
  {
    "health" : "yellow",
    "status" : "open",
    "index" : "logstash-2015.05.18",
    "uuid" : "h8m1NlTIQS6Z49mHACYokA",
    "pri" : "5",
    "rep" : "1",
    "docs.count" : "4631",
    "docs.deleted" : "0",
    "store.size" : "27mb",
    "pri.store.size" : "27mb"
  },
  {
    "health" : "yellow",
    "status" : "open",
    "index" : "shakespeare",
    "uuid" : "xwoP7Pc3Q3ySZ47MosHrFA",
    "pri" : "5",
    "rep" : "1",
    "docs.count" : "111396",
    "docs.deleted" : "0",
    "store.size" : "29mb",
    "pri.store.size" : "29mb"
  },
  {
    "health" : "yellow",
    "status" : "open",
    "index" : "logstash-2015.05.20",
    "uuid" : "ewX3DcZHScOK4Mi6lF6Hqg",
    "pri" : "5",
    "rep" : "1",
    "docs.count" : "4750",
    "docs.deleted" : "0",
    "store.size" : "30.3mb",
    "pri.store.size" : "30.3mb"
  },
  {
    "health" : "yellow",
    "status" : "open",
    "index" : "logstash-2015.05.19",
    "uuid" : "siQA71vbSYOgE3a8M7HDhg",
    "pri" : "5",
    "rep" : "1",
    "docs.count" : "4624",
    "docs.deleted" : "0",
    "store.size" : "29.3mb",
    "pri.store.size" : "29.3mb"
  }
]
  • 以yaml格式返回结果
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
curl -XGET "http://elasticsearch-1:9200/_cat/indices?format=yaml&pretty"
---
- health: "yellow"
  status: "open"
  index: ".kibana"
  uuid: "V_t2TnJKQ9WV8E1C_uzIfA"
  pri: "1"
  rep: "1"
  docs.count: "6"
  docs.deleted: "0"
  store.size: "37.3kb"
  pri.store.size: "37.3kb"
- health: "yellow"
  status: "open"
  index: "bank"
  uuid: "3OEfeSwsTyKyC2zKHbuzJQ"
  pri: "5"
  rep: "1"
  docs.count: "1000"
  docs.deleted: "0"
  store.size: "640.8kb"
  pri.store.size: "640.8kb"
- health: "yellow"
  status: "open"
  index: "logstash-2015.05.18"
  uuid: "h8m1NlTIQS6Z49mHACYokA"
  pri: "5"
  rep: "1"
  docs.count: "4631"
  docs.deleted: "0"
  store.size: "27mb"
  pri.store.size: "27mb"
- health: "yellow"
  status: "open"
  index: "shakespeare"
  uuid: "xwoP7Pc3Q3ySZ47MosHrFA"
  pri: "5"
  rep: "1"
  docs.count: "111396"
  docs.deleted: "0"
  store.size: "29mb"
  pri.store.size: "29mb"
- health: "yellow"
  status: "open"
  index: "logstash-2015.05.20"
  uuid: "ewX3DcZHScOK4Mi6lF6Hqg"
  pri: "5"
  rep: "1"
  docs.count: "4750"
  docs.deleted: "0"
  store.size: "30.3mb"
  pri.store.size: "30.3mb"
- health: "yellow"
  status: "open"
  index: "logstash-2015.05.19"
  uuid: "siQA71vbSYOgE3a8M7HDhg"
  pri: "5"
  rep: "1"
  docs.count: "4624"
  docs.deleted: "0"
  store.size: "29.3mb"
  pri.store.size: "29.3mb"

_cluster APIs

官方文档见这里

Cluster Health

官方文档见这里

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
curl -XGET "http://elasticsearch-1:9200/_cluster/health?format=yaml&pretty"
---
cluster_name: "elasticsearch"
status: "yellow"
timed_out: false
number_of_nodes: 1
number_of_data_nodes: 1
active_primary_shards: 26
active_shards: 26
relocating_shards: 0
initializing_shards: 0
unassigned_shards: 26
delayed_unassigned_shards: 0
number_of_pending_tasks: 0
number_of_in_flight_fetch: 0
task_max_waiting_in_queue_millis: 0
active_shards_percent_as_number: 50.0

Cluster State

显示集群状态，官方文档见这里

查看版本信息

1
2
3
4
5
6
curl -XGET "http://elasticsearch-1:9200/_cluster/state/version?" 
{
  "cluster_name" : "elasticsearch",
  "version" : 16,
  "state_uuid" : "w0Bp1nGfRdCGxhA5zD1BBw"
}

查看master节点

1
2
3
4
5
curl -XGET "http://elasticsearch-1:9200/_cluster/state/master_node?format=json&pretty"
{
  "cluster_name" : "elasticsearch",
  "master_node" : "JLyKyWH_QnSiXXuRNCj_3g"
}

查看节点

1
2
3
4
5
6
7
8
9
10
11
12
curl -XGET "http://elasticsearch-1:9200/_cluster/state/nodes?format=json&pretty"
{
  "cluster_name" : "elasticsearch",
  "nodes" : {
    "JLyKyWH_QnSiXXuRNCj_3g" : {
      "name" : "elasticsearch-1",
      "ephemeral_id" : "qdnVkiizToCVPhmDieH3Mg",
      "transport_address" : "172.16.80.201:9300",
      "attributes" : { }
    }
  }
}

Cluster States

显示集群统计信息，官方文档见这里

1
curl -XGET "http://elasticsearch-1:9200/_cluster/stats?format=json&pretty"

Cluster Update Settings

用于更新集群设置，可以永久persistent和临时transient设置某些参数

  • 新启动的集群是没有配置的信息
1
2
3
4
5
curl -XGET "http://elasticsearch-1:9200/_cluster/settings?format=json&pretty"
{
  "persistent" : { },
  "transient" : { }
}
  • 临时修改集群recovery最大速度
1
2
3
4
5
6
7
8
9
curl -XPUT \
     -H "Content-Type: application/json" \
     "http://elasticsearch-1:9200/_cluster/settings?format=json&pretty" \
     -d '
{
    "persistent" : {
        "indices.recovery.max_bytes_per_sec" : "50mb"
    }
}'

Node Stats

1
curl -XGET "http://elasticsearch-1:9200/_nodes/stats?format=json&pretty"

Node Info

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# 不带过滤参数，默认显示所有节点信息
GET /_nodes
# 获取所有节点信息
GET /_nodes/_all
# 获取当前节点信息
GET /_nodes/_local
# 获取master节点信息
GET /_nodes/_master
# 以主机名作为条件过滤节点，或者使用通配符匹配节点
GET /_nodes/node_name_goes_here
GET /_nodes/node_name_goes_*
# 可以使用IP地址加通配符过滤节点
GET /_nodes/10.0.0.3,10.0.0.4
GET /_nodes/10.0.0.*
# 以role作为过滤条件
GET /_nodes/_all,master:false
GET /_nodes/data:true,ingest:true
GET /_nodes/coordinating_only:true
# 根据elasticsearch.yml文件定义的node.attr.rack属性过滤节点，例如`node.attr.rack: 2`
GET /_nodes/rack:2
GET /_nodes/ra*:2
GET /_nodes/ra*:2*

插件

官方文档见这里

二进制文件elasticsearch/bin/elasticsearch-plugin

查看默认可用的插件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
elasticsearch-plugin install --help
Install a plugin

The following official plugins may be installed by name:
  analysis-icu
  analysis-kuromoji
  analysis-phonetic
  analysis-smartcn
  analysis-stempel
  analysis-ukrainian
  discovery-azure-classic
  discovery-ec2
  discovery-file
  discovery-gce
  ingest-attachment
  ingest-geoip
  ingest-user-agent
  lang-javascript
  lang-python
  mapper-attachments
  mapper-murmur3
  mapper-size
  repository-azure
  repository-gcs
  repository-hdfs
  repository-s3
  store-smb
  x-pack

CRUD

新增索引

1
2
3
4
5
6
7
8
9
10
curl -X PUT "http://elasticsearch-1:9200/twitter" -H 'Content-Type: application/json' -d'
{
    "settings" : {
        "index" : {
            "number_of_shards" : 3, 
            "number_of_replicas" : 2 
        }
    }
}
'

删除索引

1
curl -X DELETE "http://elasticsearch-1:9200/twitter"

获取索引

1
curl -X GET "http://elasticsearch-1:9200/twitter"

更新索引

1
2
3
4
5
6
7
8
9
curl -X PUT "http://elasticsearch-1:9200/twitter/_settings" \
     -H 'Content-Type: application/json' \
     -d'
{
    "index" : {
        "number_of_replicas" : 2
    }
}
'

新增文档

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
curl -XPUT "http://elasticsearch-1:9200/students/class1/1?pretty" -d '
{
  "first_name": "Jing",
  "last_name": "Guo",
  "gender": "Male",
  "age": 25,
  "courses": "Xiang Long Shi Ba Zhang"
}'

curl -XPUT "http://elasticsearch-1:9200/students/class1/2?pretty" -d '
{
  "first_name": "Rong",
  "last_name": "Huang",
  "gender": "Female",
  "age": 23,
  "courses": "Luo Ying Shen Jian Zhang"
}'

curl -XPUT "http://elasticsearch-1:9200/students/class2/1?pretty" -d '
{
  "first_name": "Guo",
  "last_name": "Yang",
  "gender": "Male",
  "age": 2,
  "courses": "An Ran Xiao Hun Zhang"
}'

获取文档

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
curl -XGET "http://elasticsearch-1:9200/students/class1/1?pretty"
{
  "_index" : "students",
  "_type" : "class1",
  "_id" : "1",
  "_version" : 3,
  "found" : true,
  "_source" : {
    "first_name" : "Jing",
    "last_name" : "Guo",
    "gender" : "Male",
    "age" : 25,
    "courses" : "Xiang Long Shi Ba Zhang"
  }
}
curl -XGET "http://elasticsearch-1:9200/students/class1/2?pretty"
{
  "_index" : "students",
  "_type" : "class1",
  "_id" : "2",
  "_version" : 1,
  "found" : true,
  "_source" : {
    "first_name" : "Rong",
    "last_name" : "Huang",
    "gender" : "Female",
    "age" : 23,
    "courses" : "Luo Ying Shen Jian Zhang"
  }
}

更新文档

  • 直接PUT覆盖
  • 使用_updateAPI
1
2
3
4
5
6
7
8
curl -XPOST "http://elasticsearch-1:9200/students/class1/2/_update?pretty" \
     -H "Accept: application/json" \
     -d '
{
  "doc" : {
    "age" : 22
  }
}'

删除文档

1
curl -XDELETE "http://elasticsearch-1:9200/students/class1/2?pretty"

查询数据

Query DSL

官方文档

查询操作阶段

  • 分散阶段
  • 合并阶段

查询方式

1
curl -XGET "http://elasticsearch-1:9200/students/_search?pretty"
  • 通过发送REST request body查询，查询参数见官方文档
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# 匹配所有
curl -XGET "http://elasticsearch-1:9200/students/_search?pretty" -d '
{
    "query": {
        "match_all": {}
    }
}'

# 匹配年龄
curl -XGET "http://elasticsearch-1:9200/students/_search?pretty" -d '
{
  "query":{
    "bool":{
      "must":[
        {
          "range":{
            "age":{
              "gt":"21",
              "lt":"24"
            }
          }
        }
      ]
    }
  }
}'

多索引、多类型查询

  • _search：所有索引
  • /INDEX_NAME/_search：单索引
  • /INDEX1,INDEX2/_search：多索引
  • /students/class1/_search：单类型搜索
  • /students/class1,class2/_search：多类型搜索

Mapping和Analysis

  • Elasticsearch对每个文档，会取得所有域的所有值，生成一个名为“_all”的域
  • 执行查询的时候，如果未指定query_string，则在“_all”域上执行查询操作
1
2
3
4
# 在_all域搜索
curl /_search?q="Keyword"
# 在指定的FIELD_NAME这个域搜索
curl /_search?q=FIELD_NAME:"Keyword"

数据类型

见官方文档

  • array
  • binary
  • range
  • boolean
  • date
  • geo-point
  • geo-shape
  • IP
  • keyword
  • nested
  • numeric
  • object
  • text
  • token count
  • percolatro
  • join