elasticSearch入门

单机安装ElasticSearch

安装前，请确保已经安装JDK1.8
安装前，请确保已经安装nodejs6.0以上

官网：https://www.elastic.co/products/elasticsearch
下载安装包：https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.6.0.tar.gz
解压安装包：tar -vxf elasticsearch-5.6.0.tar.gz
cd elasticsearch-5.6.0.tar.gz

启动前，检查JDK环境
java -v
请确保已经安装JDK1.8

启动elasticsearch
sh ./bin/elasticsearch
当日志输出started时，表示启动成功

验证服务
127.0.0.1:9200
elasticsearch服务默认监听9200端口

访问：http://127.0.0.1:9200
如果出现版本信息，则安装成功

实用插件Head安装

打开github：https://github.com/mobz/elasticsearch-head
下载插件包：https://codeload.github.com/mobz/elasticsearch-head/zip/master
unzip elasticsearch-head-master.zip

cd elasticsearch-head-master

检查Node环境
node -v
请确保已经安装nodejs6.0以上

安装插件
npm install
启动插件
npm run start
输出日志表示启动成功
Started connect web server on http://localhost:9100
访问
http://localhost:9100

ElasticSearch整合elasticsearch-head插件
cd elasticsearch-5.6.0

vim config/elasticsearch.yml
在配置文件的最后面加上
允许head插件跨域访问rest接口
http.cors.allowed: true
http.cors.allow-origin: "*"
:wq

后台启动
./bin/elasticsearch -d

再次重新启动elasticsearch-head插件
cd elasticsearch-head-master
启动插件
npm run start
访问
http://localhost:9100

集群安装
1个master、2个slave

master节点配置

配置当前节点为主节点
cd elasticsearch-5.6.0
修改配置
vim config/elasticsearch.yml
在配置文件的最后面加上

# 指定集群的名字
cluster.name: myes
# 指定当前节点的名字
node.name: master
# 指定当前节点为master
node.master: true
# 指定绑定的IP
network.host: 127.0.0.1
# 使用默认端口：9200
:wq

ps -ef | grep 'pwd'

kill pid
重新启动
./bin/elasticsearch -d
检查服务是否正常启动
http://localhost:9200

slave节点配置

mkdir es_slave
cp elasticsearch-5.6.0.tar.gz es_slave/
cd es_slave
tar -vxf elasticsearch-5.6.0.tar.gz
cp -r elasticsearch-5.6.0 es_slave1
cp -r elasticsearch-5.6.0 es_slave2

修改es_slave1配置
cd es_slave1
vim config/elasticsearch.yml
在配置文件的最后面加上

# 指定集群的名字：需要和master节点一致
cluster.name: myes
# 指定当前节点的名字
node.name: slave1
# 指定绑定的IP
network.host: 127.0.0.1
# 指定当前节点绑定端口号8200
http.port: 8200
# 该配置主要是为了找到master节点
discovery.zen.ping.unicast.hosts: ["127.0.0.1"]
:wq

启动服务
./bin/elasticsearch -d
检查服务是否正常启动
http://localhost:9100

安装之前的步骤配置slave2
cd es_slave2
vim config/elasticsearch.yml
在配置文件的最后面加上

# 指定集群的名字：需要和master节点一致
cluster.name: myes
# 指定当前节点的名字
node.name: slave2
# 指定绑定的IP
network.host: 127.0.0.1
# 指定当前节点绑定端口号8000
http.port: 8000
# 该配置主要是为了找到master节点
discovery.zen.ping.unicast.hosts: ["127.0.0.1"]
:wq

启动服务
./bin/elasticsearch -d
检查服务是否正常启动
http://localhost:9100

一个 ES Index 在集群模式下，有多个 Node （节点）组成。每个节点就是 ES 的Instance (实例)。
每个节点上会有多个 shard （分片）， P1 P2 是主分片, R1 R2 是副本分片
每个分片上对应着就是一个 Lucene Index（底层索引文件）
Lucene Index 是一个统称
由多个 Segment （段文件，就是倒排索引）组成。每个段文件存储着就是 Doc 文档
commit point记录了所有 segments 的信息

一个集群是由一个或多个ES组成的集合
每一个集群都有一个唯一的名字
每一个节点都是通过集群的名字来加入集群的
每一个节点都有自己的名字
节点能够存储数据，参与集群索引数据以及搜索数据的独立服务

索引：含有相同属性的文档集合；Elasticsearch 会以 JSON 文档的形式存储数据。每个文档都会在一组键 ( 字段或属性的名称 ) 和它们对应的值 ( 字符串、数字、布尔值、日期、数值组、地理位置或其他类型的数据 ) 之间建立联系。
Elasticsearch 使用的是一种名为倒排索引的数据结构，这一结构的设计可以允许十分快速地进行全文本搜索。倒排索引会列出在所有文档中出现的每个特有词汇，并且可以找到包含每个词汇的全部文档。
在索引过程中，Elasticsearch 会存储文档并构建倒排索引，这样用户便可以近实时地对文档数据进行搜索。索引过程是在索引 API 中启动的，通过此 API 您既可向特定索引中添加 JSON 文档，也可更改特定索引中的 JSON 文档。

类型：索引可以定义一个或多个类型，文档必须属于一个类型
（通常会定义有相同字段的文档作为一个类型）
文档：文档是可以被索引的基本数据单位

索引相当于SQL里的DataBase，也就是数据库
类型相当于SQL里的Table，也就是表
文档相当于SQL里的一行记录，也就是一行数据

假设有一个信息查询系统，使用ES做存储。那么里面的数据就可以分为各种各样的索引，比如：汽车索引、图书索引、家具索引等等。图书索引又可以细分为各种类型，比如：科普类、小说类、技术类等等。具体到每一本书籍，就是文档，就是整个图书里面最小的存储单位。

分片：每个索引都有多个分片，每个分片是一个Lucene索引
备份：拷贝一份分片就完成了分片的备份
ES默认在创建索引时，会创建5个分片、1个备份
分片的数量只能在创建索引时设置，而不能在后期进行修改
备份是可以动态修改的

"Set the shape to semi-transparent by calling set_trans(5)"
分析后
set, the, shape, to, semi, transparent, by, calling, set_trans, 5

set, the, shape, to, semi, transparent, by, calling, set, trans

Set, the, shape, to, semi-transparent, by, calling, set_trans(5)

action.auto_create_index: false

PUT /my_index
{
    "settings": { ... any settings ... },
    "mappings": {
        "properties": { ... any properties ... }
    }
}

settings: 用来设置分片,副本等配置信息
mappings: 字段映射，类型等
properties: 属性

创建索引
PUT /test-index-users
{
  "settings": {
		"number_of_shards": 1,  //分片数为1个
		"number_of_replicas": 1 //副本数为1个
	},
  "mappings": {
    "properties": {
      "name": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "age": {
        "type": "long"
      },
      "remarks": {
        "type": "text"
      }
    }
  }
}

返回结果：
{
 "acknowledged":true,
 "shards_acknowledged":true,
 "index":"test-index-users"
}

查看索引结构
GET /test-index-users/_mapping

返回结果：
{
  "test-index-users": {
    "mappings": {
      "properties": {
        "age": {
          "type": "long"
        },
        "name": {
          "fields": {
            "keyword": {
              "ignore_above": 256,
              "type": "keyword"
            }
          },
          "type": "text"
        },
        "remarks": {
          "type": "text"
        }
      }
    }
  }
}

查看索引设置
GET /test-index-users/_settings

返回结果：
{
  "test-index-users": {
    "settings": {
      "index": {
        "apack": {
          "metadata": {
            "app_id": "prod-es-4fs",
            "index_max_storage": "60.0",
            "shard_max_storage": "20.0"
          }
        },
        "blocks": {
          "read_only_allow_delete": "false"
        },
        "creation_date": "1740131244421",
        "indexing": {
          "slowlog": {
            "source": "1000",
            "threshold": {
              "index": {
                "debug": "50ms",
                "info": "100ms",
                "trace": "20ms",
                "warn": "200ms"
              }
            }
          }
        },
        "mapping": {
          "field_name_length": {
            "limit": "100"
          },
          "nested_objects": {
            "limit": "100"
          }
        },
        "max_prefix_length": "50",
        "max_refresh_listeners": "20",
        "max_regex_length": "50",
        "max_terms_count": "1024",
        "max_wildcard_length": "50",
        "number_of_replicas": "1",
        "number_of_shards": "3",
        "provided_name": "test-index-users",
        "refresh_interval": "1s",
        "routing": {
          "allocation": {
            "include": {
              "_tier_preference": "data_content"
            }
          }
        },
        "search": {
          "slowlog": {
            "threshold": {
              "fetch": {
                "debug": "80ms",
                "info": "100ms",
                "trace": "50ms",
                "warn": "200ms"
              },
              "query": {
                "debug": "100ms",
                "info": "200ms",
                "trace": "50ms",
                "warn": "500ms"
              }
            }
          }
        },
        "unassigned": {
          "node_left": {
            "delayed_timeout": "5m"
          }
        },
        "uuid": "a2vXgFIFR1Ouwi5hP-XdAA",
        "version": {
          "created": "7100099"
        }
      }
    }
  }
}


插入一组测试数据
POST /test-index-users/_doc
{
  "name":"myname",
  "age":18,
  "remarks":"hello world"
}

返回结果：
{
  "_index": "test-index-users",
  "_type": "_doc",
  "_id": "B3noJ5UBCmMdHuxJFhcB",
  "_version": 1,
  "result": "created",
  "_shards": {
    "total": 2,
    "successful": 2,
    "failed": 0
  },
  "_seq_no": 0,
  "_primary_term": 1
}


查看数据
GET /test-index-users/_search
{
  "query":{
    "match_all":{}
  }
}

返回结果
{
  "_shards": {
    "failed": 0,
    "skipped": 0,
    "successful": 3,
    "total": 3
  },
  "hits": {
    "hits": [
      {
        "_id": "B3noJ5UBCmMdHuxJFhcB",
        "_index": "test-index-users",
        "_score": 1,
        "_source": {
          "age": 18,
          "name": "myname",
          "remarks": "hello world"
        },
        "_type": "_doc"
      }
    ],
    "max_score": 1,
    "total": {
      "relation": "eq",
      "value": 1
    }
  },
  "timed_out": false,
  "took": 4
}


修改索引配置
PUT /test-index-users/_settings
{
  "settings": {
    "number_of_replicas": 0
  }
}


关闭索引，一旦索引被关闭，那么这个索引只能显示元数据信息，不能够进行读写操作
POST /test-index-users/_close

打开索引
POST /test-index-users/_open

删除索引
DELETE /test-index-users

POST _search
{
  "query": {
    "bool" : {
      "must" : {
        "term" : { "user.id" : "kimchy" }
      },
      "filter": {
        "term" : { "tags" : "production" }
      },
      "must_not" : {
        "range" : {
          "age" : { "gte" : 10, "lte" : 20 }
        }
      },
      "should" : [
        { "term" : { "tags" : "env1" } },
        { "term" : { "tags" : "deployed" } }
      ],
      "minimum_should_match" : 1,
      "boost" : 1.0
    }
  }
}
在filter元素下指定的查询对评分没有影响 , 评分返回为0。分数仅受已指定查询的影响

GET _search
{
  "query": {
    "bool": {
      "filter": {
        "term": {
          "status": "active"
        }
      }
    }
  }
}
查询为所有文档分配0分，因为没有指定评分查询

GET _search
{
  "query": {
    "bool": {
      "must": {
        "match_all": {}
      },
      "filter": {
        "term": {
          "status": "active"
        }
      }
    }
  }
}
此bool查询具有match_all查询，该查询为所有文档指定1.0分

GET /_search
{
  "query": {
    "bool": {
      "should": [
        { "match": { "name.first": { "query": "shay", "_name": "first" } } },
        { "match": { "name.last": { "query": "banon", "_name": "last" } } }
      ],
      "filter": {
        "terms": {
          "name.last": [ "banon", "kimchy" ],
          "_name": "test"
        }
      }
    }
  }
}
每个query条件都可以有一个_name属性，用来追踪搜索出的数据到底match了哪个条件

创建数据
POST /test-dsl-boosting/_bulk
{ "index": { "_id": 1 }}
{ "content":"Apple Mac" }
{ "index": { "_id": 2 }}
{ "content":"Apple Fruit" }
{ "index": { "_id": 3 }}
{ "content":"Apple employee like Apple Pie and Apple Juice" }


查询数据
GET /test-dsl-boosting/_search
{
  "query": {
    "boosting": {
      "positive": {
        "term": {
          "content": "apple"
        }
      },
      "negative": {
        "term": {
          "content": "pie"
        }
      },
      "negative_boost": 0.5
    }
  }
}
查询结果对匹配pie的做降级显示处理

创建数据
POST /test-dsl-constant/_bulk
{ "index": { "_id": 1 }}
{ "content":"Apple Mac" }
{ "index": { "_id": 2 }}
{ "content":"Apple Fruit" }


查询数据
GET /test-dsl-constant/_search
{
  "query": {
    "constant_score": {
      "filter": {
        "term": { "content": "apple" }
      },
      "boost": 1.2
    }
  }
}
查询到的每条数据的score都是1.2

创建数据
POST /test-dsl-dis-max/_bulk
{ "index": { "_id": 1 }}
{"title": "Quick brown rabbits","body":  "Brown rabbits are commonly seen."}
{ "index": { "_id": 2 }}
{"title": "Keeping pets healthy","body":  "My quick brown fox eats rabbits on a regular basis."}


用户输入词组 “Brown fox” 然后点击搜索按钮。事先，我们并不知道用户的搜索项是会在 title 还是在 body 字段中被找到，但是，用户很有可能是想搜索相关的词组。用肉眼判断，文档 2 的匹配度更高，因为它同时包括要查找的两个词
GET /test-dsl-dis-max/_search
{
    "query": {
        "bool": {
            "should": [
                { "match": { "title": "Brown fox" }},
                { "match": { "body":  "Brown fox" }}
            ]
        }
    }
}
根据分数计算，查询结果中文档1在前，文档2在后


使用 dis_max 即分离 最大化查询（Disjunction Max Query） 。分离（Disjunction）的意思是 或（or） ，这与可以把结合（conjunction）理解成 与（and） 相对应。分离最大化查询（Disjunction Max Query）指的是： 将任何与任一查询匹配的文档作为结果返回，但只将最佳匹配的评分作为查询的评分结果返回
GET /test-dsl-dis-max/_search
{
    "query": {
        "dis_max": {
            "queries": [
                { "match": { "title": "Brown fox" }},
                { "match": { "body":  "Brown fox" }}
            ],
            "tie_breaker": 0
        }
    }
}
分数 = 第一个匹配条件分数 + tie_breaker * 第二个匹配的条件的分数
这样你就能理解通过dis_max将doc 2 置前了， 当然这里如果缺省tie_breaker字段的话默认就是0，你还可以设置它的比例（在0到1之间）来控制排名。（显然值为1时和should查询是一致的）

GET /_search
{
  "query": {
    "function_score": {
      "query": { "match_all": {} },
      "boost": "5", 
      "functions": [
        {
          "filter": { "match": { "test": "bar" } },
          "random_score": {}, 
          "weight": 23
        },
        {
          "filter": { "match": { "test": "cat" } },
          "weight": 42
        }
      ],
      "max_boost": 42,
      "score_mode": "max",
      "boost_mode": "multiply",
      "min_score": 42
    }
  }
}


GET /_search
{
  "query": {
    "function_score": {
      "query": {
        "match": { "message": "elasticsearch" }
      },
      "script_score": {
        "script": {
          "source": "Math.log(2 + doc['my-int'].value)"
        }
      }
    }
  }
}

POST /test-dsl-match/_bulk
{ "index": { "_id": 1 }}
{ "title": "The quick brown fox" }
{ "index": { "_id": 2 }}
{ "title": "The quick brown fox jumps over the lazy dog" }
{ "index": { "_id": 3 }}
{ "title": "The quick brown fox jumps over the quick dog" }
{ "index": { "_id": 4 }}
{ "title": "Brown fox brown dog" }

GET /test-dsl-match/_search
{
    "query": {
        "match": {
            "title": "QUICK!"
        }
    }
}
执行上面这个 match 查询的步骤是：
检查字段类型：标题 title 字段是一个 string 类型（ analyzed ）已分析的全文字段，这意味着查询字符串本身也应该被分析。

分析查询字符串：将查询的字符串 QUICK! 传入标准分析器中，输出的结果是单个项 quick 。因为只有一个单词项，所以 match 查询执行的是单个底层 term 查询。

查找匹配文档：用 term 查询在倒排索引中查找 quick 然后获取一组包含该项的文档，本例的结果是文档：1、2 和 3 。

为每个文档评分：用 term 查询计算每个文档相关度评分 _score ，这是种将词频（term frequency，即词 quick 在相关文档的 title 字段中出现的频率）和反向文档频率（inverse document frequency，即词 quick 在所有文档的 title 字段中出现的频率），以及字段的长度（即字段越短相关度越高）相结合的计算方式。

GET /test-dsl-match/_search
{
    "query": {
        "match": {
            "title": "BROWN DOG"
        }
    }
}
因为 match 查询必须查找两个词（ ["brown","dog"] ），它在内部实际上先执行两次 term 查询，然后将两次查询的结果合并作为最终结果输出。为了做到这点，它将两个 term 查询包入一个 bool 查询中
所以上述查询的结果，和如下语句查询结果是等同的
GET /test-dsl-match/_search
{
  "query": {
    "bool": {
      "should": [
        {
          "term": {
            "title": "brown"
          }
        },
        {
          "term": {
            "title": "dog"
          }
        }
      ]
    }
  }
}
上面等同于should（任意一个满足），是因为 match还有一个operator参数，默认是or, 所以对应的是should。
所以上述查询也等同于
GET /test-dsl-match/_search
{
  "query": {
    "match": {
      "title": {
        "query": "BROWN DOG",
        "operator": "or"
      }
    }
  }
}
如果是需要and操作呢，即同时满足
GET /test-dsl-match/_search
{
  "query": {
    "match": {
      "title": {
        "query": "BROWN DOG",
        "operator": "and"
      }
    }
  }
}
他就等同于bool的must
GET /test-dsl-match/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "title": "brown"
          }
        },
        {
          "term": {
            "title": "dog"
          }
        }
      ]
    }
  }
}

GET /test-dsl-match/_search
{
  "query": {
    "match": {
      "title": {
        "query":"quick brown dog",
        "minimum_should_match": "75%"
      }
    }
  }
}
以上查询也等同于
GET /test-dsl-match/_search
{
  "query": {
    "bool": {
      "should": [
        { "match": { "title": "quick" }},
        { "match": { "title": "brown"   }},
        { "match": { "title": "dog"   }}
      ],
      "minimum_should_match": 2 
    }
  }
}

GET /test-dsl-match/_search
{
  "query": {
    "match_phrase": {
      "title": {
        "query": "quick brown f"
      }
    }
  }
}
f并不是一个分词，不满足term查询，所以最终查不出任何内容了

GET /test-dsl-match/_search
{
  "query": {
    "match_phrase_prefix": {
      "title": {
        "query": "quick brown f"
      }
    }
  }
}
这个查询就有结果了
prefix的意思不是整个text的开始匹配，而是最后一个词项满足term的prefix查询而已

GET /test-dsl-match/_search
{
  "query": {
    "match_bool_prefix": {
      "title": {
        "query": "quick brown f"
      }
    }
  }
}
结果是3,2,4
本质上可以转换为
GET /test-dsl-match/_search
{
  "query": {
    "bool" : {
      "should": [
        { "term": { "title": "quick" }},
        { "term": { "title": "brown" }},
        { "prefix": { "title": "f"}}
      ]
    }
  }
}

{
  "query": {
    "multi_match" : {
      "query":    "Will Smith",
      "fields": [ "title", "*_name" ] 
    }
  }
}
同时查询title字段和以_name结尾的字段

GET /test-dsl-match/_search
{
  "query": {
    "query_string": {
      "query": "(lazy dog) OR (brown dog)",
      "default_field": "title"
    }
  }
}
本质上查询这四个分词（term）or的结果而已,查询结果是4,3,1

GET /test-dsl-match/_search
{
  "query": {
    "simple_query_string" : {
        "query": "\"over the\" + (lazy | quick) + dog",
        "fields": ["title"],
        "default_operator": "and"
    }
  }
}
结果是2,3

GET /test-dsl-match/_search
{
  "query": {
    "intervals" : {
      "title" : {
        "all_of" : {
          "ordered" : true,
          "intervals" : [
            {
              "match" : {
                "query" : "quick",
                "max_gaps" : 0,
                "ordered" : true
              }
            },
            {
              "any_of" : {
                "intervals" : [
                  { "match" : { "query" : "jump over" } },
                  { "match" : { "query" : "quick dog" } }
                ]
              }
            }
          ]
        }
      }
    }
  }
}

PUT /test-dsl-term-level
{
  "mappings": {
    "properties": {
      "name": {
        "type": "keyword"
      },
      "programming_languages": {
        "type": "keyword"
      },
      "required_matches": {
        "type": "long"
      }
    }
  }
}

POST /test-dsl-term-level/_bulk
{ "index": { "_id": 1 }}
{"name": "Jane Smith", "programming_languages": [ "c++", "java" ], "required_matches": 2}
{ "index": { "_id": 2 }}
{"name": "Jason Response", "programming_languages": [ "java", "php" ], "required_matches": 2}
{ "index": { "_id": 3 }}
{"name": "Dave Pdai", "programming_languages": [ "java", "c++", "php" ], "required_matches": 3, "remarks": "hello world"}

GET /test-dsl-term-level/_search
{
  "query": {
    "term": {
      "programming_languages": "php"
    }
  }
}
查询结果是2,3

GET /test-dsl-term-level/_search
{
  "query": {
    "terms": {
      "programming_languages": ["php","c++"]
    }
  }
}
查询结果是1，2,3

GET /test-dsl-term-level/_search
{
  "query": {
    "terms_set": {
      "programming_languages": {
        "terms": [ "java", "php" ],
        "minimum_should_match_field": "required_matches"
      }
    }
  }
}
查询结果是2

GET /test-dsl-term-level/_search
{
  "query": {
    "exists": {
      "field": "remarks"
    }
  }
}

//查询存在remarks字段的文档，结果是3

GET /test-dsl-term-level/_search
{
  "query": {
    "ids": {
      "values": [3, 1]
    }
  }
}
查询结果是1,3；查询和id顺序无关

GET /test-dsl-term-level/_search
{
  "query": {
    "prefix": {
      "name": {
        "value": "Jan"
      }
    }
  }
}
查询结果是1

GET /test-dsl-term-level/_search
{
  "query": {
    "wildcard": {
      "name": {
        "value": "D*ai",
        "boost": 1.0,
        "rewrite": "constant_score"
      }
    }
  }
}
查询结果为3，name="Dave Pdai"

GET /test-dsl-term-level/_search
{
  "query": {
    "range": {
      "required_matches": {
        "gte": 3,
        "lte": 4
      }
    }
  }
}
查询结果为文档3

GET /test-dsl-term-level/_search
{
  "query": {
    "regexp": {
      "name": {
        "value": "Ja.*",
        "case_insensitive": true
      }
    }
  }
}
查询结果为文档1,2

GET /test-dsl-term-level/_search
{
  "query": {
    "fuzzy": {
      "remarks": {
        "value": "hell"
      }
    }
  }
}
查询结果为文档3

关于汽车交易的信息：车型、颜色、制造商、售价、何时被出售等。
POST /test-agg-cars/_bulk
{ "index": {}}
{ "price" : 10000, "color" : "red", "make" : "honda", "sold" : "2014-10-28" }
{ "index": {}}
{ "price" : 20000, "color" : "red", "make" : "honda", "sold" : "2014-11-05" }
{ "index": {}}
{ "price" : 30000, "color" : "green", "make" : "ford", "sold" : "2014-05-18" }
{ "index": {}}
{ "price" : 15000, "color" : "blue", "make" : "toyota", "sold" : "2014-07-02" }
{ "index": {}}
{ "price" : 12000, "color" : "green", "make" : "toyota", "sold" : "2014-08-19" }
{ "index": {}}
{ "price" : 20000, "color" : "red", "make" : "honda", "sold" : "2014-11-05" }
{ "index": {}}
{ "price" : 80000, "color" : "red", "make" : "bmw", "sold" : "2014-01-01" }
{ "index": {}}
{ "price" : 25000, "color" : "blue", "make" : "ford", "sold" : "2014-02-12" }

GET /test-agg-cars/_search
{
    "size" : 0,
    "aggs" : { 
        "popular_colors" : { 
            "terms" : { 
              "field" : "color.keyword"
            }
        }
    }
}
聚合操作被置于顶层参数 aggs 之下（如果你愿意，完整形式 aggregations 同样有效）。
可以为聚合指定一个我们想要名称，本例中是： popular_colors 。
定义单个桶的类型 terms 。
因为我们设置了 size 参数，所以不会有 hits 搜索结果返回。
popular_colors 聚合是作为 aggregations 字段的一部分被返回的。
每个桶的 key 都与 color 字段里找到的唯一词对应。它总会包含 doc_count 字段，告诉我们包含该词项的文档数量。
每个桶的数量代表该颜色的文档数量。

GET /test-agg-cars/_search
{
    "size" : 0,
    "aggs" : { 
        "popular_colors" : { 
            "terms" : { 
              "field" : "color.keyword"
            }
        },
        "make_by" : { 
            "terms" : { 
              "field" : "make.keyword"
            }
        }
    }
}

GET /test-agg-cars/_search
{
   "size" : 0,
   "aggs": {
      "colors": {
         "terms": {
            "field": "color.keyword"
         },
         "aggs": { 
            "avg_price": { 
               "avg": {
                  "field": "price" 
               }
            }
         }
      }
   }
}
我们需要给度量起一个名字（ avg_price ）这样可以稍后根据名字获取它的值。最后，我们指定度量本身（ avg ）以及我们想要计算平均值的字段（ price ）

GET /test-agg-cars/_search
{
  "runtime_mappings": {
    "make.length": {
      "type": "long",
      "script": "emit(doc['make.keyword'].value.length())"
    }
  },
  "size" : 0,
  "aggs": {
    "make_length": {
      "histogram": {
        "interval": 1,
        "field": "make.length"
      }
    }
  }
}

GET /test-agg-cars/_search
{
  "size": 0,
  "aggs": {
    "make_by": {
      "filter": { "term": { "type": "honda" } },
      "aggs": {
        "avg_price": { "avg": { "field": "price" } }
      }
    }
  }
}
查询制造商为honda的均价

设计一个新的例子, 日志系统中，每条日志都是在文本中，包含warning/info等信息。
PUT /test-agg-logs/_bulk
{ "index" : { "_id" : 1 } }
{ "body" : "warning: page could not be rendered" }
{ "index" : { "_id" : 2 } }
{ "body" : "authentication error" }
{ "index" : { "_id" : 3 } }
{ "body" : "warning: connection timed out" }
{ "index" : { "_id" : 4 } }
{ "body" : "info: hello pdai" }


对包含不同日志类型的日志进行分组，这就需要filters:
GET /test-agg-logs/_search
{
  "size": 0,
  "aggs" : {
    "messages" : {
      "filters" : {
        "other_bucket_key": "other_messages",
        "filters" : {
          "infos" :   { "match" : { "body" : "info"   }},
          "warnings" : { "match" : { "body" : "warning" }}
        }
      }
    }
  }
}

GET /test-agg-cars/_search
{
  "size": 0,
  "aggs": {
    "price_ranges": {
      "range": {
        "field": "price",
        "ranges": [
          { "to": 20000 },
          { "from": 20000, "to": 40000 },
          { "from": 40000 }
        ]
      }
    }
  }
}

GET /ip_addresses/_search
{
  "size": 10,
  "aggs": {
    "ip_ranges": {
      "ip_range": {
        "field": "ip",
        "ranges": [
          { "to": "10.0.0.5" },
          { "from": "10.0.0.5" }
        ]
      }
    }
  }
}

返回结果
{
  ...

  "aggregations": {
    "ip_ranges": {
      "buckets": [
        {
          "key": "*-10.0.0.5",
          "to": "10.0.0.5",
          "doc_count": 10
        },
        {
          "key": "10.0.0.5-*",
          "from": "10.0.0.5",
          "doc_count": 260
        }
      ]
    }
  }
}


此外还可以用CIDR Mask分组
GET /ip_addresses/_search
{
  "size": 0,
  "aggs": {
    "ip_ranges": {
      "ip_range": {
        "field": "ip",
        "ranges": [
          { "mask": "10.0.0.0/25" },
          { "mask": "10.0.0.127/25" }
        ]
      }
    }
  }
}
返回结果
{
  ...

  "aggregations": {
    "ip_ranges": {
      "buckets": [
        {
          "key": "10.0.0.0/25",
          "from": "10.0.0.0",
          "to": "10.0.0.128",
          "doc_count": 128
        },
        {
          "key": "10.0.0.127/25",
          "from": "10.0.0.0",
          "to": "10.0.0.128",
          "doc_count": 128
        }
      ]
    }
  }
}

增加key显示
GET /ip_addresses/_search
{
  "size": 0,
  "aggs": {
    "ip_ranges": {
      "ip_range": {
        "field": "ip",
        "ranges": [
          { "to": "10.0.0.5" },
          { "from": "10.0.0.5" }
        ],
        "keyed": true // here
      }
    }
  }
}

返回结果
{
  ...

  "aggregations": {
    "ip_ranges": {
      "buckets": {
        "*-10.0.0.5": {
          "to": "10.0.0.5",
          "doc_count": 10
        },
        "10.0.0.5-*": {
          "from": "10.0.0.5",
          "doc_count": 260
        }
      }
    }
  }
}

自定义key显示
GET /ip_addresses/_search
{
  "size": 0,
  "aggs": {
    "ip_ranges": {
      "ip_range": {
        "field": "ip",
        "ranges": [
          { "key": "infinity", "to": "10.0.0.5" },
          { "key": "and-beyond", "from": "10.0.0.5" }
        ],
        "keyed": true
      }
    }
  }
}

返回结果
{
  ...

  "aggregations": {
    "ip_ranges": {
      "buckets": {
        "infinity": {
          "to": "10.0.0.5",
          "doc_count": 10
        },
        "and-beyond": {
          "from": "10.0.0.5",
          "doc_count": 260
        }
      }
    }
  }
}

GET /test-agg-cars/_search
{
  "size": 0,
  "aggs": {
    "range": {
      "date_range": {
        "field": "sold",
        "format": "yyyy-MM",
        "ranges": [
          { "from": "2014-01-01" },  
          { "to": "2014-12-31" } 
        ]
      }
    }
  }
}

avg平均值
计算班级的平均分
POST /exams/_search?size=0
{
  "aggs": {
    "avg_grade": { "avg": { "field": "grade" } }
  }
}
返回
{
  ...
  "aggregations": {
    "avg_grade": {
      "value": 75.0
    }
  }
}

max最大值
计算销售最高价
POST /sales/_search?size=0
{
  "aggs": {
    "max_price": { "max": { "field": "price" } }
  }
}
返回
{
  ...
  "aggregations": {
      "max_price": {
          "value": 200.0
      }
  }
}

min最小值
计算销售最低价
POST /sales/_search?size=0
{
  "aggs": {
    "min_price": { "min": { "field": "price" } }
  }
}
返回
{
  ...

  "aggregations": {
    "min_price": {
      "value": 10.0
    }
  }
}

sum计算和
计算销售总价
POST /sales/_search?size=0
{
  "query": {
    "constant_score": {
      "filter": {
        "match": { "type": "hat" }
      }
    }
  },
  "aggs": {
    "hat_prices": { "sum": { "field": "price" } }
  }
}
返回
{
  ...
  "aggregations": {
    "hat_prices": {
      "value": 450.0
    }
  }
}

value_count统计数量
销售数量统计
POST /sales/_search?size=0
{
  "aggs" : {
    "types_count" : { "value_count" : { "field" : "type" } }
  }
}
返回
{
  ...
  "aggregations": {
    "types_count": {
      "value": 7
    }
  }
}

weighted_avg带权重的avg
POST /exams/_search
{
  "size": 0,
  "aggs": {
    "weighted_grade": {
      "weighted_avg": {
        "value": {
          "field": "grade"
        },
        "weight": {
          "field": "weight"
        }
      }
    }
  }
}
返回
{
  ...
  "aggregations": {
    "weighted_grade": {
      "value": 70.0
    }
  }
}

cardinality 基数（distinct去重）
POST /sales/_search?size=0
{
  "aggs": {
    "type_count": {
      "cardinality": {
        "field": "type"
      }
    }
  }
}
返回
{
  ...
  "aggregations": {
    "type_count": {
      "value": 3
    }
  }
}

median_absolute_deviation：中位值
GET reviews/_search
{
  "size": 0,
  "aggs": {
    "review_average": {
      "avg": {
        "field": "rating"
      }
    },
    "review_variability": {
      "median_absolute_deviation": {
        "field": "rating" 
      }
    }
  }
}
返回
{
  ...
  "aggregations": {
    "review_average": {
      "value": 3.0
    },
    "review_variability": {
      "value": 2.0
    }
  }
}

stats包含avg,max,min,sum和count
POST /exams/_search?size=0
{
  "aggs": {
    "grades_stats": { "stats": { "field": "grade" } }
  }
}
返回
{
  ...

  "aggregations": {
    "grades_stats": {
      "count": 2,
      "min": 50.0,
      "max": 100.0,
      "avg": 75.0,
      "sum": 150.0
    }
  }
}

matrix_stats 针对矩阵模型
使用矩阵统计量来描述收入与贫困之间的关系。
GET /_search
{
  "aggs": {
    "statistics": {
      "matrix_stats": {
        "fields": [ "poverty", "income" ]
      }
    }
  }
}
返回
{
  ...
  "aggregations": {
    "statistics": {
      "doc_count": 50,
      "fields": [ {
          "name": "income",
          "count": 50,
          "mean": 51985.1,
          "variance": 7.383377037755103E7,
          "skewness": 0.5595114003506483,
          "kurtosis": 2.5692365287787124,
          "covariance": {
            "income": 7.383377037755103E7,
            "poverty": -21093.65836734694
          },
          "correlation": {
            "income": 1.0,
            "poverty": -0.8352655256272504
          }
        }, {
          "name": "poverty",
          "count": 50,
          "mean": 12.732000000000001,
          "variance": 8.637730612244896,
          "skewness": 0.4516049811903419,
          "kurtosis": 2.8615929677997767,
          "covariance": {
            "income": -21093.65836734694,
            "poverty": 8.637730612244896
          },
          "correlation": {
            "income": -0.8352655256272504,
            "poverty": 1.0
          }
        } ]
    }
  }
}


extended_stats根据从汇总文档中提取的数值计算统计信息
GET /exams/_search
{
  "size": 0,
  "aggs": {
    "grades_stats": { "extended_stats": { "field": "grade" } }
  }
}
返回
{
  ...

  "aggregations": {
    "grades_stats": {
      "count": 2,
      "min": 50.0,
      "max": 100.0,
      "avg": 75.0,
      "sum": 150.0,
      "sum_of_squares": 12500.0,
      "variance": 625.0,
      "variance_population": 625.0,
      "variance_sampling": 1250.0,
      "std_deviation": 25.0,
      "std_deviation_population": 25.0,
      "std_deviation_sampling": 35.35533905932738,
      "std_deviation_bounds": {
        "upper": 125.0,
        "lower": 25.0,
        "upper_population": 125.0,
        "lower_population": 25.0,
        "upper_sampling": 145.71067811865476,
        "lower_sampling": 4.289321881345245
      }
    }
  }
}

string_stats 针对字符串
用于计算从聚合文档中提取的字符串值的统计信息。
POST /my-index-000001/_search?size=0
{
  "aggs": {
    "message_stats": { "string_stats": { "field": "message.keyword" } }
  }
}
返回
{
  ...

  "aggregations": {
    "message_stats": {
      "count": 5,
      "min_length": 24,
      "max_length": 30,
      "avg_length": 28.8,
      "entropy": 3.94617750050791
    }
  }
}

percentiles 百分数范围
针对从聚合文档中提取的数值计算一个或多个百分位数。
GET latency/_search
{
  "size": 0,
  "aggs": {
    "load_time_outlier": {
      "percentiles": {
        "field": "load_time" 
      }
    }
  }
}
返回
{
  ...

 "aggregations": {
    "load_time_outlier": {
      "values": {
        "1.0": 5.0,
        "5.0": 25.0,
        "25.0": 165.0,
        "50.0": 445.0,
        "75.0": 725.0,
        "95.0": 945.0,
        "99.0": 985.0
      }
    }
  }
}

percentile_ranks 百分数排行
根据从汇总文档中提取的数值计算一个或多个百分位等级。
GET latency/_search
{
  "size": 0,
  "aggs": {
    "load_time_ranks": {
      "percentile_ranks": {
        "field": "load_time",   
        "values": [ 500, 600 ]
      }
    }
  }
}
返回
{
  ...

 "aggregations": {
    "load_time_ranks": {
      "values": {
        "500.0": 90.01,
        "600.0": 100.0
      }
    }
  }
}

geo_bounds
汇总展示了如何针对具有商店业务类型的所有文档计算位置字段的边界框
PUT /museums
{
  "mappings": {
    "properties": {
      "location": {
        "type": "geo_point"
      }
    }
  }
}

POST /museums/_bulk?refresh
{"index":{"_id":1}}
{"location": "52.374081,4.912350", "name": "NEMO Science Museum"}
{"index":{"_id":2}}
{"location": "52.369219,4.901618", "name": "Museum Het Rembrandthuis"}
{"index":{"_id":3}}
{"location": "52.371667,4.914722", "name": "Nederlands Scheepvaartmuseum"}
{"index":{"_id":4}}
{"location": "51.222900,4.405200", "name": "Letterenhuis"}
{"index":{"_id":5}}
{"location": "48.861111,2.336389", "name": "Musée du Louvre"}
{"index":{"_id":6}}
{"location": "48.860000,2.327000", "name": "Musée d'Orsay"}

POST /museums/_search?size=0
{
  "query": {
    "match": { "name": "musée" }
  },
  "aggs": {
    "viewport": {
      "geo_bounds": {
        "field": "location",    
        "wrap_longitude": true  
      }
    }
  }
}
返回
{
  ...
  "aggregations": {
    "viewport": {
      "bounds": {
        "top_left": {
          "lat": 48.86111099738628,
          "lon": 2.3269999679178
        },
        "bottom_right": {
          "lat": 48.85999997612089,
          "lon": 2.3363889567553997
        }
      }
    }
  }
}

================
geo_centroid
================
geo_line
================

top_hits 分桶后的top hits
POST /sales/_search?size=0
{
  "aggs": {
    "top_tags": {
      "terms": {
        "field": "type",
        "size": 3
      },
      "aggs": {
        "top_sales_hits": {
          "top_hits": {
            "sort": [
              {
                "date": {
                  "order": "desc"
                }
              }
            ],
            "_source": {
              "includes": [ "date", "price" ]
            },
            "size": 1
          }
        }
      }
    }
  }
}
返回
{
  ...
  "aggregations": {
    "top_tags": {
       "doc_count_error_upper_bound": 0,
       "sum_other_doc_count": 0,
       "buckets": [
          {
             "key": "hat",
             "doc_count": 3,
             "top_sales_hits": {
                "hits": {
                   "total" : {
                       "value": 3,
                       "relation": "eq"
                   },
                   "max_score": null,
                   "hits": [
                      {
                         "_index": "sales",
                         "_type": "_doc",
                         "_id": "AVnNBmauCQpcRyxw6ChK",
                         "_source": {
                            "date": "2015/03/01 00:00:00",
                            "price": 200
                         },
                         "sort": [
                            1425168000000
                         ],
                         "_score": null
                      }
                   ]
                }
             }
          },
          {
             "key": "t-shirt",
             "doc_count": 3,
             "top_sales_hits": {
                "hits": {
                   "total" : {
                       "value": 3,
                       "relation": "eq"
                   },
                   "max_score": null,
                   "hits": [
                      {
                         "_index": "sales",
                         "_type": "_doc",
                         "_id": "AVnNBmauCQpcRyxw6ChL",
                         "_source": {
                            "date": "2015/03/01 00:00:00",
                            "price": 175
                         },
                         "sort": [
                            1425168000000
                         ],
                         "_score": null
                      }
                   ]
                }
             }
          },
          {
             "key": "bag",
             "doc_count": 1,
             "top_sales_hits": {
                "hits": {
                   "total" : {
                       "value": 1,
                       "relation": "eq"
                   },
                   "max_score": null,
                   "hits": [
                      {
                         "_index": "sales",
                         "_type": "_doc",
                         "_id": "AVnNBmatCQpcRyxw6ChH",
                         "_source": {
                            "date": "2015/01/01 00:00:00",
                            "price": 150
                         },
                         "sort": [
                            1420070400000
                         ],
                         "_score": null
                      }
                   ]
                }
             }
          }
       ]
    }
  }
}

=================
top_metrics
POST /test/_bulk?refresh
{"index": {}}
{"s": 1, "m": 3.1415}
{"index": {}}
{"s": 2, "m": 1.0}
{"index": {}}
{"s": 3, "m": 2.71828}
POST /test/_search?filter_path=aggregations
{
  "aggs": {
    "tm": {
      "top_metrics": {
        "metrics": {"field": "m"},
        "sort": {"s": "desc"}
      }
    }
  }
}
返回
{
  "aggregations": {
    "tm": {
      "top": [ {"sort": [3], "metrics": {"m": 2.718280076980591 } } ]
    }
  }
}

-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly
更改为
-XX:+UseG1GC
-XX:MaxGCPauseMillis=50