ES mappings

原创 2017年01月03日 10:41:11

参考:https://www.elastic.co/guide/en/elasticsearch/reference/5.1/dynamic-mapping.html

Mapping 是定义document,其包含的字段,字段存储和索引,例如使用mappIng定义:

  • 哪个string field应 full text field,即分词。
  • 哪些字段包含number,date, geolocation
  • 是否需要所有字段索引到_all字段
  • 日期字段的format
  • 自定义规则,控制动态添加的字段。自动发现新添加字段的类型,并加到mapping中。
    • _default_ mapping
    • Dynamic field mappings
    • Dynamic templates

_default_

PUT my_index
{
  "mappings": {
    "_default_": { //方式1: 
      "_all": {
        "enabled": false
      }
    },
    "user": {}, 
    "blogpost": { 
      "_all": {
        "enabled": true
      }
    }
  }
}

Dynamic field mapping

设置级别:document 和 object。
参数类型:
dynamic=true|false|strict
true,默认,新发现的字段,添加到mapping中。
false,新发现的字段会存储,但不会添加到mapping中
strict,若发现新的字段,则抛异常添加document失败

PUT my_index
{
  "mappings": {
    "my_type": {
      "dynamic": false, //type级别默认设置
      "properties": {
        "user": { 
          "properties": {
            "social_networks": { 
              "dynamic": true,//某些Object字段,可覆盖上层设置

PUT demo
{
   "mappings": {
      "_default_": {//索引级别,所有type的默认设置
         "dynamic": "strict"
      },
      "mytype": {
         "properties": {
            "name": {
               "type": "string"
            }
         }
      }
   }
}

Dynamic templates

https://www.elastic.co/guide/en/elasticsearch/reference/2.1/dynamic-templates.html

PUT my_index
{
   "mappings": {
      "my_type": {
         "dynamic_templates": [
            {
               "integers": {
                  "match_mapping_type": "long",
                  "mapping": {
                     "type": "integer"
                  }
               }
            },
            {
               "longs_as_strings": {
                  "match_mapping_type": "string",
                  "match": "long_*",
                  "unmatch": "*_text",
                  "mapping": {
                     "type": "long"
                  }
               }
            },
            {
               "strings": {
                  "match_mapping_type": "string",
                  "mapping": {
                     "type": "string",
                     "fields": {
                        "raw": {
                           "type": "string",
                           "index": "not_analyzed",
                           "ignore_above": 256
                        }
                     }
                  }
               }
            }
         ]
      }
   }
}

{name} and {dynamic_type} 占位符:
replaced in the mapping with the field name and detected dynamic type.

PUT my_index
{
  "mappings": {
    "my_type": {
      "dynamic_templates": [
        {
          "named_analyzers": {
            "match_mapping_type": "string",
            "match": "*",
            "mapping": {
              "type": "string",
              "analyzer": "{name}"//如字段名称为english
            }
          }
        },
        {
          "no_doc_values": {
            "match_mapping_type":"*",
            "mapping": {
              "type": "{dynamic_type}",//非string字段,禁用doc_values
                  "doc_values": false
            }
          }
        }
      ]
    }
  }
}

Index templates

Allow you to configure the default mappings, settings and aliases for new indices, whether created automatically or explicitly.

禁止自动type创建

PUT data/_settings//特定索引禁止
{
  "index.mapper.dynamic":false 
}
PUT _template/template_all //通过模板禁止
{
  "template": "*",
  "order":0,//lower order being applied first, and higher orders overriding them.
  "settings": {
    "index.mapper.dynamic": false 
  }
}

Override default template

PUT _template/disable_all_field
{
  "order": 0,
  "template": "*", //匹配所有索引
  "mappings": {
    "_default_": { //匹配所有type
      "_all": { //禁用_all
        "enabled": false
      }
    }
  }
}

Indices APIs

Put Mapping

为新的索引添加新的type,或为已有的type添加新的字段。

PUT twitter //创建新的index、type
{
  "mappings": {
    "tweet": {
      "properties": {
        "message": {
          "type": "string"
        }
      }
    }
  }
}

PUT twitter/_mapping/user //已有index,添加新的type
{
  "properties": {
    "name": {
      "type": "string"
    }
  }
}

PUT twitter/_mapping/tweet //已有index、type,添加新的字段
{
  "properties": {
    "user_name": {
      "type": "string"
    }
  }
}

Multi-index操作

PUT /{index}/_mapping/{type}
{ body }

{index} 接受 多索引名字(逗号分隔)匹配符

Mapping parameters » fields

index the same field in different ways for different purposes(multi-fields).

PUT /my_index
{
  "mappings": {
    "my_type": {
      "properties": {
        "city": {
          "type": "string",
          "fields": {//fields:multi-fields
            "raw": { //引用:city.raw
              "type":  "string",
              "index": "not_analyzed"
            }
          }
        }
      }
    }
  }
}

The fields setting is allowed to have different settings for fields of the same name in the same index.

注意:Multi-fields do not change the original _source field.

Updating field mappings

一般不允许更新字段的mapping。一些例外情况:

  • 新properties可添加到Object datatype字段
  • 新multi-fields可添加到已有字段
  • doc_values可disabled,但不可以disabled
  • ignore_above 可更新
PUT my_index 
{
  "mappings": {
    "user": {
      "properties": {
        "name": {//字段1,name字段为Object类型
          "properties": {
            "first": {//属性1,可动态添加新的属性
              "type": "string"
            }
          }
        },
        "user_id": {//字段2:string类型
          "type": "string",
          "index": "not_analyzed"
        }
      }
    }
  }
}
PUT my_index/_mapping/user
{
  "properties": {
    "name": {
      "properties": {
        "last": { //Object类型字段,添加新的属性
          "type": "string"
        }
      }
    },
    "user_id": {
      "type": "string",
      "index": "not_analyzed",
      "ignore_above": 100 //可更新
    }
  }
}

Conflicts between fields in different types

字段冲突 : “Merge failed with failures…”.
同一index内,同一name的字段,在不同type的字段必须有相同的类型。

PUT my_index/_mapping/type_one?update_all_types 
{
  "properties": {
    "text": {
      "type": "string",
      "analyzer": "standard",
      "search_analyzer": "whitespace"
    }
  }
}

update_all_types 解决Merge failed with failures..

个别情况,相同名称的字段,在同一index、不同type允许不同mapping的情况:
copy_to, dynamic, enabled, ignore_above, include_in_all, and properties parameters.

copy_to
创建自定义_all字段,the values of multiple fields can be copied into a group field, which can then be queried as a single field.

PUT /my_index
{
  "mappings": {
    "my_type": {
      "properties": {
        "first_name": {
          "type": "string",
          "copy_to": "full_name" 
        },
        "last_name": {
          "type": "string",
          "copy_to": "full_name" 
        },
        "full_name": {//类似自定义的_all字段,_source中没有该字段,但可用于查询
          "type": "string"
        }
      }
    }
  }
}
GET /my_index/_search
{
  "query": {
    "match": {
      "full_name": { //从自定义_all字段 full——name 查询
        "query": "John Smith",
        "operator": "and"
      }
    }
  }
}

注意:

  • It is the field value which is copied, not the terms (which result from the analysis process).
  • _source field will not be modified to show the copied values.
  • The same value can be copied to multiple fields, with “copy_to”: [ “field_1”, “field_2” ]

dynamic
true|false|strict
enabled
just store without indexing.

Can be applied only to the mapping type and to object fields, causes Elasticsearch to skip parsing of the contents of the field entirely. The JSON can still be retrieved from the _source field, but it is not searchable or stored in any other way.

not JSON objects, will also ignore 
PUT my_index
{
  "mappings": {
    "session": {
      "properties": {
        "user_id": {
          "type":  "string",
          "index": "not_analyzed"
        },
        "last_updated": {
          "type": "date"
        },
        "session_data": { 
          "enabled": false//not searchable 
        }
      }
    }
  }
}

整个type禁用

PUT my_index
{
  "mappings": {
    "session": { //session type禁用
      "enabled": false
    }
  }
}

the document is stored in the _source field, which means it can be retrieved, but none of its contents are indexed in any way。

Checking the mapping reveals that no fields have been added.

ignore_above

PUT my_index
{
  "mappings": {
    "my_type": {
      "properties": {
        "message": {
          "type": "string",
          "index": "not_analyzed",//主要用于not_analyzed字段,排序,过滤、聚合场景,太长的没意义
          "ignore_above": 20 //限制特定字段最大值,设置的是字符数,但lucene是字节。若使用UTF-8文本,可能需要设置32766/3 = 10922 
        }
     ...

include_in_all
control over which fields are included in the _all field.

PUT my_index
{
  "mappings": {
    "my_type": {
      "properties": {
        "date": { 
          "type": "date",
          "include_in_all": false //该字段不包含在_all中
        }
      }
    }
  }
}

properties
Type mappings, object fields and nested fields contain sub-fields, called properties.
两者都可以有不同的类型

PUT my_index
{
  "mappings": {
    "my_type": { 
      "properties": {
        "manager": { //Object 字段
          "properties": {
            "age":  { "type": "integer" },
            "name": { "type": "string"  }
          }
        },
        "employees": { //嵌套 字段
          "type": "nested",
          "properties": {
            "age":  { "type": "long" },//同index、type不同properties,可有不同mapping
            "name": { "type": "string"  }
          }
        }
      }
    }
  }
}

PUT my_index/my_type/1 
{
  "region": "US",
  "manager": {//Object
    "name": "Alice White",
    "age": 30
  },
  "employees": [//嵌套
    {
      "name": "John Smith",
      "age": 34
    },
    {
      "name": "Peter Brown",
      "age": 26
    }
  ]
}

ES mapping最佳实践

Level 1 :在特定type配置动态字段

  "mappings": {
    "my_type": {
      "dynamic": false, //type级别默认设置

Level 2 :在特定索引配置,默认type的动态字段

PUT demo
{
   "mappings": {
      "_default_": {
         "dynamic": "strict"
      }

Level 3:特定object字段,配置动态字段

elasticsearch的mapping设置

PUT /my_index { "mappings": { "my_type": { "properties": { "...
  • u012307002
  • u012307002
  • 2016年04月29日 14:10
  • 1173

elasticsearch中的mapping简介

最近项目准备用到elasticsearch, 首先需要搞清楚elasticsearch的一些概念,在网上发现这篇文章不错,以通俗易懂的语言讲明白了mapping的概念。 默认mapping elas...
  • lvhong84
  • lvhong84
  • 2014年04月17日 16:46
  • 56090

ES将删除映射类型(Removal of mapping type)

重要: 在Elasticsearch6.0.0或更新的版本中创建的索引可能只包含一个单一的映射类型。 在Elasticsearch5.x中创建的多映射类型的索引在Elasticsearch6.x中...
  • tianzeyu1992
  • tianzeyu1992
  • 2017年11月23日 10:22
  • 345

elasticsearch 2.3.4 java API 连接,ik分词器,设置集群节点,创建index,mapping的几种方式

1、默认集群连接 Client client = TransportClient.builder().build() .addTransportAddress(new Ine...
  • u013378306
  • u013378306
  • 2016年07月16日 13:04
  • 4477

elastic客户端开发

es客户端开发 1.将%ES_HOME%/lib目录下的jar包加入到项目的CLASSPATH中 2.代码实例 package org.test.es; import java.io.I...
  • dingzfang
  • dingzfang
  • 2015年01月16日 14:09
  • 862

Elasticsearch之_default_—— 为索引添加默认映射

前篇说过,ES可以自动为文档设定索引。但是问题也来了——如果默认设置的索引不是我们想要的,该怎么办呢? 要知道ES这种搜索引擎都是以Index为实际的分区,Index里面包含了不同的类型,不同的...
  • shuyun123456789
  • shuyun123456789
  • 2016年12月30日 12:16
  • 1894

ElasticSearch(六)--映射和分析

映射mapping机制用于进行字段类型确认,将每个字段匹配为一种确定的数据类型(string, boolean, date); 分析analysis机制用于进行全文文本Full Text的分词,以建...
  • WuyZhen_CSDN
  • WuyZhen_CSDN
  • 2016年05月22日 22:55
  • 11958

ES02--索引mapping的写入、查看与修改

mapping的写入与查看 首先创建一个索引: curl -XPOST "http://127.0.0.1:9200/productindex" {"acknowledged":true} ...
  • envinfo2012
  • envinfo2012
  • 2017年06月17日 17:34
  • 273

elasticsearch 学习博客系列<二> ES 中 index 设置 Mapping(表结构)

通过上一篇 我们了解到ES中的一些概念,回顾一下: 几个基本名词 index: es里的index相当于一个数据库。  type: 相当于数据库里的一个表。  id: 唯一,相当于主键。  n...
  • qq_29631809
  • qq_29631809
  • 2017年06月01日 22:40
  • 1795

Elasticsearch学习笔记——Mapping映射

Mapping简述 Elasticsearch是一个schema-less的系统,但并不代表no shema,而是会尽量根据JSON源数据的基础类型猜测你想要的字段类型映射。 Elasti...
  • jiaminbao
  • jiaminbao
  • 2017年11月28日 16:11
  • 144
内容举报
返回顶部
收藏助手
不良信息举报
您举报文章:ES mappings
举报原因:
原因补充:

(最多只允许输入30个字)