ES mappings

原创 2017年01月03日 10:41:11

参考:https://www.elastic.co/guide/en/elasticsearch/reference/5.1/dynamic-mapping.html

Mapping 是定义document,其包含的字段,字段存储和索引,例如使用mappIng定义:

  • 哪个string field应 full text field,即分词。
  • 哪些字段包含number,date, geolocation
  • 是否需要所有字段索引到_all字段
  • 日期字段的format
  • 自定义规则,控制动态添加的字段。自动发现新添加字段的类型,并加到mapping中。
    • _default_ mapping
    • Dynamic field mappings
    • Dynamic templates

_default_

PUT my_index
{
  "mappings": {
    "_default_": { //方式1: 
      "_all": {
        "enabled": false
      }
    },
    "user": {}, 
    "blogpost": { 
      "_all": {
        "enabled": true
      }
    }
  }
}

Dynamic field mapping

设置级别:document 和 object。
参数类型:
dynamic=true|false|strict
true,默认,新发现的字段,添加到mapping中。
false,新发现的字段会存储,但不会添加到mapping中
strict,若发现新的字段,则抛异常添加document失败

PUT my_index
{
  "mappings": {
    "my_type": {
      "dynamic": false, //type级别默认设置
      "properties": {
        "user": { 
          "properties": {
            "social_networks": { 
              "dynamic": true,//某些Object字段,可覆盖上层设置

PUT demo
{
   "mappings": {
      "_default_": {//索引级别,所有type的默认设置
         "dynamic": "strict"
      },
      "mytype": {
         "properties": {
            "name": {
               "type": "string"
            }
         }
      }
   }
}

Dynamic templates

https://www.elastic.co/guide/en/elasticsearch/reference/2.1/dynamic-templates.html

PUT my_index
{
   "mappings": {
      "my_type": {
         "dynamic_templates": [
            {
               "integers": {
                  "match_mapping_type": "long",
                  "mapping": {
                     "type": "integer"
                  }
               }
            },
            {
               "longs_as_strings": {
                  "match_mapping_type": "string",
                  "match": "long_*",
                  "unmatch": "*_text",
                  "mapping": {
                     "type": "long"
                  }
               }
            },
            {
               "strings": {
                  "match_mapping_type": "string",
                  "mapping": {
                     "type": "string",
                     "fields": {
                        "raw": {
                           "type": "string",
                           "index": "not_analyzed",
                           "ignore_above": 256
                        }
                     }
                  }
               }
            }
         ]
      }
   }
}

{name} and {dynamic_type} 占位符:
replaced in the mapping with the field name and detected dynamic type.

PUT my_index
{
  "mappings": {
    "my_type": {
      "dynamic_templates": [
        {
          "named_analyzers": {
            "match_mapping_type": "string",
            "match": "*",
            "mapping": {
              "type": "string",
              "analyzer": "{name}"//如字段名称为english
            }
          }
        },
        {
          "no_doc_values": {
            "match_mapping_type":"*",
            "mapping": {
              "type": "{dynamic_type}",//非string字段,禁用doc_values
                  "doc_values": false
            }
          }
        }
      ]
    }
  }
}

Index templates

Allow you to configure the default mappings, settings and aliases for new indices, whether created automatically or explicitly.

禁止自动type创建

PUT data/_settings//特定索引禁止
{
  "index.mapper.dynamic":false 
}
PUT _template/template_all //通过模板禁止
{
  "template": "*",
  "order":0,//lower order being applied first, and higher orders overriding them.
  "settings": {
    "index.mapper.dynamic": false 
  }
}

Override default template

PUT _template/disable_all_field
{
  "order": 0,
  "template": "*", //匹配所有索引
  "mappings": {
    "_default_": { //匹配所有type
      "_all": { //禁用_all
        "enabled": false
      }
    }
  }
}

Indices APIs

Put Mapping

为新的索引添加新的type,或为已有的type添加新的字段。

PUT twitter //创建新的index、type
{
  "mappings": {
    "tweet": {
      "properties": {
        "message": {
          "type": "string"
        }
      }
    }
  }
}

PUT twitter/_mapping/user //已有index,添加新的type
{
  "properties": {
    "name": {
      "type": "string"
    }
  }
}

PUT twitter/_mapping/tweet //已有index、type,添加新的字段
{
  "properties": {
    "user_name": {
      "type": "string"
    }
  }
}

Multi-index操作

PUT /{index}/_mapping/{type}
{ body }

{index} 接受 多索引名字(逗号分隔)匹配符

Mapping parameters » fields

index the same field in different ways for different purposes(multi-fields).

PUT /my_index
{
  "mappings": {
    "my_type": {
      "properties": {
        "city": {
          "type": "string",
          "fields": {//fields:multi-fields
            "raw": { //引用:city.raw
              "type":  "string",
              "index": "not_analyzed"
            }
          }
        }
      }
    }
  }
}

The fields setting is allowed to have different settings for fields of the same name in the same index.

注意:Multi-fields do not change the original _source field.

Updating field mappings

一般不允许更新字段的mapping。一些例外情况:

  • 新properties可添加到Object datatype字段
  • 新multi-fields可添加到已有字段
  • doc_values可disabled,但不可以disabled
  • ignore_above 可更新
PUT my_index 
{
  "mappings": {
    "user": {
      "properties": {
        "name": {//字段1,name字段为Object类型
          "properties": {
            "first": {//属性1,可动态添加新的属性
              "type": "string"
            }
          }
        },
        "user_id": {//字段2:string类型
          "type": "string",
          "index": "not_analyzed"
        }
      }
    }
  }
}
PUT my_index/_mapping/user
{
  "properties": {
    "name": {
      "properties": {
        "last": { //Object类型字段,添加新的属性
          "type": "string"
        }
      }
    },
    "user_id": {
      "type": "string",
      "index": "not_analyzed",
      "ignore_above": 100 //可更新
    }
  }
}

Conflicts between fields in different types

字段冲突 : “Merge failed with failures…”.
同一index内,同一name的字段,在不同type的字段必须有相同的类型。

PUT my_index/_mapping/type_one?update_all_types 
{
  "properties": {
    "text": {
      "type": "string",
      "analyzer": "standard",
      "search_analyzer": "whitespace"
    }
  }
}

update_all_types 解决Merge failed with failures..

个别情况,相同名称的字段,在同一index、不同type允许不同mapping的情况:
copy_to, dynamic, enabled, ignore_above, include_in_all, and properties parameters.

copy_to
创建自定义_all字段,the values of multiple fields can be copied into a group field, which can then be queried as a single field.

PUT /my_index
{
  "mappings": {
    "my_type": {
      "properties": {
        "first_name": {
          "type": "string",
          "copy_to": "full_name" 
        },
        "last_name": {
          "type": "string",
          "copy_to": "full_name" 
        },
        "full_name": {//类似自定义的_all字段,_source中没有该字段,但可用于查询
          "type": "string"
        }
      }
    }
  }
}
GET /my_index/_search
{
  "query": {
    "match": {
      "full_name": { //从自定义_all字段 full——name 查询
        "query": "John Smith",
        "operator": "and"
      }
    }
  }
}

注意:

  • It is the field value which is copied, not the terms (which result from the analysis process).
  • _source field will not be modified to show the copied values.
  • The same value can be copied to multiple fields, with “copy_to”: [ “field_1”, “field_2” ]

dynamic
true|false|strict
enabled
just store without indexing.

Can be applied only to the mapping type and to object fields, causes Elasticsearch to skip parsing of the contents of the field entirely. The JSON can still be retrieved from the _source field, but it is not searchable or stored in any other way.

not JSON objects, will also ignore 
PUT my_index
{
  "mappings": {
    "session": {
      "properties": {
        "user_id": {
          "type":  "string",
          "index": "not_analyzed"
        },
        "last_updated": {
          "type": "date"
        },
        "session_data": { 
          "enabled": false//not searchable 
        }
      }
    }
  }
}

整个type禁用

PUT my_index
{
  "mappings": {
    "session": { //session type禁用
      "enabled": false
    }
  }
}

the document is stored in the _source field, which means it can be retrieved, but none of its contents are indexed in any way。

Checking the mapping reveals that no fields have been added.

ignore_above

PUT my_index
{
  "mappings": {
    "my_type": {
      "properties": {
        "message": {
          "type": "string",
          "index": "not_analyzed",//主要用于not_analyzed字段,排序,过滤、聚合场景,太长的没意义
          "ignore_above": 20 //限制特定字段最大值,设置的是字符数,但lucene是字节。若使用UTF-8文本,可能需要设置32766/3 = 10922 
        }
     ...

include_in_all
control over which fields are included in the _all field.

PUT my_index
{
  "mappings": {
    "my_type": {
      "properties": {
        "date": { 
          "type": "date",
          "include_in_all": false //该字段不包含在_all中
        }
      }
    }
  }
}

properties
Type mappings, object fields and nested fields contain sub-fields, called properties.
两者都可以有不同的类型

PUT my_index
{
  "mappings": {
    "my_type": { 
      "properties": {
        "manager": { //Object 字段
          "properties": {
            "age":  { "type": "integer" },
            "name": { "type": "string"  }
          }
        },
        "employees": { //嵌套 字段
          "type": "nested",
          "properties": {
            "age":  { "type": "long" },//同index、type不同properties,可有不同mapping
            "name": { "type": "string"  }
          }
        }
      }
    }
  }
}

PUT my_index/my_type/1 
{
  "region": "US",
  "manager": {//Object
    "name": "Alice White",
    "age": 30
  },
  "employees": [//嵌套
    {
      "name": "John Smith",
      "age": 34
    },
    {
      "name": "Peter Brown",
      "age": 26
    }
  ]
}

ES mapping最佳实践

Level 1 :在特定type配置动态字段

  "mappings": {
    "my_type": {
      "dynamic": false, //type级别默认设置

Level 2 :在特定索引配置,默认type的动态字段

PUT demo
{
   "mappings": {
      "_default_": {
         "dynamic": "strict"
      }

Level 3:特定object字段,配置动态字段

版权声明:本文为博主原创文章,未经博主允许不得转载。 举报

相关文章推荐

elasticsearch中文分词

由于elasticsearch基于lucene,所以天然地就多了许多lucene上的中文分词的支持,比如 IK, Paoding, MMSEG4J等lucene中文分词原理上都能在elasticsea...

elasticsearch 学习博客系列<二> ES 中 index 设置 Mapping(表结构)

通过上一篇 我们了解到ES中的一些概念,回顾一下: 几个基本名词 index: es里的index相当于一个数据库。  type: 相当于数据库里的一个表。  id: 唯一,相当于主键。  n...

我是如何成为一名python大咖的?

人生苦短,都说必须python,那么我分享下我是如何从小白成为Python资深开发者的吧。2014年我大学刚毕业..

elasticsearch添加mapping

//EsMapping.java public class EsMapping { private static Map> INDEX_TYPE_MAPPING = new HashMap>()...

JPA Advanced Mappings(映射)

JPA Advanced Mappings(映射)JPA是一个使用java规范发布的库。因此,它支持所有面向对象的实体持久性概念。原文链接:http://blogxinxiucan.sh1.newto...

[ElasticSearch] mappings and type

http://www.cnblogs.com/buzzlight/p/elasticsearch_mapping_fields.htmlrequire 'elasticsearch'$client =...

Hibernate Many-to-Many Mappings

创建表: create table EMPLOYEE ( id INT NOT NULL auto_increment, first_name VARCHAR(20) default N...

JPA Advanced Mappings(映射)

JPA Advanced Mappings(映射) JPA是一个使用java规范发布的库。因此,它支持所有面向对象的实体持久性概念。 继承是面向对象语言的核心概念,因此我们可以在实...

8、Spring MVC 之 Handler mappings

在Spring之前的版本中,用户必须定义一个或者多个HandlerMapping bean在web应用容器中用来映射web请求到合适的handlers。自从引进了注解控制器,你通常不再需要这样做了.因...

Elasticsearch笔记七之setting,mapping,分片查询方式

es中setting,mapping,分片查询方式
  • ty4315
  • ty4315
  • 2016-09-08 23:32
  • 5454

Remedy change 自定义Approval Mappings

需求是这样的: 假设现在Change Coordinator = 'Test Group' 的需要根据条件Gender走不同level 1的审批,最后走相同Group的Level2流程 我们设想的...
返回顶部
收藏助手
不良信息举报
您举报文章:深度学习:神经网络中的前向传播和反向传播算法推导
举报原因:
原因补充:

(最多只允许输入30个字)