How to make elasticsearch add the timestamp field to every document in all indices?

问题

Elasticsearch experts,

I have been unable to find a simple way to just tell ElasticSearch to insert the _timestamp field for all the documents that are added in all the indices (and all document types).

I see an example for specific types: http://www.elasticsearch.org/guide/reference/mapping/timestamp-field/

and also see an example for all indices for a specific type (using _all): http://www.elasticsearch.org/guide/reference/api/admin-indices-put-mapping/

but I am unable to find any documentation on adding it by default for all documents that get added irrespective of the index and type.

回答1:

You can do this by providing it when creating your index.

$curl -XPOST localhost:9200/test -d '{
"settings" : {
    "number_of_shards" : 1
},
"mappings" : {
    "_default_":{
        "_timestamp" : {
            "enabled" : true,
            "store" : true
        }
    }
  }
}'

That will then automatically create a _timestamp for all stuff that you put in the index. Then after indexing something when requesting the _timestamp field it will be returned.

回答2:

Elasticsearch used to support automatically adding timestamps to documents being indexed, but deprecated this feature in 2.0.0

From the version 5.5 documentation:

The _timestamp and _ttl fields were deprecated and are now removed. As a replacement for _timestamp, you should populate a regular date field with the current timestamp on application side.

回答3:

Adding another way to get indexing timestamp. Hope this may help someone.

Ingest pipeline can be used to add timestamp when document is indexed. Here, is a sample example:

PUT _ingest/pipeline/indexed_at
{
  "description": "Adds indexed_at timestamp to documents",
  "processors": [
    {
      "set": {
        "field": "_source.indexed_at",
        "value": "{{_ingest.timestamp}}"
      }
    }
  ]
}

Earlier, elastic search was using named-pipelines because of which 'pipeline' param needs to be specified in the elastic search endpoint which is used to write/index documents. (Ref: link) This was bit troublesome as you would need to make changes in endpoints on application side.

With Elastic search version >= 6.5, you can now specify a default pipeline for an index using index.default_pipeline settings. (Refer link for details)

Here is the to set default pipeline:

PUT ms-test/_settings
{
  "index.default_pipeline": "indexed_at"
}

I haven't tried out yet, as didn't upgraded to ES 6.5, but above command should work.

回答4:

first create index and properties of the index , such as field and datatype and then insert the data using the rest API.

below is the way to create index with the field properties.execute the following in kibana console

`PUT /vfq-jenkins
{
"mappings": {
"properties": {
"BUILD_NUMBER": { "type" : "double"},
"BUILD_ID" : { "type" : "double" },
"JOB_NAME" : { "type" : "text" },
"JOB_STATUS" : { "type" : "keyword" },
"time" : { "type" : "date" }
 }}}`

the next step is to insert the data into that index: curl -u elastic:changeme -X POST http://elasticsearch:9200/vfq-jenkins/_doc/?pretty -H Content-Type: application/json -d '{ "BUILD_NUMBER":"83","BUILD_ID":"83","JOB_NAME":"OMS_LOG_ANA","JOB_STATUS":"SUCCESS" , "time" : "2019-09-08'T'12:39:00" }'

来源：https://stackoverflow.com/questions/17136138/how-to-make-elasticsearch-add-the-timestamp-field-to-every-document-in-all-indic

标签

mapping

timestamp

ElasticSearch