ElasticSearch

How to index (ingest) geo data (Geometry, GeometryCollection) as GeoShape in ElasticSearch with C#, Nest, NetTopologySuite from GeoJson file / string?

久未见 提交于 2021-01-29 15:41:36
问题 Summary I want to to properly index (ingest) geo data (Geometry, GeometryCollection) as GeoShape in ElasticSearch using C#, Nest and NetTopologySuite (NTS) from GeoJson files or string representations. I'm using the following stack: ElasticSearch 7.10.1 NEST 7.10.1 NetTopologySuite 2.1.0 NetTopologySuite.IO.GeoJSON 2.0.4 In my GitHub GIST you can find the two sample files (postal-area.geojson and the geojson file as a sample for Scenario #7) along with the code presented bellow with what i've

How to mount my ES data folder to freed filesystem in Mac OS

感情迁移 提交于 2021-01-29 15:09:28
问题 I am running ES locally and have defined path.data in my home dir aka /Users/prerna/es/data but all the time ES complains that high-disk watermark exceeded as shown in below message 2020-09-16 18:16:07,390][INFO ][cluster.routing.allocation.decider] [Karkas] high disk watermark exceeded on one or more nodes, rerouting shards [2020-09-16 18:16:37,393][WARN ][cluster.routing.allocation.decider] [Karkas] high disk watermark [0b] exceeded on [trwvlC3dRUOPYp0QQ9l9GQ][Karkas] free: -1b[100%],

AWS ElasticSearch domain creation permission

为君一笑 提交于 2021-01-29 14:33:38
问题 I tried to give permission required to create an ES domain in AWS to an IAM account but I couldn't find any existing policy to attach except AdministratorAccess. Is there a custom policy or something I can assign to the IAM account? I searched the web but couldn't find anything helpful either. 回答1: Amazon ES supports three types of access policies: Resource-based Policies Identity-based Policies IP-based Policies To get detailed information, about how to create your own custom policies you

ElasticSearch filter by nested boolean type fields

和自甴很熟 提交于 2021-01-29 14:22:51
问题 I need to query on multiple nested fields on boolean types. Structure of mapping: "mappings" : { "properties" : { "leaders" : { "type" : "nested", "properties" : { "except_1" : { "type" : "boolean" }, "except_2" : { "type" : "boolean" }, "counter" : { "type" : "integer" } } } } } I am trying to use query both except1 and except2 only to False . Below my try, unfortunately it returns True and False for both fields and I cannot fix it. "query": { "nested": { "path": "leaders", "query": { "bool"

Consistent search on two indexes - elasticsearch

前提是你 提交于 2021-01-29 14:10:55
问题 I have a product index and a store index. The product index includes documents that include the title and description fields of a product which are queried during search. The store index includes the id, price, and quantity of the product, in addition to the store locations. I want to search for a product, get the documents, then retrieve all the stores that include all the resulted product ids and aggregate on them. I have tried multiple options, but was not satisfied with the results: Joins

Logstash: configuring aggregate + elapsed filters

北战南征 提交于 2021-01-29 14:02:07
问题 I have these logs: "03.08.2020 10:56:38","Event LClick","Type Menu","t=0","beg" "03.08.2020 10:56:38","Event LClick","Type Menu","Detail SomeDetail","t=109","end" "03.08.2020 10:56:40","Event LClick","t=1981","beg" "03.08.2020 10:56:40","Event LClick","t=2090","end" "03.08.2020 10:56:41","Event LClick","Type ToolBar","t=3026","beg" "03.08.2020 10:56:43","Event LClick","Type ToolBar","Detail User_Desktop","t=4477","end" "03.08.2020 10:56:44","Event FormActivate","Name Form_Name:IsaA","t=5444"

Using nested values in script_score

北战南征 提交于 2021-01-29 13:50:00
问题 I am attempting to use nested values in a script score, but I am having issues making it work, because I am unable to iterate over the field by accessing it through doc . Also, when I try to query it in Kibana like _type:images AND _exists_:colors , it will not match any documents, even though the field is clearly present in all my docs when I view them individually. I am however able to access it using params._source , but I have read that it can be slow slow and is not really recommended. I

Document count is same but index size is growing every logstash run

回眸只為那壹抹淺笑 提交于 2021-01-29 13:36:37
问题 I'm sending elasticsearch using the logstash of the data contained in the mysql database. but each time logstash runs, the number of documents remains the same, but the index size increases. first run count: 333 | size in bytes : 206kb now count:333 | size in bytes : 1.6MB input { jdbc { jdbc_connection_string => "jdbc:mysql://***rds.amazonaws.com:3306/" jdbc_user => "***" jdbc_password => "***" jdbc_driver_library => "***\mysql-connector-java-5.1.46/mysql-connector-java-5.1.46-bin.jar" jdbc

Is it possible Logstash push same content from log file to ElasticSearch

帅比萌擦擦* 提交于 2021-01-29 13:30:54
问题 The logstash config sets log files as input source and then sends the content to ElasticSearch . The input part is like below input{ file{ path => "/data/logs/backend.log*" start_position => "beginning" } } Then the log file will be rolling by size, which means at first the log file name is backend.log , when the file reaches size 10M, then it is renamed to backend.log.1 , and a new empty backend.log is created to log content. So the question is whether logstash will send the content from

How to use “OR” in Dev Tool Query

血红的双手。 提交于 2021-01-29 12:51:52
问题 Hi Bellow Search provides me Log where it has both "value": "HB" and "value": "1234567" as, I am using Term, however, What I am looking for this if this match ("value": "HB" OR "value": "TR" ) AND "value": "1234567" but not understanding how to do in below, Can anyone please help me GET _search { "query": { "bool": { "must": [ { "match": {"log.file.path":"mylog.log" } } { "term": { "GPS-LOG.COMMAND": { "value": "HB" } } }, { "term": { "GPS-LOG.IMEI": { "value": "1234567" } } } ], "filter": {