Cannot load index to elasticsearch from external file, using logstash

问题

i am running one instance of elastic and one of logstash in parallel on the same computer.

when trying to load a file into elastic, using logstash that is running the config file below, i get the follwing output msgs on elastic and no file is loaded (when input is configured to be stdin everything seems to be working just fine)

any ideas?

" [2014-06-17 22:42:24,748][INFO ][cluster.service ] [Masked Marvel] removed {[logstash- Eitan-PC-5928-2010][Ql5fyvEGQyO96R9NIeP32g][Eitan-PC][inet[Eitan-PC/10.0.0.5:9301]]{client=true, data=false},}, reason: zen-disco-node_failed([logstash-Eitan-PC-5928-2010][Ql5fyvEGQyO96R9NIeP32g][Eitan-PC][inet[Eitan-PC/10.0.0.5:9301]]{client=true, data=false}), reason transport disconnected (with verified connect)

[2014-06-17 22:43:00,686][INFO ][cluster.service ] [Masked Marvel] added {[logstash-Eitan-PC-5292-4014][m0Tg-fcmTHW9aP6zHeUqTA][Eitan-PC][inet[/10.0.0.5:9301]]{client=true, data=false},}, reason: zen-disco-receive(join from node[[logstash-Eitan-PC-5292-4014][m0Tg-fcmTHW9aP6zHeUqTA][Eitan-PC][inet[/10.0.0.5:9301]]{client=true, data=false}]) "

config file:

    input { 
            file {
                path => "c:\testLog.txt"
            }
        } 


    output {
        elasticsearch { host => localhost  
                index=> amat1
                 }

}

回答1:

When you use "elasticsearch" as your output http://logstash.net/docs/1.4.1/outputs/elasticsearch as opposed to "elasticsearch_http" http://logstash.net/docs/1.4.1/outputs/elasticsearch_http you are going to want to set "protocol".

The reason is that it can have 3 different values, "node", "http" or "transport" with different behavior for each and the default selection is not well documented.

From the look of your log files it appears it's trying to use "node" protocol as I see connection attempts on port 9301 which indicates (along with other log entries) that logstash is trying to join the cluster as a node. This can fail for any number of reasons including mismatch on the cluster name.

I'd suggest setting protocol to "http" - that change has fixed similar issues before.

See also:

http://logstash.net/docs/1.4.1/outputs/elasticsearch#cluster http://logstash.net/docs/1.4.1/outputs/elasticsearch#protocol

EDIT:

A few other issues I see in your config -

• Your host and index should be strings, which in a logstash config file should be wrapped with double quotes, "localhost" and "amat1". No quotes may work but they recommend you use quotes.

  http://logstash.net/docs/1.4.1/configuration#string

• If you don't use "http" as the protocol or don't use "elasticsearch_http" as the output you should set cluster equal to your ES cluster name (as it will be trying to become a node of the cluster).

• You should set start_position under file in input to "beginning". Otherwise it will default to reading from the end of the file and you won't see any data. This a particular problem with Windows right now as the other way of tracking position within a file, sincedb, is broken on Windows:

  https://logstash.jira.com/browse/LOGSTASH-1587

  http://logstash.net/docs/1.4.1/inputs/file#start_position

• You should change your path to your log file to this: "C:/testLog.txt". Logstash prefers forward slashes and upper case drive letters under Windows.

  https://logstash.jira.com/browse/LOGSTASH-430

来源：https://stackoverflow.com/questions/24272915/cannot-load-index-to-elasticsearch-from-external-file-using-logstash