Read log file from a remote machine with file input plugin using logstash

问题

Presently I have my logs and logstash running on the same machine, so I read my logs placed on my local machine with this config(using pull model)

input {     
    file {
        path => "/home/Desktop/Logstash-Input/**/*_log"
        start_position => "beginning"
    }
}

Now, we have logstash running on a different machine and want to read the logs remote mechine.

Is there a way to set the ip in file input of config file?

EDIT: I manage to do this with logstash-forwarder which is a push model(log shipper/logstash-forwarder will ship log to logstash index server) but still i am looking for a pull model without shipper, where logstash index server will go and contact directly to remote host.

回答1:

Take a look to FileBeat: https://www.elastic.co/products/beats/filebeat

It´s not a pull model but it seems a better choice than logstash-forwarder.

It monitors log files and forwards them to Logstash or Elasticsearh. It keeps also the state of log files and guarantees that events will be delivered at least one time (depends on log rotation speed). It's really easy to configure:

Input configuration:

input_type: log
paths:
- /opt/app/logs

Output configuration

output.logstash:
  hosts: ["remote_host:5044"]
  index: filebeat_logs

In the logstash side you must install and configure the Beats input plugin:

input {
  beats {
    port => 5044
  }
}

回答2:

Logstash doesn't contain any magic to read files from other computer's file systems (and that's probably a good thing). You'll either have to mount the remote file system that contains the logs you're interested in or you have to install a log shipper (like e.g. Logstash) on the remote machine and configure it to send the data to your current Logstash instance (or an intermediate broker like Redis, RabbitMQ, or Kafka).

You could also use the syslog daemon (that's probably already installed on the machine) to ship logs via the syslog protocol, but keep in mind that there's no guarantee of the maximum allowed length of each message.

回答3:

You can add the remote system IP in the path and access the logs from Remote machine.

input {     
file {
    path => "\\IP address/home/Desktop/Logstash-Input/**/*_log"
    start_position => "beginning"
}}

来源：https://stackoverflow.com/questions/31155534/read-log-file-from-a-remote-machine-with-file-input-plugin-using-logstash