Linux logstash org.elasticsearch.discovery.MasterNotDiscoveredException 错误
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/17236466/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
logstash org.elasticsearch.discovery.MasterNotDiscoveredException error
提问by krisdigitx
I have installed logstash 1.1.13 with elasticcsearch-0.20.6 the below config for logstash.conf
我已经使用elasticcsearch-0.20.6安装了logstash 1.1.13，logstash.conf的以下配置
input {
tcp {
port => 524
type => rsyslog
}
udp {
port => 524
type => rsyslog
}
}
filter {
grok {
type => "rsyslog"
pattern => [ "<%{POSINT:syslog_pri}>%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{PROG:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" ]
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{@source_host}" ]
}
syslog_pri {
type => "rsyslog"
}
date {
type => "rsyslog"
syslog_timestamp => [ "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
mutate {
type => "rsyslog"
exclude_tags => "_grokparsefailure"
replace => [ "@source_host", "%{syslog_hostname}" ]
replace => [ "@message", "%{syslog_message}" ]
}
mutate {
type => "rsyslog"
remove => [ "syslog_hostname", "syslog_message", "syslog_timestamp" ]
}
}
output {
elasticsearch {
host => "127.0.0.1"
port => 9300
node_name => "sysloG33r-1"
bind_host => "localhost"
}
}
and
和
elasticsearch.yml
弹性搜索.yml
cluster:
name: syslogcluster
node:
name: "sysloG33r-1"
path:
data: /var/lib/elasticsearch
path:
logs: /var/log/elasticsearch
network:
host: "0.0.0.0"
and started logstash with command
并使用命令启动 logstash
[root@clane elasticsearch]# java -jar /usr/local/bin/logstash/bin/logstash.jar agent -f /etc/logstash/logstash.conf
Using experimental plugin 'syslog_pri'. This plugin is untested and may change in the future. For more information about plugin statuses, see http://logstash.net/docs/1.1.13/plugin-status {:level=>:warn}
date: You used a deprecated setting 'syslog_timestamp => ["MMM d HH:mm:ss", "MMM dd HH:mm:ss"]'. You should use 'match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]' {:level=>:warn}
PORT SETTINGS 127.0.0.1:9300
log4j, [2013-06-21T14:40:08.013] WARN: org.elasticsearch.discovery: [sysloG33r-1] waited for 30s and no initial state was set by the discovery
Failed to index an event, will retry {:exception=>org.elasticsearch.discovery.MasterNotDiscoveredException: waited for [1m], :event=>{"@source"=>"tcp://10.66.59.35:34662/", "@tags"=>[], "@fields"=>{"syslog_pri"=>["78"], "syslog_program"=>["crond"], "syslog_pid"=>["6511"], "received_at"=>["2013-06-21T13:40:01.845Z"], "received_from"=>["10.66.59.35"], "syslog_severity_code"=>6, "syslog_facility_code"=>9, "syslog_facility"=>"clock", "syslog_severity"=>"informational"}, "@timestamp"=>"2013-06-21T12:40:01.000Z", "@source_host"=>"kent", "@source_path"=>"/", "@message"=>"(root) CMD (/opt/bin/firewall-state.sh)", "@type"=>"rsyslog"}, :level=>:warn}
and elasticsearch
和弹性搜索
/usr/local/bin/elasticsearch start
I can see all the correct java ports for elasticsearch(9200,9300) and logstash(524)
我可以看到 elasticsearch(9200,9300) 和 logstash(524) 的所有正确的 java 端口
tcp 0 0 :::524 :::* LISTEN 12557/java
tcp 0 0 :::9200 :::* LISTEN 10782/java
tcp 0 0 :::9300 :::* LISTEN 10782/java
tcp 0 0 ::ffff:127.0.0.1:9301 :::* LISTEN 12557/java
udp 0 0 :::524 :::* 12557/java
udp 0 0 :::54328 :::* 10782/java
however i see this error on logstash, any ideas?
但是我在logstash上看到了这个错误，有什么想法吗？
Failed to index an event, will retry {:exception=>org.elasticsearch.discovery.MasterNotDiscoveredException: waited for [1m], :event=>{"@source"=>"tcp://10.66.59.35:33598/", "@tags"=>[], "@fields"=>{"syslog_pri"=>["78"], "syslog_program"=>["crond"], "syslog_pid"=>["12983"], "received_at"=>["2013-06-21T12:07:01.541Z"], "received_from"=>["10.66.59.35"], "syslog_severity_code"=>6, "syslog_facility_code"=>9, "syslog_facility"=>"clock", "syslog_severity"=>"informational"}, "@timestamp"=>"2013-06-21T11:07:01.000Z", "@source_host"=>"kent", "@source_path"=>"/", "@message"=>"(root) CMD (/opt/bin/firewall-state.sh)", "@type"=>"rsyslog"}, :level=>:warn}
回答by jgoldschrafe
I'm going to assume you've checked the obvious things, like "is ElasticSearch running?" and "can I open a TCP connection to port 9300 on localhost?"
我假设您已经检查了一些显而易见的事情，例如“ElasticSearch 是否正在运行？” 和“我可以在本地主机上打开到端口 9300 的 TCP 连接吗？”
Even though you're using a hostparameter in your elasticsearchoutput, what's probably happening is that the ElasticSearch client in Logstash is trying to discover cluster members by multicast (which is how a new install is typically configured by default), and is failing. This is common on EC2, as well as many other environments where firewall configurations may interfere with multicast discovery. If this is the only member in your cluster, setting the following in your elasticsearch.ymlshould do the trick:
即使您host在elasticsearch输出中使用了参数，可能发生的情况是 Logstash 中的 ElasticSearch 客户端正在尝试通过多播（这是新安装通常默认配置的方式）发现集群成员，并且失败了。这在 EC2 以及防火墙配置可能会干扰多播发现的许多其他环境中很常见。如果这是您集群中的唯一成员，则在您的集群中设置以下内容elasticsearch.yml应该可以解决问题：
discovery:
zen:
ping:
multicast:
enabled: false
unicast:
hosts: <your_ip>[9300-9400]
On AWS, there's also an EC2 discovery plugin that will clear this right up for you.
在 AWS 上，还有一个 EC2 发现插件可以为您解决这个问题。
This question really belongs on Server Fault rather than Stack Overflow, by the way.
顺便说一下，这个问题确实属于服务器故障而不是堆栈溢出。
回答by Hadrien
I had a similar issue, and it came from my ip configuration. In a nutshell, check that you have only one ip address on the logstash host. If not, it can choose the wrong one.
我有一个类似的问题，它来自我的 ip 配置。简而言之，检查您在 logstash 主机上是否只有一个 IP 地址。如果不是，它可以选择错误的。
Posted the same answer here: Logstash with Elasticsearch
在这里发布了相同的答案：Logstash with Elasticsearch
回答by Ysak
I came across same kind of issue and fixed by adding cluster option in the elasticsearch conf in logstash. Since you have modified the cluster name in elasticsearch.yml, the logstash client will be not able to find the cluster using the default value.
我遇到了同样的问题，并通过在logstash的elasticsearch conf中添加集群选项来解决。由于您在elasticsearch.yml 中修改了集群名称，因此logstash 客户端将无法使用默认值找到集群。
Try doing this also
也尝试这样做
