why is the first member of the tuple received from kafka using spark direct stream is null

时光总嘲笑我的痴心妄想 提交于 2019-12-10 21:43:17

问题


when reading messages from kafka using KafkaUtils.createDirectStream, the v1._1 member of the Tuple2 is null:

KafkaUtils.createDirectStream(
                streamingContext,
                String.class,
                String.class,
                StringDecoder.class,
                StringDecoder.class,
                kafkaParams,
                topicsSet
                ).map(new Function<Tuple2<String,String>, String>() {
                    @Override
                    public String call(Tuple2<String, String> v1)
                            throws Exception {
                        System.out.println(v1._1);
                        return null;
                    }
                });

while the _2 member contains the message itself that was passed to kafka.

I have two questions:

1) why is v1._1 null?

2) is there a way to pass the topic name in kafka (the same way the message is put into kafka) so that v1._1 will contain the topic name?

来源:https://stackoverflow.com/questions/36778977/why-is-the-first-member-of-the-tuple-received-from-kafka-using-spark-direct-stre

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!