I have my custom Java Object and wish to leverage JVM\'s in built serialization to send it to a Kafka topic, but serialization fails with below error
Since you are using ByteArraySerializer,you need to instantiate a byte[] producer.
Producer producer = new KafkaProducer<>(props);
and then while producing pass the byte[] after serializing or some other method,for instance,
producer.send(new ProducerRecord("test", new Payload().toString().getBytes()));
If you are passing just a Payload Object to the producer then it will be better to have key serializer and value serializer as whatever you intend to pass and while reading you need to read from that data.
It is good practice to use Serializable and ByteArraySerializer/ByteArrayDeserializer.