发表新帖
发表新帖

Writing Custom Kafka Serializer

前端 未结
关注
3 530
爱一瞬间的悲伤
爱一瞬间的悲伤 2020-11-27 14:17

I am using my own class in a Kafka message which has a bunch of String data types.

I therefore cannot use the default serializer class or the StringSerializer

3条回答
  • 北荒
    北荒 (楼主)
    2020-11-27 14:55

    You need to implement both encode and decoder

    public class JsonEncoder implements Encoder {
        private static final Logger LOGGER = Logger.getLogger(JsonEncoder.class);

        public JsonEncoder(VerifiableProperties verifiableProperties) {
            /* This constructor must be present for successful compile. */
        }

        @Override
        public byte[] toBytes(Object object) {
            ObjectMapper objectMapper = new ObjectMapper();
            try {
                return objectMapper.writeValueAsString(object).getBytes();
            } catch (JsonProcessingException e) {
                LOGGER.error(String.format("Json processing failed for object: %s", object.getClass().getName()), e);
            }
            return "".getBytes();
        }
    }



    The decoder code
    


    public class JsonDecoder  implements Decoder {
    private static final Logger LOGGER = Logger.getLogger(JsonEncoder.class);
    public JsonDecoder(VerifiableProperties verifiableProperties) {
        /* This constructor must be present for successful compile. */
    }

    @Override
    public Object fromBytes(byte[] bytes) {
        ObjectMapper objectMapper = new ObjectMapper();
        try {
            return objectMapper.readValue(bytes, Map.class);
        } catch (IOException e) {
            LOGGER.error(String.format("Json processing failed for object: %s", bytes.toString()), e);
        }
        return null;
    }
}



    The pom entry
    


    
    com.fasterxml.jackson.core
    jackson-databind
    2.4.1.3


    


    Set the default encoder in the Kafka property
    


    properties.put("serializer.class","kafka.serializer.DefaultEncoder");

    


    The writer and reader code is as follows
    


    byte[] bytes = encoder.toBytes(map);
        KeyedMessage message =new KeyedMessage(this.topic, bytes);

JsonDecoder decoder = new JsonDecoder(null);
Map map = (Map) decoder.fromBytes(it.next().message());

    
    
    
             
    
                                                        
    
            
            
    
              
                
                0
              
                   
                
               讨论(0)
              
                                                  
              
              
    
                          
             
       
          
              
    
                                       
    

    

                
    
                         
          
                              			
        
        
        
    
          
    
            
            
    
              
              
    
            
    


                                 
              
    
            
    
                          
    

        
         
    
                
                
    
                  
                
    
                
    
                   看不清?
                
    
              
    
                                  
    
                    
   
                 
             
              
    提交回复
    
            
    
          
    
        
    
      
       
      
    
    
    
          
 
     
 
    
        
    热议问题