Flink 1.11.1:jdbc-connector 添加支持Clickhouse

文章目录

• • 解决的问题：
  • 做的代码改动:
  • 编译使用
  • 替换已有的flink-connector-jdbc包

解决的问题：

Flink JDBC写数据到Clickhouse默认没有实现，需要自己实现，需新添加ClickhouseJDBCDialect

做的代码改动:

1、修改
flink-release-1.11.1/flink-connectors/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/dialect/JdbcDialects.java

2、添加ClickhouseJDBCDialect.java

package org.apache.flink.connector.jdbc.dialect;

import org.apache.flink.table.types.logical.LogicalTypeRoot;
import org.apache.flink.table.types.logical.RowType;
import org.apache.flink.connector.jdbc.internal.converter.JdbcRowConverter;
import org.apache.flink.connector.jdbc.internal.converter.ClickhouseRowConverter;
import java.util.Arrays;
import java.util.List;
import java.util.Optional;
import java.util.stream.Collectors;

/**
 * JDBC dialect for Clickhouse.
 */
public class ClickhouseJDBCDialect implements JdbcDialect {
   
   

	private static final long serialVersionUID = 1L;


	private static final int MAX_TIMESTAMP_PRECISION = 6;
	private static final int MIN_TIMESTAMP_PRECISION = 1;


	private static final int MAX_DECIMAL_PRECISION = 65;
	private static final int MIN_DECIMAL_PRECISION = 1;

        @Override
        public JdbcRowConverter getRowConverter(RowType rowType) {
   
   
                return new ClickhouseRowConverter(rowType);
        }

        @Override
        public boolean canHandle(String url) {
   
   
            return url.startsWith("jdbc:clickhouse:");
        }

        @Override
        public Optional<String> defaultDriverName() {
   
   
            return Optional.of("ru.yandex.clickhouse.ClickHouseDriver");
        }
	
	@Override
        public String quoteIdentifier(String identifier) {
   
   
            return "`" + identifier + "`";
        }

        @Override
        public Optional<String> getUpsertStatement(String tableName, String[] fieldNames, String[] uniqueKeyFields) {
   
   
            String columns = Arrays.stream(fieldNames)
                    .collect(Collectors.joining(", "));
            String placeholders = Arrays.stream(fieldNames)
                    .map(f -> quoteIdentifier(f))
                    .collect(Collectors.joining(", "));
            return Optional.of(getInsertIntoStatement(tableName, fieldNames));
        }


        
        @Override
	public String dialectName() {
   
   
		return "Clickhouse";
	}



}

编译使用

root@bigdata-dev1:/opt/dev/flink-release-1.11.1/flink-connectors/flink-connector-jdbc# mvn package -Dmaven.test.skip=true -Dcheckstyle.skip=true

替换已有的flink-connector-jdbc包

来源：oschina

链接：https://my.oschina.net/u/4315748/blog/4666408