Spring Batch JdbcPagingItemReader paging not work

问题

I use spring batch to do a data migration job. there are a lot of data so that I decided to use the JdbcPagingItemReader to read the data by page. Below is how I define the reader:

private JdbcPagingItemReader<Map<String, Object>> buildItemReader(final DataSource dataSource, String tableName,
        String tenant){

    String tenantName = tenantHelper.determineTenant(tableName);

    Map<String, Object> sqlParameterValues = new HashMap<>();
    sqlParameterValues.put("tableName", tableName);
    sqlParameterValues.put("tenantName", tenantName);
    sqlParameterValues.put("tenant", tenant);

    JdbcPagingItemReader<Map<String, Object>> itemReader = new JdbcPagingItemReader<>();
    itemReader.setDataSource(dataSource);
    itemReader.setPageSize(2);
    itemReader.setFetchSize(2);
    itemReader.setQueryProvider(generateSqlPagingQueryProvider(tableName,tenantName,tenant));
    //itemReader.setParameterValues(sqlParameterValues);
    itemReader.setRowMapper(new ColumnMapRowMapper());
    try {
        itemReader.afterPropertiesSet();
    } catch (Exception e) {
        e.printStackTrace();
    }

    return itemReader;
}

private PostgresPagingQueryProvider generateSqlPagingQueryProvider(String tableName, String tenantName,
        String tenant) {
    PostgresPagingQueryProvider provider = new PostgresPagingQueryProvider();
    Map<String, Order> sortKeys = new LinkedHashMap<>();
    String sortKey = getSortKeyBytable(tableName);
    sortKeys.put(sortKey, Order.ASCENDING);

    provider.setSelectClause("select *");
    provider.setFromClause("from " + tableName);
    provider.setWhereClause("where " + tenantName + " ='" + tenant + "'");
    provider.setSortKeys(sortKeys);
    return provider;
}

the sortkey I specified is the primary key in the table, it is a string. But the paging is not working as expected. But it throws no error it just read all the data.

In the spring batch document, it provides an example which use the int type id as the sort key, I am wondering if the spring batch paging read only support the int type sort key word? And can not support the string sort key?

Is this a limitation of spring batch?

回答1:

Its not a limitation of spring batch. This is how a database works. It sorts string based on its ascii value.

回答2:

The above code in the question did work and solve my memory issue, the reason it did not work is because I still calling the Cursor Reader in my spring batch step defination. So the paging reader can solve the memory issue

来源：https://stackoverflow.com/questions/55369572/spring-batch-jdbcpagingitemreader-paging-not-work

标签

java

spring-batch