问题
I'm using jdbcTemplate to Batch-Insert into 2 tables. The 1st table is easy, and has an ID. The 2nd table has an FK Reference USER_ID which I need to obtain from Table 1 before inserting.
Suppose I have this:
Main Java Code (here I split up into batches <= 1000)
for(int i = 0; i < totalEntries.size(); i++) {
// Add to Batch-Insert List; if list size ready for batch-insert, or if at the end, batch-persist & clear list
batchInsert.add(user);
if (batchInsert.size() == 1000 || i == totalEntries.size() - 1) {
// 1. Batch is ready, insert into Table 1
nativeBatchInsertUsers(jdbcTemplate, batchInsert);
// 2. Batch is ready, insert into Table 2
nativeBatchInsertStudyParticipants(jdbcTemplate, batchInsert);
// Reset list
batchInsert.clear();
}
}
Method to Batch-Insert into Table 1 (note I'm getting the Seq Val here for USERS_T)
private void nativeBatchInsertUsers(JdbcTemplate jdbcTemplate, final List<UsersT> batchInsert) {
String sqlInsert_USERS_T = "INSERT INTO PUBLIC.USERS_T (id, password, user_name) " +
"VALUES (nextval('users_t_id_seq'), ?, ? " +
")";
// Insert into USERS_T using overridden JdbcTemplate's Native-SQL batchUpdate() on the string "sqlInsert_USERS_T"
jdbcTemplate.batchUpdate(sqlInsert_USERS_T, new BatchPreparedStatementSetter() {
@Override
public int getBatchSize() {
return batchInsert.size();
}
@Override
public void setValues(PreparedStatement ps, int i) throws SQLException {
ps.setString(1, null);
ps.setString(2, batchInsert.get(i).getUsername());
// etc.
});
}
Method to Batch-Insert into Table 2
private void nativeBatchInsertStudyParticipants(JdbcTemplate jdbcTemplate, final List<UsersT> batchInsertUsers) {
String sqlInsert_STUDY_PARTICIPANTS_T =
"INSERT INTO PUBLIC.STUDY_PARTICIPANTS_T (id, study_id, subject_id, user_id) "
"VALUES (nextval('study_participants_t_id_seq'), ?, ?, ?
")";
// Insert into STUDY_PARTICIPANTS_T using overridden JdbcTemplate's Native-SQL batchUpdate() on the string "sqlInsert_USERS_T"
jdbcTemplate.batchUpdate(sqlInsert_STUDY_PARTICIPANTS_T, new BatchPreparedStatementSetter() {
@Override
public int getBatchSize() {
return batchInsert.size();
}
@Override
public void setValues(PreparedStatement ps, int i) throws SQLException {
// PROBLEM: For Param #4, USER_ID, need to get the USERS_T.ID from Batch-Insert #1
}
});
}
When I come to the 2nd Batch-Insert, one of the columns is an FK back to USERS_T.ID which is called STUDY_PARTICIPANTS_T.USER_ID. Is it possible for me to obtain it by keeping the jdbcTemplate.batchUpdate() logic?
回答1:
Here's the answer.
1) One solution, if you're using jdbcTemplate (Spring JDBC), is to reserve your own ID range in advance. Then provide the manually-calculated IDs for each row yourself. E.g.
@Transactional(readOnly = false, rollbackFor = Exception.class)
public void doMultiTableInsert(List<String> entries) throws Exception {
// 1. Obtain current Sequence values
Integer currTable1SeqVal = table1DAO.getCurrentTable1SeqVal();
Integer currTable2SeqVal = table2DAO.getCurrentTable2SeqVal();
// 2. Immediately update the Sequences to the calculated final value (this reserves the ID range immediately)
table1DAO.setTable1SeqVal(currTable1SeqVal + entries.size());
table2DAO.setTable2SeqVal(currTable2SeqVal + entries.size());
for(int i = 0; i < entries.size(); i++) {
// Prepare Domain object...
UsersT user = new User();
user.setID(currTable1SeqVal + 1 + i); // Set ID manually
user.setCreatedDate(new Date());
// etc.
StudyParticipantsT sp = new StudyParticipantsT();
sp.setID(currTable2SeqVal + 1 + i); // Set ID manually
// etc.
user.setStudyParticipant(sp);
// Add to Batch-Insert List
batchInsertUsers.add(user);
// If list size ready for Batch-Insert (in this ex. 1000), or if at the end of all subjectIds, perform Batch Insert (both tables) and clear list
if (batchInsertUsers.size() == 1000 || i == subjectIds.size() - 1) {
// Part 1: Insert batch into USERS_T
nativeBatchInsertUsers(jdbcTemplate, batchInsertUsers);
// Part 2: Insert batch into STUDY_PARTICIPANTS_T
nativeBatchInsertStudyParticipants(jdbcTemplate, batchInsertUsers);
// Reset list
batchInsertUsers.clear();
}
}
}
then your Batch-Insert submethods referenced above:
1)
private void nativeBatchInsertUsers(JdbcTemplate jdbcTemplate, final List<UsersT> batchInsertUsers) {
String sqlInsert = "INSERT INTO PUBLIC.USERS_T (id, password, ... )"; // etc.
jdbcTemplate.batchUpdate(sqlInsert, new BatchPreparedStatementSetter() {
@Override
public int getBatchSize() {
return batchInsertUsers.size();
}
@Override
public void setValues(PreparedStatement ps, int i) throws SQLException {
ps.setInt(1, batchInsertUsers.get(i).getId()); // ID (provided by ourselves)
ps.setDate(2, batchInsertUsers.get(i).getCreatedDate());
//etc.
}
});
}
2)
private void nativeBatchInsertStudyParticipants(JdbcTemplate jdbcTemplate, final List<UsersT> batchInsertUsers) {
String sqlInsert = "INSERT INTO PUBLIC.STUDY_PARTICIPANTS_T (id, ... )"; // etc.
jdbcTemplate.batchUpdate(sqlInsert, new BatchPreparedStatementSetter() {
@Override
public int getBatchSize() {
return batchInsertUsers.size();
}
@Override
public void setValues(PreparedStatement ps, int i) throws SQLException {
ps.setInt(1, batchInsertUsers.get(i).getStudyParticipants().getId()); // ID (provided by ourselves)
//etc.
}
});
}
There are ways to get/set Sequence values, e.g. in Postgres it's
SELECT last_value FROM users_t_id_seq; -- GET SEQ VAL
SELECT setval('users_t_id_seq', 621938); -- SET SEQ VAL
Note also that everything is under @Transactional. If there are any exceptions in the method all data gets rolled back (for all exceptions, rollbackFor = Exception.class). The only thing that doesn't get rolled back is the manual Sequence update. But that's OK, sequences can have gaps.
2) Another solution, if you're willing to drop down to the PreparedStatement level, is Statement.RETURN_GENERATED_KEYS:
PreparedStatement ps = con.prepareStatement(sql, Statement.RETURN_GENERATED_KEYS)
After you execute ps, the ResultSet will contain your IDs in the order they were created. You can iterate over the ResultSet and store the IDs in a separate list.
while (rs.next()) {
generatedIDs.add(rs.getInt(1));
}
Remember that in this case you're responsible for your own Transaction Management. You need to conn.setAutoCommit(false); to have the batches pile up without real persistence, and then conn.commit(); / conn.rollback();.
来源：https://stackoverflow.com/questions/60356118/java-spring-jdbc-batch-insert-into-2-tables-obtain-fk-id-from-1st-batch-insert