How to query any table of RDS using Golang SDK

问题

I am writing an AWS lambda to query 10 different tables from RDS(SQL Server) using Golang SDK. What I have learned so far is we have to create a similar struct for the table to query it. But as I want to query 10 tables, So I don't want to create the struct for every table, even the table schema may get changed someday.

Lately, I want to create a CSV file per table as the backup with the queried data and upload it to S3. So is it possible to directly import the CSV file into a lambda, so that I can directly upload it to S3? You can see my current code below

func executeQuery(dbconnection *sql.DB) {

    println("\n\n----------Executing Query ----------")
    query := "select TOP 5 City,State,Country from IMBookingApp.dbo.Address"

    rows, err := dbconnection.Query(query)
    if err != nil {
        fmt.Println("Error:")
        log.Fatal(err)
    }

    println("rows", rows)

    defer rows.Close()
    count := 0
    for rows.Next() {
        var City, State, Country string

        rows.Columns

        err := rows.Scan(&City, &State, &Country)

        if err != nil {
            fmt.Println("Error reading rows: " + err.Error())
        }

        fmt.Printf("City: %s, State: %s, Country: %s\n", City, State, Country)
        count++
    }
}

This code can only work for the Address table, and not for other tables

I have also tried it with GORM

package main

import (
    "fmt"
    "github.com/jinzhu/gorm"
    _ "github.com/jinzhu/gorm/dialects/mssql"
)

type Currency struct {
    CurrencyId  int    `gorm:"column:CurrencyId;"`
    Code        string `gorm:"column:Code;"`
    Description string `gorm:"column:Description;"`
}

func main() {
    db, err := gorm.Open("mssql", "sqlserver://***")
    db.SingularTable(true)

    gorm.DefaultTableNameHandler = func(dbVeiculosGorm *gorm.DB, defaultTableName string) string {
        return "IMBookingApp.dbo.Currency"
    }

    fmt.Println("HasTable-Currency:", db.HasTable("ClientUser"))
    var currency Currency
    db.Debug().Find(&currency)

    fmt.Println("Currency:", currency)
    fmt.Println("Error", err)

    defer db.Close()

}

With both the approaches I couldn't find any way to make the code generic for multiple tables. I would appreciate it if anyone can give me some suggestions or if you can point to some resources.

回答1:

I did not test this code but is should give you an idea how to fetch Rows into strings array.

    defer rows.Close()
    columns, err := rows.Columns()
    if err != nil {
        panic(err)
    }
    for rows.Next() {
        receiver := make([]*string, len(columns))

        err := rows.Scan(receiver)
        if err != nil {
            fmt.Println("Error reading rows: " + err.Error())
        }
    }

GO internally converts many types into strings - https://github.com/golang/go/blob/master/src/database/sql/convert.go#L219

If data is cannot be converted you have 2 options:

1. Easy - update your SQL query to return strings or string compatible data
2. Complicated. Use slice of interface{} instead of slice of *string and fill it in with default values of correct type based on rows.ColumnTypes(). Later you will have to convert real values into strings to save into csv.

回答2:

Below code worked for me -

conn, _ := getConnection()   // Get database connection

rows, err := conn.Query(query)
if err != nil {
    fmt.Println("Error:")
    log.Fatal(err)
}

defer rows.Close()

columns, err := rows.Columns()

if err != nil {
    panic(err)
}

for rows.Next() {
    receiver := make([]string, len(columns))

    is := make([]interface{}, len(receiver))
    for i := range is {
        is[i] = &receiver[i]

        // each is[i] will be of type interface{} - compatible with Scan()
        // using the underlying concrete `*string` values from `receiver`
    }

    err := rows.Scan(is...)

    if err != nil {
        fmt.Println("Error reading rows: " + err.Error())
    }
    fmt.Println("receiver", receiver)

Reference:- sql: expected 3 destination arguments in Scan, not 1 in Golang

来源：https://stackoverflow.com/questions/59610203/how-to-query-any-table-of-rds-using-golang-sdk