How to load csv files from the internet into an Access database?

问题

I have the following array that contains ticker symbols: Public Shared tickerArray() As String = {"GOOG", "AAPL", "V", "MSFT"}. I need to use this loop: For Each tickerValue In Form1.tickerArray to load the csv file for each ticker symbol into one large table in Microsoft Access. The csv files are located at "http://ichart.yahoo.com/table.csv?s=" & tickerValue. I also need the respective ticker symbol to be loaded into each line of the table that is imported from the csv file in order to make each line unique. So the columns in the database should be: "Ticker, Date, Open, High, Low, Close, Volumne & Adj Close".

I've found information about loading a local csv file into Access but I can't seem to figure out how to load csv files from the internet into Access through vb.net.

Also, I will need to update this table frequently so I need to only insert new unique lines from the csv file. No duplicates.

Any suggestions? Thanks!

UPDATE: Here is the code I have so far.

Imports System.Data
Imports System.Data.OleDb
Imports System.Net
Imports System.IO

Public Class Form1

Public Shared tickerArray() As String = {"GOOG", "AAPL", "V", "MSFT"}

Private Sub btnUpdate_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnUpdate.Click
    Dim myConnection As OleDbConnection
    Dim DBpath As String = 
    Dim sConnectionString As String = "Provider=Microsoft.ACE.OLEDB.12.0; Data Source=" & DBpath & ";Persist Security Info=True"
    myConnection = New OleDbConnection(sConnectionString)
    myConnection.Open()

    Dim strURL As String
    Dim strBuffer As String
    For Each tickerValue In Form1.tickerArray
        'Creates the request URL for Yahoo.
        strURL = "http://ichart.yahoo.com/table.csv?s=" & tickerValue
        strBuffer = RequestWebData(strURL)
        'Create array.
        Dim sReader As New StringReader(strBuffer)
        Dim List As New List(Of String)
        Do While sReader.Peek >= 0
            List.Add(sReader.ReadLine)
        Loop
        Dim lines As String() = List.ToArray
        sReader.Close()
        For Each Line In lines.Skip(1)
            MsgBox(Line)
            Dim myInsert As String = TextToInsert(Line, tickerValue)
            Dim cmd As New OleDbCommand(myInsert, myConnection)
            cmd.ExecuteNonQuery()
        Next
    Next
End Sub

Function TextToInsert(ByVal inputString As String, ByVal ticker As String)
    Dim Dt As String = inputString.Split(",")(0).Trim
    Dim Open As String = inputString.Split(",")(1).Trim
    Dim High As String = inputString.Split(",")(1).Trim
    Dim Low As String = inputString.Split(",")(1).Trim
    Dim Close As String = inputString.Split(",")(1).Trim
    Dim Volume As String = inputString.Split(",")(1).Trim
    Dim Adj_Close As String = inputString.Split(",")(1).Trim
    Dim SQLstr As String
    SQLstr = "INSERT INTO Historical (Ticker, Date, Open, High, Low, Close, Volume, Adj Close) " & "VALUES (" & "'" & ticker & "','" & Dt & "','" & Open & "','" & High & "'," & "'" & Low & "','" & Close & "','" & Volume & "','" & Adj_Close & "'" & ")"
    Return SQLstr
End Function

Private Function RequestWebData(ByVal pstrURL As String) As String
    Dim objWReq As WebRequest
    Dim objWResp As WebResponse
    Dim strBuffer As String
    'Contact the website
    objWReq = HttpWebRequest.Create(pstrURL)
    objWResp = objWReq.GetResponse()
    'Read the answer from the Web site and store it into a stream
    Dim objSR As StreamReader
    objSR = New StreamReader(objWResp.GetResponseStream)
    strBuffer = objSR.ReadToEnd
    objSR.Close()

    objWResp.Close()

    Return strBuffer
End Function

End Class

I get error code "OleDBException was unhandled. Syntax error in INSERT INTO statement." at line cmd.ExecuteNonQuery() Please help!

回答1:

you may use the System.Data.OleDb Namespace to define function to make insert into the db. Something like (in a very rough way):

Dim myConnection As OleDbConnection

Dim sConnectionString  As String = "Provider=Microsoft.ACE.OLEDB.12.0; Data Source=" & DBpath & ";Persist Security Info=True"

myConnection = New OleDbConnection(sConnectionString)

myConnection.Open()

Then make a cycle on each line in the csv

Dim myInsert as String= functionTextToInsert(inputString)

Dim cmd As New OleDbCommand(myInsert, myConnection)

cmd.ExecuteNonQuery()

The functionTextToInsert(ByVal inputString as string) is a function that converts a line from the csv in a insert string: "max;min;vol" -> "Insert into MYTABLE (Column_01,Column_02,Column_03) VALUES (max,min,vol);"

回答2:

Try this:

Writing large number of records (bulk insert) to Access in .NET/C#

I've never worked with DAO, so I can't be much help here, but...

if this MSSQL syntax works on MS ACCESS, use a command object and pass it a parameter containing your csv as a varchar(max), or similar access data type, which will the content of the CSV from your app:

BULK
INSERT mytable
FROM @myCSVstring
WITH
(
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n'
)
GO

Two things, the SO post states that DAO is a lot faster, and your terminators may be different, but I'd start with these.

来源：https://stackoverflow.com/questions/15160781/how-to-load-csv-files-from-the-internet-into-an-access-database

标签

database

vb.net

ms-access

csv