Posting huge datasets to a webservice from excel VBA

问题

Hello fellow Excel vba coders

I have this great macro in my excel sheet, where i compile XML based on the rows the user puts in - after this it post the xml to a webservice.

You can have a look at my code below - It is fairly simple:

Set XMLHttpRequest = New MSXML2.XMLHTTP    
With XMLHttpRequest
    .Open "POST", URL, False
    .setRequestHeader "Content-Type", "text/xml; encoding='utf-8'"
    .setRequestHeader "Content-Length", strLength
    .send strXML
End With

Right now it works great when there is less than 200 rows, yet it times out when the row number gets above 1000 rows. The string of XML I post is really big, and i'm quite sure that's the reason it times out.

Now my problem is, how do i post this huge dataset, that exceed 1.000 rows, maybe even above 20.000 rows, to a webservice?

So far i have spend a lot of time to look for a possible solution around the web, but have yet to find a way to handle this. So far i have the following ideas to solve the problem:

Copy the sheet to a new workbook, take the location of the new workbook and convert the file to a Base64 string and post the entire file to a new .asmx service and handle the "workbook" in C# code.
Convert the huge string to some kind of byte array and post that to a new .asmx webservice and handle the C# code.

I really hope one of you guys can point me in the right direction and help me solve this problem?

回答1:

Instead of sending the entire sheet as a single object, send each row individually.

This will require your web service to be modified to accept just a row, rather than, what I would guess is currently being sent as an array of rows.

This would allow you to process any number of rows as the most you are processing at any one time is exactly one.

回答2:

It is possible the issue of configuration on server side... See this post and this.

回答3:

Ok - I found a solution to my problem, and it seems like the best way to handle this problem.

I use the following function to make a copy of the workbook:

Private Function saveAS(Path As String)
    Application.EnableEvents = False
    Application.DisplayAlerts = False

    ActiveWorkbook.Sheets.Copy
    ActiveWorkbook.saveAS Path, FileFormat:=51
    ActiveWorkbook.Close savechanges:=True

    Application.EnableEvents = True
    Application.DisplayAlerts = True
End Function

Then i encode the file into a base64string, like so:

Private Function EncodeFileBase64(Filename As String) As String
    Dim arrData() As Byte
    Dim fileNum As Integer

    Filename = Filename + ".xlsx"
    fileNum = FreeFile
    Open Filename For Binary As fileNum
    ReDim arrData(LOF(fileNum) - 1)
    Get fileNum, , arrData
    Close fileNum

    Dim objXML As MSXML2.DOMDocument
    Dim objNode As MSXML2.IXMLDOMElement

    Set objXML = New MSXML2.DOMDocument
    Set objNode = objXML.createElement("b64")

    objNode.DataType = "bin.base64"
    objNode.nodeTypedValue = arrData
    EncodeFileBase64 = objNode.Text

    Set objNode = Nothing
    Set objXML = Nothing
End Function

And then i send the encoded string to my own .asmx webservice and work with it in C# The webmethod looks like this:

    [WebMethod]
    [ScriptMethod(UseHttpGet = true)]
    public string UploadXML(string base64string)
    {
        try
        {
            byte[] bytes = Convert.FromBase64String(base64string);

            using (MemoryStream ms = new MemoryStream(bytes))
            {
                using (var package = new ExcelPackage(ms))
                {
                    ExcelWorkbook workBook = package.Workbook;
                    ExcelWorksheet settings = workBook.Worksheets.SingleOrDefault(w => w.Name == "sheet1");
                    ExcelWorksheet data = workBook.Worksheets.SingleOrDefault(w => w.Name == "sheet2");

                    //Getting data
                    string SS1 = (string)settings.Cells[8, 3].Value;
                    string ss2 = (string)settings.Cells[7, 3].Value;
                }
            }
            return "success";

        }
        catch (Exception ee)
        {
            return ee.Message;
        }
    }

I just need to find a good way to pull out all the data in a smart algorithm, i dont think that will be a problem at all :)

回答4:

I know you have a solution but I'd like to give an opinion...

If the data is really so tabular then shouldn't it be in a database like SQL Server? If you are uploading to a database then SQL Server has some nice bulk upload features to load workbooks efficiently.

来源：https://stackoverflow.com/questions/50983004/posting-huge-datasets-to-a-webservice-from-excel-vba

标签

excel

vba

excel-vba

web-services