How to traverse files (conditionally) faster than using FileSystemObject

Question

I\'ve written some VBA code using file objects to go into a folder, search for particular files (CSV) that meet certain criteria (contain \"HR\" in filename and created with

Answer 1

In addition to using the Dir function instead of FileSystemObject, if you cannot automate PowerQuery, and all you need is the data and not the formatting, consider making a direct data connection to the source workbooks using ADODB.

Add a reference to Microsoft ActiveX Data Objects 6.1 Library (via Tools -> References...). There may be versions other than 6.1; choose the highest.

Then you can use something like the following code:

Dim fso As New Scripting.FileSystemObject
Dim filepath As Variant
For Each filepath In filesToProcess
    ' Check the date last modified
    fileDate = fso.GetFile(item).DateLastModified ' modify as needed
    If firstDate < fileDate And secondDate > fileDate Then

        Dim connectionString As String
        connectionString = _
            "Provider=Microsoft.ACE.OLEDB.12.0;" & _
            "Data Source=""" & filepath & """;" & _
            "Extended Properties=""Excel 12.0;HDR=No"""

        Dim worksheetName As String
        worksheetName = "Sheet1"
        ' There can be multiple worksheets per workbook.
        ' If you are only interested in one worksheet per workbook, then fill in worksheetName somehow
        ' Otherwise, you will probably need an inner loop to iterate over all the worksheets

        Dim sql As String
        sql = _
            "SELECT * " & _
            "FROM [" & worksheetName & "$]"

        Dim rs As New ADODB.Recordset
        rs.Open sql, connectionString

        destinationWorksheet.Range("A1").CopyFromRecordset rs

        rs.Close
        Set rs = Nothing
    End If
Next