How to read a Text file using T-SQL?

Question

What\'s the best way to read a text file using T-SQL? I\'ve seen the BULK INSERT and many different functions but non of them are what I\'m looking for.

I need to r

Answer 1

I use a very simple CLR procedure that reads the entire file and separates the lines into rows -- returning a one column table of values. Like I said, the CLR code is very simple:

[MyFileIO.vb]

Imports System
Imports System.IO
Imports System.Data
Imports System.Data.SqlClient
Imports System.Data.SqlTypes
Imports Microsoft.SqlServer.Server
Imports System.Collections
Imports System.Runtime.InteropServices

Partial Public Class TextFiles
     _
    Public Shared Function FileToTable(ByVal FileName As String) As IEnumerable

        Dim s() As String

        Using sr As New StreamReader(FileName)
            s = Split(sr.ReadToEnd, vbCrLf)
        End Using

        Return s
    End Function


    Public Shared Sub GetNextSplitString(ByVal Value As Object,  ByRef Data As SqlChars)
        Data = New SqlChars(CType(Value, String))
    End Sub
End Class

Examples

select *, getdate() as [CreateDate], 1 as [AnotherColumn], 'xyz' as [ETC]
from dbo.FileToTable('c:\file.ext')

select line, left(line, 10), right(line, 10)
from dbo.FileToTable('c:\file.ext')

select ...
into [tablename]
from dbo.FileToTable('c:\file.ext')

More Details

Compile the CLR DLL like this:

c:\windows\microsoft.net\framework\v3.5\vbc.exe /target:library MyFileIO.vb

Register the CLR DLL like this:

create assembly MyFileIO from 'c:\MyFileIO.dll' with permission_set = unsafe
go
create function dbo.FileToTable (@FileName nvarchar(255)) returns table (line nvarchar(max)) as external name MyFileIO.TextFiles.FileToTable
go

If you get an error, you may need to enable the CLR support in the db:

ALTER DATABASE [dbname] SET trustworthy ON
go
sp_configure 'clr enabled', 1
GO
RECONFIGURE
GO

Whenever you change the DLL, you have to drop the procedure and the assembly and run the code from above again to re-register it.