Question : SQL Bulk Copy - Importing CSV File and Null Values Appearing

Every month we output a spreadsheet of account numbers and other account information that are processed through a vb.net stand alone application and imported into a SQL Database. Up to this month, we have had no issues with this import process. However, it was noticed this month that when the account number passed 20xxxxxxxx that it imported in as a NULL instead of with the account number.

Originally I thought it was a data type issue. I originally had the Account Number field set up as Numeric (18,0).  I then tried int, but still had the issue. I tried Varchar(50) and still had the issue. I also then checked previous months and seen where account numbers greater than 20xxxxxxxx came in perfectly fine. So that rules out a Data Type issue as far as I am concerned.

The only difference this month is our spreadsheet has grown a great deal and we are up to 676 records. It is at the 169th record that the last account number appears and the 170th record is when the Nulls begin to appear. However, the rest of the fields for the records past 170 come in fine. It is just the Account Number field.

I did come across some post on another site, that mentioned about bulk copy not working for larger .csv files and that Nulls would randomly be imported rather than the value. Of course, now I can not find the post to reference it here. There was no real valid reply to the person's issue that I found useful.

Finally to the question, if using SQLBulkCopy has limitations in regards to importing .csv files, what is another approach to get this spreadsheet correctly imported into the SQL Table? Or is there something I am missing in my code that would resolve this issue?

Public Function ImportCSVBulk(ByVal SQLTableName As String, ByVal CSVFile As String, ByVal SheetName As String) As Boolean
        'Create the connection strings
        Dim CSVConnString As String = BuildCSVConnString(CSVFile).ToString
        Dim sqlConnString As String = BuildSQLConnString.ToString

        'Set up the connection
        Using CSVConn As New Odbc.OdbcConnection(CSVConnString)
            'CSV command object
            Dim CSVCmd As New Odbc.OdbcCommand("Select * From " & SheetName & ".csv", CSVConn)
            'Open the Excel connection
            CSVConn.Open()
            'Set up the DataReader
            Dim CSVRead As Odbc.OdbcDataReader = CSVCmd.ExecuteReader
            'Set up the SQLBulkCopy
            Using bulkCopy As New SqlBulkCopy(sqlConnString)
                'Set the destination table
                bulkCopy.DestinationTableName = SQLTableName
                Try
                    'Write the data to the server
                    bulkCopy.WriteToServer(CSVRead)
                    'Success
                    Return True
                Catch ex As Exception
                    MsgBox(Err.Number & " " & Err.Description)
                    'Error
                    Return False
                Finally
                    'Close the DataReader
                    If Not CSVRead.IsClosed Then
                        CSVRead.Close()
                    End If
                End Try
            End Using 'end bulkCopy
        End Using 'end CSVConn
    End Function

Let me know if attachments of the spreadsheet and or table would help. I can't send the real information, but I can give you a sample of the layout.

Thanks in advance for any help you can provide.

Answer : SQL Bulk Copy - Importing CSV File and Null Values Appearing

Some things to try:

1. Open the CSV file in notepad rather Excel and see if there is anything strange in those lines. Excel will apply formatting which may hide the issue.

2. Cut that data down to th 20xxxx lines and import just that and make sure that it is to do with that (rather than the line number)


There are many alternative methods to importing CSV files.

BCP.EXE (a command line tool that comes with SQL Server)
BULK INSERT (a T-SQL statement which is essentially the same asBCP except that you run it from within SQL Server
SSIS (SQL Integration Services - an ETL tool which is probably over the top for this purpose)

There are three unecessary things that are currently adding to the possible issues:

1. A CSV file is a straight text file. It is not necessary to open it in Excel OR use an ODNC connection to open it.
2. Throwing a .Net app into the mix adds to the number of things that can go wrong.

If I was doing it I would just use BCP.EXE to load it (though it can be a bit of a pain to set up first time)