Question : Simple Insert Statement very slow

Hi Folks,

I've got a very simple insert statement, as follows

Insert Into MyTable(StringID, StringText)
select 0,N'123452.46093750'
 UNION
select 1,N'468330.90625000'
 UNION
select 2,N'152484.99609375'
 UNION
select 3,N'476937.99609375'
 UNION
select 4,N'30D6-5FFC'
 UNION
select 5,N'AE79-9EED'
 UNION
select 6,N'10.5.0.75'
 UNION
select 7,N'00-21-9B-01-5C-C4'
 UNION
select 8,N'Microsoft XPS Document Writer'
 UNION
select 9,N'ActiveReports 6'
 UNION
select 10,N'6.0.2250.0'
 UNION
select 11,N'24'
......

The UNIONS carry on for 1800 records, and each statement is broken out into batches of 100, just so I don't hit any limits on statement size or number of unions.

This insert takes 4 seconds to insert 1800 rows, why is this so slow ? (MS rants on about TPC figures etc, surely the latest and greatest could manage more than 450 rows per second !!)

The table has no indexes, no relationships:

      Create Table MyTable
      (
            StringID int not null,
            StringText nvarchar(4000) not null
      )

I've change the database logging to "Simple", its Sql 2008 developer edition running on a Quad Core 2.66 box with 4GB RAM. Nothing else is running on that box except Visual Studio 2010.

I've tried ") values (" instead of select, and i've tried the new comma based sql 2008 multiple insert syntax - all the same

Any ideas ? Anything to try ?

Answer : Simple Insert Statement very slow

Here's what resets the DB before each test.
All tests are 10000 inserts.

dbcc freeproccache
dbcc dropcleanbuffers
set nocount on
GO
;
drop table MyTable3
GO
      Create Table MyTable3
      (
            StringID int not null,
            StringText nvarchar(4000) not null
      )
;
=========================
test1 - union form, one big statement (10000 one shot), generated using the below and a few manual tweaks

declare @i int set @i=1
while @i < 10001
begin
print 'select '+right(@i,10)+',N'''+convert(varchar(max),newid())+''' union';
set @i=@i+1
end
=========================

insert MyTable3
select 1,N'A0B8022B-CFE9-43C9-97E5-DE89822AC32F' union
select 2,N'F87C8670-1339-458E-8F1C-C2E9B717F4AA' union
select 3,N'35ACFFFC-52CF-4805-8C30-A4EB41F3A218' union
..........
select 10000,..

Time - 7 minutes+, killed

=========================
test2 - union ALL form
=========================

Time - faster than above.  SORT then DISTINCT is not required.  Union on its own does a distinct

=========================
test3 - union ALL form, 100 per batch, separated by ;
=========================

Time - even faster.  Each batch is committed and does not take much handling in RAM.  Moreover, Query optimizer does not spend much time on long queries.  QO time goes up exponentially for long queries.

=========================
test4 - truncate table, insert again (following from test3 without reset)
=========================

Time - faster than 3.  QO does not have to do any planning, all 100 [static] queries coming from plan cache

=========================
test5 - using parameterised inserts. QO plans only one insert. All other inserts go by plan
Inserts generated using

declare @i int set @i=1
while @i < 101
begin
print 'exec sp_executesql N''insert MyTable3 values(@a,@b)'',N''@a int,@b nvarchar(4000)'',@a='+right(@i,10)+',@b=N'''+convert(varchar(max),newid())+'''';
set @i=@i+1
end
=========================

Time taken - 6s! Fastest.

===================

Conclusions

(1) Sql Server Query Optimizer takes exponentially longer to compile the longer the query text.

(2) Large insert blocks using union or values(),(),() patterns sit in a HEAP in memory, then dequeued from HEAP on insert - bad.

(3) Parameterized Queries are good.  Better if you batch them into say 10 inserts, but each insert would then require (for the example above) 20 parameters in the 10-batch insert.

(4) **No test above, but my guess**  Using an sp for the insert, e.g usp_insert @a, @b  which does the insert and your bulk script is of the form

exec usp_insert 1, N'test text';
exec usp_insert 2, N'test text';
etc

will be as fast as the Parameterized inserts using sp_executesql, because an SP is compiled once only and re-used#.

# - which is the source of parameter sniffing problems....