bulk insert with identity column

Today, I was looking for geting a solution to insert a lot of line into SQL server with high performance. I got a way is using BULK INSERT ... to insert huge record into table. It is really high performance ( insert a 50MB text file into DB cost about 10s). How to use the command, you can get the detail from here(http://msdn.microsoft.com/en-us/library/ms188365.aspx). I don't want to introduce the usage of this command. I'd like to introduce the way if there is an identity column in the table, and you don't want to insert any value in this column. How to do it?
There are serveral solutions can be used for this, 
1. BULK INSERT record into a temporary table, then select into the records to actual table. I don't think this is a good solution for high perfornamce insert record. Because we need write one record twice.

2. Use format file to skip some columns. We also use BULK INSERT command, but we need define a format file. You can use command bcp to do this work. E.G. "bcp d2d.dbo.tbl_log format nul -n -f abc.xml -T".
The table schema like this:
-----------------------
id              int  identity(1,1),
create_time     datetime,
type            nvarchar(16),
job_id          nvarchar(16),
message         nvarchar(1024)
----------------------
So, the format file will like this:
----------------------
8.0
5
1       SQLINT        0       4       ""                        1     id              ""
2       SQLDATETIME   1       8       ""                        2     create_time     ""
3       SQLNCHAR      2       32      ""                        3     type            SQL_Latin1_General_CP1_CI_AS
4       SQLNCHAR      2       32      ""                        4     job_id          SQL_Latin1_General_CP1_CI_AS
5       SQLNCHAR      2       2048    ""                        5     message         SQL_Latin1_General_CP1_CI_AS
----------------------
According MSDN description, we can set some attributes to zero to indicate which column we want to skip. Just like:
----------------------
2       SQLDATETIME   0       0       ""                        0     create_time     ""
----------------------
But it didn't work when I following this. I don't know why it didn't work, may be my source file with invalid format or my use it with a wrong way. Anyway, I got an another way to resolve this problem.

3. The easiest and workable way is put a whitespace in the identity column. SQL server will ignore it. My source file like this:
----------------------
  - 2009/11/12 23:11:12 - Information -   - Server time zone information: id="GMT-07:00",useDaylight="false" 
  - 2010/05/06 07:01:19 - Information - Job00000001 - Incremental backup
----------------------
The SQL statement is:
----------------------
begin transaction T1
bulk insert d2d.dbo.tbl_log from 'G:\\testSQLBulk.txt'
with (
    FIELDTERMINATOR =' - ',
    ROWTERMINATOR ='\n',
)
commit transaction T1
---------------------

Finally, I found the BULK INSERT command isn't a transcation operation. So, we should create custom transcation when using it.

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值