相信开发过Sql Server SSIS的人都遇到过在数据流中数据源假死的问题，特别是Excel Source特别容易假死，当job执行到数据流中的Excel Source时，既不报错也不执行，也没有任何日志，Job就一直死在这里了。。。然后当你停掉假死的Job，重新执行Job，Excel Source又莫名其妙的顺利执行完成了。。。毫无疑问这是SSIS中的一个Bug，但是问题是如何避免这个Bug。
下面是我收集到的一些关于SSIS中数据流的数据源假死信息，供大家参考：
解决办法一，将数据流中数据源连接器的Delay Validation一定要设置为True：
I have a Data Flow Task that is hanging on excecution. The flow is simple, makes two queries to different tables (Both with a couple of joins), then sorts and merges the otuputs through a common id, adds a static column to all the records, saves the row count in a user variable for later use and finally inserts into a table on another DB. We are using OLE DB Sources and Destination. Source is MSSQL 2000 and Destination is MSSQL 2012
Symptoms:
- When excecuting, the Data Flow gets the usual yellow "running" icon. However when you double click to see the Data Flow, non of the elements have any yellow, red or green mark.
- This goes on for long periods of time, at first it lasted around 20 minutes, after that it started getting longer or simply not returning at all.
- Output shows: Information: 0x40043006 at Load Sandbox Table, SSIS.Pipeline: Prepare for Execute phase is beginning. Information: 0x40043007 at Load Sandbox Table, SSIS.Pipeline: Pre-Execute phase is beginning. And nothing more until the excecution is stopped.
- Yes, this has worked before. And yes, we have used a single query (in a stored procedure) to do this ETL but we wanted to migrate all the steps to SSIS.
Failed solutions:
- There are no lookups.
- Default buffer size for the task flow was increased to 40485760 then to 80971520.
- Default buffer max rows for task was set to 1000000.
- Delay Validation was set to True for the task.
- All elements inside the task were set Validate External Data to False.
- Both queries had: SET FMTONLY OFF; SET NOCOUNT ON; added at the beggining.
- Both queries had MAXDOP set to 1.
- Setting project's Run 64 bit Runtime to False.
- Changed destination load from Table or View to Table or View - Fast load with no locks or constraints.
- Set rows per batch to 1000 for fast load.
- Some work arounds propose to separate the task flow into two or more task flows. But this is not possible since what we need to do is a merge of the information found on both source queries.
Extra bits: I really hope someone can help me. I am fairly new to SSIS, this is the first time I use it. I usually work with Pentaho for my ETL but the client needs the solution to be implemented on SSIS. I've been battling with this issue for a couple of days now and I'm starting to run out of ideas to solve it.
When ran through the command line it gets stuck too and I get the following output:
Progress: 2013-03-19 14:36:26.21 Source: Load Sandbox Table Validating: 0% complete End Progress Progress: 2013-03-19 14:36:26.21 Source: Load Sandbox Table Validating: 12% complete End Progress Progress: 2013-03-19 14:36:26.22 Source: Load Sandbox Table Validating: 25% complete End Progress Progress: 2013-03-19 14:36:26.22 Source: Load Sandbox Table Validating: 37% complete End Progress Progress: 2013-03-19 14:36:26.23 Source: Load Sandbox Table Validating: 50% complete End Progress Progress: 2013-03-19 14:36:26.25 Source: Load Sandbox Table Validating: 62% complete End Progress Progress: 2013-03-19 14:36:26.25 Source: Load Sandbox Table Validating: 75% complete End Progress Progress: 2013-03-19 14:36:26.25 Source: Load Sandbox Table Validating: 87% complete End Progress Progress: 2013-03-19 14:36:26.25 Source: Load Sandbox Table Validating: 100% complete End Progress Warning: 2013-03-19 14:36:26.26 Code: 0x80047076 Source: Load Sandbox Table SSIS.Pipeline Description: The output column "ITEM_OID (1)" (47) on output "Merge Join Outp ut" (28) and component "Merge Join" (11) is not subsequently used in the Data Fl ow task. Removing this unused output column can increase Data Flow task performa nce. End Warning Progress: 2013-03-19 14:36:26.27 Source: Load Sandbox Table Prepare for Execute: 0% complete End Progress Progress: 2013-03-19 14:36:26.27 Source: Load Sandbox Table Prepare for Execute: 12% complete End Progress Progress: 2013-03-19 14:36:26.27 Source: Load Sandbox Table Prepare for Execute: 25% complete End Progress Progress: 2013-03-19 14:36:26.27 Source: Load Sandbox Table Prepare for Execute: 37% complete End Progress Progress: 2013-03-19 14:36:26.27 Source: Load Sandbox Table Prepare for Execute: 50% complete End Progress Progress: 2013-03-19 14:36:26.27 Source: Load Sandbox Table Prepare for Execute: 62% complete End Progress Progress: 2013-03-19 14:36:26.27 Source: Load Sandbox Table Prepare for Execute: 75% complete End Progress Progress: 2013-03-19 14:36:26.27 Source: Load Sandbox Table Prepare for Execute: 87% complete End Progress Progress: 2013-03-19 14:36:26.27 Source: Load Sandbox Table Prepare for Execute: 100% complete End Progress Progress: 2013-03-19 14:36:26.31 Source: Load Sandbox Table Pre-Execute: 0% complete End Progress Progress: 2013-03-19 14:36:26.31 Source: Load Sandbox Table Pre-Execute: 12% complete End Progress Progress: 2013-03-19 14:36:26.31 Source: Load Sandbox Table Pre-Execute: 25% complete End Progress Progress: 2013-03-19 14:36:26.34 Source: Load Sandbox Table Pre-Execute: 37% complete End Progress Progress: 2013-03-19 14:36:45.69 Source: Load Sandbox Table Pre-Execute: 50% complete End Progress
After that it freezes again.
SOLUTION (Posting this here because I can't answer my own question for another 5 hours, I'll do it when I'm allowed to.) I finally got it. It turns out there is an issue with the validation, but not only SSIS elements go through that validation, as stated in the fourth failed solution of the question. The CONNECTIONS also get validated and have their own Delay Validation property, which needs to be set to true. After that the excecution time went from 40+ mins or no running to less than a minute for the full process (This is just one step of a much bigger process) I hope people with this same problem can find this solution easily because there's a lot of people running into this problem and nearly no solutions posted online.
In a nutshell: Check that all your elements involved in the task, including the DB connections have Delay Validation Property set to True.
解决办法二，将Oledb Source中的表或视图改为Sql Command
Fixed my problem by changing the Data access mode to SQL Command and pasting my view into the SQL command text in the OLE DB Source.
解决办法三，将数据流中Source和Destination控件属性中的ValidateExternalMetadata属性设置为False
引用老外的原话
We already had our Delayed Validations set to True and couldnt/didnt want to change it to a SQL statement.
I came across ValidateExternalMetadata in the dataflows which I changed to False and that seem to work like a champ.
