SSIS package issue for data warehouse

Todd O
Todd O
Tenth Anniversary Kudos 1 Name Dropper Facilitator 1
in Blackbaud CRM Developer Discussions
I have a new custom package that is pulling from out of the box warehouse tables vs from the OLTP database. All custom packages all run at the end of the refresh and I was expecting to my pull in this new package to include the rows inserted ahead it but I am only getting the rows that existed prior to the start time of the ETL refresh. I don't see this with our other custom packages where they are in a sequence and build on each other, i.e. a detail level table gets upserted and the a summary tables that runs after sees those changes. The non-custom tables build on each other as well, constituent is refreshed early and then later packages see the changes. I'm wondering if I have a bad setting in one of my containers or tasks but am not sure what. If anyone has seen this before or has any insight I would appreciate it.


Thanks,

Todd

Comments

  • Mitchell Gibbs
    Mitchell Gibbs
    Fifth Anniversary Kudos 1 Name Dropper Participant
    It sounds like you may be reading from BBETL_DB_CONN_OLTP vs BBETL_DB_CONN_DW. The OLTP connection is going against a snapshot, which sounds like what you're seeing. But that's only OLTP, not the DW connection. Odd. Packages should definitely have no trouble dealing with data in the warehouse from prior packages. You might want to recheck the connection being used?
  • Todd O
    Todd O
    Tenth Anniversary Kudos 1 Name Dropper Facilitator 1
    Thanks for the response. The read is using BBETL_DB_CONN_DW, its just behaving like its running off a snapshot. My workaround has been to run the ETL refresh twice, once for the normal run and again so this new custom package can see the changes from the first run. If the warehouse doesn't run off a snapshot its got to be something in my package but I'm not sure what settings to check.

Categories