Solution for hours long export

Bernie Fitzsimmons

We recently exported our annual appeal mailing list and it took 20 hours. Solutions for how to improve the process welcomed. Queue module? How to use the Queue module in Blackbaud Hosting Services

JoAnn Strommen

@Bernie Fitzsimmons I have used Queue, can't say for anything quite that large. With break between BB and MS, set up changed a little but has worked okay for us. Still have some issues with it not running. Co-worker did case with support but their suggestions did not fix.

Definitely faster than running during business hours on your open session.

Dariel Dixon

@Bernie Fitzsimmons You need to give some specifics here. How many records did you export? How many fields? Are you hosted or self-hosted?

I'm thinking that there may have been an issue with the query if it took quite that long. No way to tell for sure.

Marie Stark

@Bernie Fitzsimmons I usually break it up into multiple exports and combine them in Excel. Obviously not the best solution!

Bernie Fitzsimmons

@Dariel Dixon I exported about 250K donor. Fields are a lot. I had to pull donor address, addressee and salutation, special mailing types attributes, solicit codes and 5 yr AA gift history. Hosted by BB. Thank you.

Dariel Dixon

@Bernie Fitzsimmons With those constraints, that sounds about right. 250K is a lot of records, and the fields you are using (specifically gift information as those can be calculations). Remember that the hosted environment is a shared environment, so computer resources may be limited. I would imagine that you might need to run something like this overnight.

Also, it might be worth considering if all these fields are necessary in a mailing list. I don't know how you might be segmenting, but this might be a bit excessive. If you know how the segments break down, it might be worth exporting them separately.

If you are using a third party to determine your segmentation strategy, I would not give them all of the demographic data until they determine the segments. 5 year gift history is a lot, and I would imagine that part is the biggest reason why the export is taking so long.

Keith Wilson

@Bernie Fitzsimmons I ran into this issue with a much smaller, seemingly insignificant amount of data. I have since learned the issue was trying to export from hosted and save to a local drive. Be sure when you are exporting that you are also saving the file to Blackbaud's Hosted FTP site for your organization. From there you can move the file to your local drive/desktop or wherever. In addition, yes, queue is a great approach because you can do other things while the export is processing--like sleeping if you schedule the queue to run overnight!