Constituent List endpoint missing records?

Hello John T,

Please use web view and build list for all constituents. Export the constituents list from web view. Also please use database view query the system.

John T,

I pulled all constituents list using APIs and never experienced this kind of issue. Also pulled information using database view using queries.

@John T we are also having this issue - so far it seems to be where the user is attempting to use a fixed list but we don't know if it's limited to that, it could be coincidental that is the case and I thought I saw some documentation that fixed weren't supported but then I would think we'd get no results not almost all but with thousands dropped. In our case, it is sequential segments when sorted alpha. I've been digging in and was torn between it being a fixed list root issue and/or how our dev is doing pagination. Have you had any further progress on your end?

@John T
I use the Constituent List API to get all constituents, we have ~450K constituent records in RE and were able to get all of them and store them into our data warehouse on Azure.

Here's a few things that you should note.

• By default, a single call to the API only return 500 records, you can change that with the limit request parameter, I always use 5000 which is the max
• In order to not “miss” any records, which can happen as “live” data is being changed constantly, I strongly recommend also using one of the following parameters: last_modified, date_added, or sort parameter, which is a “guarantee” that you will get “next link” to retrieve next set of 5000 records until all records are retrieved. That guarantee is done via the sort_token parameter that comes with the next_link parameter
• here's a quick look at how I retrieve all constituent record:
  • before getting into the loop, my inital URL is: https://api.sky.blackbaud.com/constituent/v1/constituents?limit=5000&include_inactive=True&include_deceased=True&sort=date_added
  • inside loop, the URL uses the returned “next_link” field, which looks like this on my 2nd call: https://api.sky.blackbaud.com/constituent/v1/constituents?sort_token=ew0KICAiRGF0ZSI6ICIyMDEwLTAzLTEzVDA5OjQ2OjEzLjM4LTA1OjAwIiwNCiAgIklkIjogIjUxMDYiLA0KICAiQ29sdW1uIjogNDUsDQogICJEaXJlY3Rpb24iOiAwDQp9&limit=5000&include_inactive=True&include_deceased=True

@John T
I think I was able to identify your database and it looks like there is a high number of non-constituent records which are not returned in the constituent list endpoint. These records are brought back in the single constituent endpoint with a “is_constituent” field to signify if they are a full constituent or a non constituent. This is a recent update we pushed to just the single record call and we have not yet added it to the list call.

Are there plans to add this to the list endpoint in the future?

@Oscar Ira
I have a bunch of power autoamte flow that does data pull for various records in RE and FE, however, they are not templates.

What is your purpose of getting the data? depending on your use-case, saving to SharePoint may not be best.

@Oscar Ira
access token and refresh token is taken care of when you use Blackbaud's connector, you do not need to worry about it.

The process you explain is wasting more resource than needed.

You can directly pull data from RE NXT in Power Automate and save into Azure Data Factory, instead of saving to SharePoint as csv file, THEN process the csv into azure data factory.

@Oscar Ira
I don't save my data in azure data factory, I save into azure SQL server database.