Constituent search not updating addresses

Chris Jones

Hi there,


I'm using Constituent Search to display a list of constituents in various areas of the UK. I don't get many results but that's just the data. An example is:
https://api.sky.blackbaud.com/constituent/v1/constituents/search?search_text=London which returns 2 results.


I'm also using Patch address to update users addresses, including city, county, country and postal code. The trouble is, when I update these addresses, the search results don't change and my new saved addresses don't show up, depsite being saved successfully.


Any ideas?


Thank you.

Steven Cinquegrana

Hi Chris,


Can you describe your process/flow? Sometimes, the wrong IDs get used, so the wrong records get updated. It's especially common to mix up the constituent_id and lookup_id for example.


And are you allowing for any latency which can affect list type data? Eg, the Constituent (List) endpoint states: This endpoint returns data with an average latency of about 30 minutes.


Cheers,

Steve Cinquegrana | CEO and Principal Developer | Protégé Solutions

 

Chris Jones

Hi Steve,<br/><br/>Thanks for your reply, much appreciated.<br/><br/>I've checked my code and I'm definitely using constituent_id, not lookup_id. It's a bit loong-winded to show here though. In any case, you can see below that the database has the updated record correctly.<br/><br/>The latency could be an issue, as you say, although I updated this nearly 24 hours ago.<br/><br/>Here's my output for one of the users I've updated recently. It has an address in London. It's taken directly from the API endpoint reference testing portal, so nothing to do with my code:

GET <a href="https://api.sky.blackbaud.com/constituent/v1/constituents/900">https://api.sky.blackbaud.com/constituent/v1/constituents/900</a><br/>{<br/>  "<span>id</span>": <span><span>"900"</span></span>,<br/>  "<span>address</span>": <span>{<br/>    "<span>id</span>": <span><span>"993"</span></span>,<br/>    "<span>address_lines</span>": <span><span>"Police Headquarters\\r\\nSt Peters Port"</span></span>,<br/>    "<span>city</span>": <span><span>"Stratford"</span></span>,<br/>    "<span>constituent_id</span>": <span><span>"900"</span></span>,<br/>    "<span>country</span>": <span><span>"United Kingdom"</span></span>,<br/>    "<span>county</span>": <span><span>"London"</span></span>,<br/>    "<span>do_not_mail</span>": <span><span>false</span></span>,<br/>    "<span>formatted_address</span>": <span><span>"Police Headquarters\\r\\nSt Peters Port\\r\\nStratford\\r\\nLondon\\r\\nE10 5QD"</span></span>,<br/>    "<span>inactive</span>": <span><span>false</span></span>,<br/>    "<span>postal_code</span>": <span><span>"E10 5QD"</span></span>,<br/>    "<span>preferred</span>": <span><span>true</span></span>,<br/>    "<span>type</span>": <span><span>"Business"</span><br/>  </span>}</span>,<br/>  "<span>date_added</span>": <span><span>"2019-05-02T16:16:07+00:00"</span></span>,<br/>  "<span>date_modified</span>": <span><span>"2020-05-19T17:08:00.259+01:00"</span></span>,<br/>  "<span>email</span>": <span>{<br/>    "<span>id</span>": <span><span>"1983"</span></span>,<br/>    "<span>address</span>": <span><span>"tom-police@police.com"</span></span>,<br/>    "<span>constituent_id</span>": <span><span>"900"</span></span>,<br/>    "<span>do_not_email</span>": <span><span>false</span></span>,<br/>    "<span>inactive</span>": <span><span>false</span></span>,<br/>    "<span>primary</span>": <span><span>true</span></span>,<br/>    "<span>type</span>": <span><span>"Email"</span><br/>  </span>}</span>,<br/>  "<span>gives_anonymously</span>": <span><span>false</span></span>,<br/>  "<span>inactive</span>": <span><span>false</span></span>,<br/>  "<span>lookup_id</span>": <span><span>"888"</span></span>,<br/>  "<span>name</span>": <span><span>"Guernsey Police"</span></span>,<br/>  "<span>phone</span>": <span>{<br/>    "<span>id</span>": <span><span>"1984"</span></span>,<br/>    "<span>constituent_id</span>": <span><span>"900"</span></span>,<br/>    "<span>do_not_call</span>": <span><span>false</span></span>,<br/>    "<span>inactive</span>": <span><span>false</span></span>,<br/>    "<span>number</span>": <span><span>"07855 986568"</span></span>,<br/>    "<span>primary</span>": <span><span>true</span></span>,<br/>    "<span>type</span>": <span><span>"Phone"</span><br/>  </span>}</span>,<br/>  "<span>type</span>": <span><span>"Organization"</span><br/></span>}

If you look at this search response, again directly from the API endpoint reference testing portal, you'll see my constituent not showing, and two showing that are not based in London. <br/>GET <a href="https://api.sky.blackbaud.com/constituent/v1/constituents/search?search_text=London">https://api.sky.blackbaud.com/constituent/v1/constituents/search?search_text=London</a><br/>

{<br/>  "<span>count</span>": <span><span>2</span></span>,<br/>  "<span>value</span>": <span>[{<br/>    "<span>id</span>": <span><span>"2122"</span></span>,<br/>    "<span>address</span>": <span><span>""</span></span>,<br/>    "<span>deceased</span>": <span><span>false</span></span>,<br/>    "<span>fundraiser_status</span>": <span><span>"None"</span></span>,<br/>    "<span>inactive</span>": <span><span>false</span></span>,<br/>    "<span>lookup_id</span>": <span><span>"2110"</span></span>,<br/>    "<span>name</span>": <span><span>"John Lyon"</span><br/>  </span>}, {<br/>    "<span>id</span>": <span><span>"1068"</span></span>,<br/>    "<span>address</span>": <span><span>"111 Kingsway Park\\r\\nDaveyhulme\\r\\nManchester\\r\\nM41 7ED"</span></span>,<br/>    "<span>deceased</span>": <span><span>false</span></span>,<br/>    "<span>email</span>": <span><span>"lyndon.levett2@ntlworld.com"</span></span>,<br/>    "<span>fundraiser_status</span>": <span><span>"None"</span></span>,<br/>    "<span>inactive</span>": <span><span>false</span></span>,<br/>    "<span>lookup_id</span>": <span><span>"1056"</span></span>,<br/>    "<span>name</span>": <span><span>"Lyndon Levett"</span><br/>  </span>}]<br/></span>}<br/><br/>Thanks,<br/>Chris

Steven Cinquegrana

Hi Chris,


Sorry, but the strange animal that Constituent Search* is doesn't include county (or state, province, etc) in its search as far as I know. Only first name, last name, preferred name, former name, alias, email address, phone number, address and lookup ID are supported. (I believe "address" here refers to address lines, not the whole address block but I could be wrong.)


But I'm a bit confused by your initial question now; I thought the issue was that the address wasn't updating, rather than that Constituent Search wasn't returning the records you expect. Or do you mean that if you change an address - eg, to London from Manchester - that you expect the record with the changed address to subsequently appear in the search results? If so, that would be for the reason above, I think. You could prove this by changing the address to something like "6 London Lane", etc and then retrying your search (allowing for latency) using the "London" keyword.


Let us know how you fair.


Cheers, Steve


* Do a search of the Community for "Constituent Search" and you'll find some interesting comments ...

 

Chris Jones

Hi Steve,


Yes it's certainly a strange beast!


Sorry for the confusion - the address was always updating, I meant that the search returns didn't seem to be updating.

I have changed my code so that address_lines gets updated as well, but it still has no effect on the search returns. I can't see any other address fields to update so at a loss.


Also, I get a strange problem when looking at the developer portal - it keeps logging me out and says your account already exists. I get stuck in a loop and can't view any endpoints.


Finally,

Steven Cinquegrana

I think I've found a(nother) problem with Constituent search.


I used the Try It on the Constituent Endpoint Reference to search for "London" (our test data will be different to your own) and the first record was:

 

{<br/>  "<span>count</span>": <span><span>8</span></span>,<br/>  "<span>value</span>": <span>[{<br/>    "<span>id</span>": <span><span>"12453"</span></span>,<br/>    "<span>address</span>": <span><span>"71 Elmsmere Rd.\\r\\nNahant, MA  01908"</span></span>,<br/>    "<span>deceased</span>": <span><span>false</span></span>,<br/>    "<span>fundraiser_status</span>": <span><span>"None"</span></span>,<br/>    "<span>inactive</span>": <span><span>false</span></span>,<br/>    "<span>lookup_id</span>": <span><span>"11976"</span></span>,<br/>    "<span>name</span>": <span><span>"London O. Panas"</span><br/></span></span>

(matching on first_name.)


Then I tried a search on "Nahant" but the record didn't show up in the results. This was expected.


Then I tried a search on "Elmsmere" but the record didn't show up in the results. I didn't expect this.


Then I tried a search on "71 Elmsmere" and, lo and behold, the record was there!


So it appears that Constituent search holds yet more surprises for us. It seems that partial string matching hasn't been implemented for address, though it has for some other fields such as last_name (a search using "anas" returned the record) but not first_name ("ondo" didn't).


I'm stumped. This is now the weirdest endpoint in the whole of SKY API and the strangest implementation of a text search I've ever seen. I've asked previously for some doco on how the search is implemented and been met with radio silence, so we're really searching in the dark with this thing, I'm afraid.


I'm not going to be able to run any more tests on this, Chris; you'll have to log a case if you think there is actually a problem somewhere. I'm just a lowly Community participant, not Blackbaud support.


Cheers, Steve

 

Chris Jones

Thanks Steve,


You've been a great help. Much appreciated. Some crazy results you found there and I agree it's an odd search mechanism. I will try to open the question to the wider community and Blackbaud themselves.


Is there anyone from Blackbaud support who can offer any more insight into the search and how I can use it to retrieve results based on town, county etc?


Many thanks,

Andy

Anthony Gallo

Hey Everyone,


We looked thru the code to clarify what is going on here. 


We will only search for addresses if the search term looks like an address.  For instance the search term would need to start with digits and then have non digits that follow "65 Fairchild Street".  If we see a term like this we will search address lines only (not the city, state or any other address field).  The search function has a pretty major performance hit so opening this up would greatly affect the experience.  


If we think there is a better option and if you all have ideas we can definitely look into improving the way this works.  We will have to keep the performance in mind with any changes we make.


Anthony

Steven Cinquegrana

Anthony‍, you've possibly seen comments in the Community by myself and others expressing frustration with this endpoint. It's quite a bizarre, counter-intuitive and limiting piece of work.


Further, there is a paucity of information about how it works internally so the results can be bewildering at times.


The example here regarding address lines, city, county/state, etc is but one of many. I don't think anyone would expect that a search which includes address would 1) not include city or state and 2) not find a street without the prefixed numbers. I don't know of any other search utility that works like that.


Also, there is this strict_search flag that only limits the search to non-phonetic searching. What? Surely if I want a phonetic search I would set a flag such as phonetic_search; strict implies that wild-carding isn't implemented, so you would only find Constituent ID 1234567890 if you searched for "1234567890", not "1", "2345", etc, for example. But this isn't the case. (And it makes it very convoluted to find low-numbered Constituent records, as mentioned here!)


I don't understand why search has been implemented this way when there are much simpler, efficient and more standard implementations.


You could limit fields by specifying one or more search_field query arguments; strict_search and phonetic_search would be optional with a loose search being the default; etc. 


For example, I would expect to be able to execute searches like this:

https://api.sky.blackbaud.com/constituent/v1/constituents/search?search_text=3&search_field=lookup_id&strict_search=true

Strict search for the string "3" in only the lookup_id field and returns only the Constituent record with lookup ID 3 if it exists.

or

https://api.sky.blackbaud.com/constituent/v1/constituents/search?search_text=Cam&search_field=city

Loose search for the string "Cam" in only the city field and returns records with Camperdown, Camden, Cameron, etc

or

https://api.sky.blackbaud.com/constituent/v1/constituents/search?search_text=walt&search_field=first_name&search_field=last_name&search_field=email

Loose search for the string "walt" in the first_name, last_name and email fields and returns records with Walter Becker, Iris Dewalt, d.swalthen@somewhere.com.


Surely this is clearer, faster and less demanding of the back-end DB and results in search records that I’m actually looking for.


Cheers, Steve