Service Pack 6: A Tour of the Factory

Today we're going on a tour of the factory.  Although this won't feel much like a tour of a factory.  Instead, this is going to feel like a journey through the life of a feature.  For the smart ones, it will feel an awful lot like landing on free parking when you decide to stop reading this drivel and opt for clicking that little "x" to close your browser tab, freeing up your next 5 minutes to do something way more productive.

If you're still with me, thanks.  I'll try to make it worth your while.  Buckle up, we're going to be looking at some constituency codes!

The Feature(s)
Stored-value Constituencies
Mark Constituencies Inactive

How it Began?
It began with a phone call long before I was a Product Manager.  I was working in Customer Support for Blackbaud CRM when I received a call about a query performing slowly.  Through some standard troubleshooting, the user and I were able to determine that including a constituency filter in the query slowed it down quite a bit.  I created a case and filed a bug for our engineering team to investigate the source of this slowness.   I didn't know it at the time, but this problem and I would cross paths again someday.

What Happened Next?
After the initial bug was reviewed, it was determined that some product changes could be made to speed up the query for that particular user, and a very specific fix was delivered in a patch (we delivered fixes in patches back then).  Over time, more of these cases began to arrive at Support, and while the filters were usually different, the result was the same...a slow query, caused by a constituency filter.  The root cause was that many constituencies (example: donor) are calculated on the fly when running a query.  The Product Manager at the time took notice, and decided to investigate a way to speed up these queries for every filter.  The chosen path to resolve this problem was to create a business process that would refresh a set of static values for constituencies, and use those values in query.  This work was scheduled for an early 4.0 SP.  

Back Together Again
Remember when I said that I'd cross paths with this problem again?  Well I did, and it happened when I became a Product Manager.  One of the functional areas that I was charged with managing was Constituents, so this development work quickly became mine to oversee.  I got up-to-speed on the changes and started to prep for the release of the feature. 

• Refresh process to update values.
• Automatic use of faster values in query.
• Minimal user disruption.

What could possibly go wrong?  It turns out that a lot could go wrong, and it was mostly with the 2nd bullet point above.  Conversations with our clients in Discovery were all positive.  Faster queries make people happy.  However, when we tested the proposed solution and dug deeper into how it works, we started to see a lot of uneasy users.  People wanted faster queries, but they wanted to ability to opt into the change.  The reasons were varied, but consistent in that an automatic swap of the values was not ideal.  Here a few of the reasons clients gave against this implementation:

• Adds another upgrade testing task
• Want the ability to still run some queries in real-time, despite the performance hit
• Want to test and see the difference in the performance for myself before deciding to move toward latent data for constituencies

We had some work left to do.

Drawing Board 2.0
With our users firmly implanted in our ear, we decided to explore ways of making this an optional change.  The nice thing was that once we decided that this would not be a forced change, the excitement for the feature returned and we had users eager to help again. With this feedback in mind, we ended up adding a new node to query.  


The Magic Stone
I don't hunt birds with stones.  I'm not even sure that I know anyone that slings stones at birds.  This isn't a comment about the pros and cons of bird hunting...with stones.  No comment.  But if there's one thing that hunting birds with stones has given us, it's a well-known phrase about solving two problems at once.  That's exactly what we did when solved the problem of constituency performance in query.  Since the stored-value table is basically a data warehouse in your transactional database, we wanted to make sure that we weren't processing more constituency records than necessary.  We had known for a while that our administrators wanted the ability to mark constituencies inactive that they no longer (or never) use.  The answer here was to give admins that ability, and when a constituency is marked inactive, to omit it form the refresh business process.  

Bird 1: Stored-Value refresh speed
Bird 2: Useless constituencies getting in the way
Stone: Mark Constituencies Inactive feature



In Closing
That concludes our tour for today.  Look for these changes and a host of others in Service Pack 6.  Documentation on these changes will be available when SP 6 is released.  While there is no gift shop to exit through, there is a comments section below for you to share your comments and questions.  

-Matt

p.s. I'm fully aware that there are pictures of Bigfoot with better clarity than the screenshots that I've included in this post.  Let's just say it was done to preserve the mystique and aura of these fantastic features :)