Focus on Mission, NOT Testing: UAT Kit Primer

Aaron Eberhart
Aaron Eberhart
Seventh Anniversary Participant
in Blackbaud CRM™ and Blackbaud Internet Solutions™ Blog
               Similar to Yoda’s disdain for seagulls poking him in the head (search online if you’re not familiar, 90 million views and counting and one of my son’s favorites), we should find disdain for those things which distract us and take us off our mission to promote GOOD throughout the world universe.  To those things, I say, “Stop it now!”
Still testing Blackbaud CRM Service Packs instead of focusing on your mission?  Stop it now!


               As one of Blackbaud’s former Infinity build, infrastructure, and automation engineers, I learned quickly about the power of automation.  We had several manual processes which were part of our build and patch system.  These manual processes consumed over 50% of our team’s hours.  I had two great managers during my time as a build engineer who quickly taught me to rework and automate those processes because we had more important things to do.  My goal is to get you into this same mindset.  Whether you are the head of CRM for your organization, a CEO, or a Software Developer, this post is for you!

               To kick things off, I’d like to pose the following three questions:

 
  1. Time:  What is the most important thing we can do with our time?
  2. Testing:  What are we really looking for from our testing?
  3. Solutions:  What solutions exist to provide what we are looking for with testing and enable us to focus our time on what is most important?

Time

            What is the most important thing we can do with our time?

               Life comes at us fast!  We are all people with our own priorities:  family, work, life, etc.  We have to find ways to prioritize and focus on the most important things for our time.  It’s been said, “Time is our only currency.”  How should we be spending this currency?  For you, the answer is focusing on the mission.  The mission being: Educating the masses; Feeding and clothing children; Promoting medical research; Preserving historic heritage; and the list goes on and on!  Your mission is not to test software.  It is not to make sure that your daily processes are flowing through the system.  This is our mission.  Our mission is to make sure that your daily processes are flowing through the system.  Our mission is to test the software so you don’t have to.  Our mission is to enable you to do your good!

               At Blackbaud, we took a look at the amount of time we were testing software for release.  We thought to ourselves, how can we free up time to deliver even more?  To deliver more features, fix more bugs, and ship fewer bugs?  How can we have tests which are consistent and repeatable?  Automation was the answer. 

               What you are about to read is true, only the names were changed to protect the innocent:

               We took a set of test cases which took 6 team members 2 weeks to test.  According to my calculator this equates to around 480 person hours of work.  We were running these tests every time we released the software and a subset of the tests a few times in between releases. We have since automated the entire test suite.  The tests now run daily in about 2 hours.  In full disclosure, we do have test failures which must be evaluated and the automated system has to be maintained once in a while, but COME ON!  2 HOURS!  DAILY!  SERIOUSLY!

               Imagine, come on… dream with me for a moment.  Imagine, you are working on upgrading to a new Service Pack and you know within a couple hours if all of your mainline daily processes are going to work.  A couple hours, this is an amazing difference in time!  Now guess what your team members are doing for those 6 weeks?  It’s not testing software!  Nope, it’s refocusing staying focused on the mission.  Yes, it IS that powerful!

 
 

 

TIME SUMMARY:  Your mission is the most important thing you do.  You not having to worry about your software working is the most important thing we do.


Testing

            What are we really looking for with our testing?

               It’s not as nerdy as you might think.  We’re not really looking for a specific business process or a data entry form.  We’re looking for CONFIDENCE.  Confidence is the consolidating factor for all testing, and it brings us to Risk Based Testing.  Risk Based Testing is an industry standard practice which has survived multiple generations of technology.  It focuses your time and energy on those things which are more risky based on its impact and likelihood.  If an area of the software is critical to the mission and used daily, we should thoroughly test it.  If it is extremely edge case and we only use the area for 3 constituents in the database, we probably don’t need as much confidence.  This brings us to customizations.  If your organization is going through the effort of writing a customization, it’s highly likely it will be used a lot and is critical to the mission.  This means you want a high level of confidence your customizations will work every time!

 

 

TESTING SUMMARY:  Determine the level of Confidence you need for the high-risk areas and get busy automating.


Solutions

               Software Engineers, this part’s for you!

            What solutions exist to provide what we are looking for with testing and enable us to focus our time on what is most important?

               To return time to our missions and automate confidence in testing, we have written, open sourced, and updated the Blackbaud User Acceptance Testing (UAT) kit.  This is the same framework we use internally for automated UI testing in Blackbaud Enterprise CRM (it could be leveraged for other web applications too!)  The summary is: it uses Gherkin-based scenarios (common language) to drive C# step definitions, tied in with the Selenium Chrome driver to drive UI automation.  These are all industry standard technologies which mimic exactly what the end user does in the system!

               At the bottom of this post are some helpful links to get you started.  The UAT kit was designed to automate testing in Blackbaud CRM.  It was designed to automate testing of customizations and extensions.  It was designed to return time to your mission and confidence in your tests!

How can you get started?

               The UAT kit has been written in such a way as to allow the testing of any part of CRM, whether it is out of the box or custom, by using a common set of underlying methods. There is something of a learning curve for the framework, but most tests can be written using only three methods which emulate what we do on web pages at the most basic level:
  • Click – WaitClick
  • Check something exists – GetEnabledElement
  • Place text into a html form element – SetTextField
These all exist in the kits base object – BaseComponent like so:

These methods are derived, via inheritance, into the other CRM system representations (Dialog, Panel etc.).  In terms of the UAT kit itself, you can be up and running with very little effort.  In fact, due to the complexity of the UI layout in CRM, the main point of difficulty is targeting elements by using XPath.  Because CRM is a single-page application, the XPaths can be very complex and are typically the sharpest part of the learning curve.

Where can you see some complex examples?

               We’re so glad you asked!  The UAT kit does require forethought about a few things.  What level of abstraction are we going to have in our Gherkins or Step Definitions?  Are our Gherkins going to be more Declarative or Imperative?  When Blackbaud started seeing questions like these, we realized: 1) we’ve got several engineer years of experience, and 2) we’ve got several tests we know work well.  So, we took a core sample of those tests which are generic for any client. We then put them into a core package and open sourced them.  These tests cover everything from adding Individual Constituents, to adding Revenue, to generating Batches.  By open sourcing these tests we hope your organization can use them as a starting point.  Take our tests, modify them to work for you and for your customizations, add them to your library, run them on upgrades, and focus on your mission!  The source code can be found via the links below.

Do we have any guidelines around writing tests?

               We do, the UAT kit website has some of our internal guidelines.  Think of them more like gutter guards when you’re at a bowling alley:  They keep you centered and heading for the pins!

Disclaimers and Links

               While we use the UAT kit internally with every build and every release, we must state that it is not an officially supported Blackbaud product and it does not have an official release cycle.  We have been updating the source code and nugets with each CRM Service Pack release, but we do not guarantee this.

So Long, Farewell, Auf Wiedersehen, Adieu

               Well, that's a wrap!  As a reminder, the UAT kitwas designed to build confidence in testing and return time to the important mission, your mission!  Take advantage of it, for your mission's sake!

Categories