Rating Summary Issues with adding ghost scores and disappearing scores in the score summaries

Grant Harrison
Grant Harrison
Third Anniversary Kudos 1 Name Dropper Participant
in Blackbaud Enrollment Management System™

Wanted to see if anyone else in the community noticed this about their rating summaries:

ce61ad31de1eda5e1768c4f1f3c77270-huge-im

Protecting Student Data there, but I wanted to show that in our rating summary, the category of Academic Rigor is 0-3 for that matrix we created.

Somehow it seems that a 6 can appear for us (we're guessing a reviewer with it open in 2 windows but not sure), throwing off rating summary totals by 3 points when we evaluate candidates. We saw it a few times on Selection Day. We luckily run a script that makes sure scores are in alignment outside of BB, but seems very troubling for other institutions that don't have those resources. Support called it a glitch when I reported it, but curious on thoughts.

I also notice some of the rating summaries disappeared while having a submitted status, so when I impersonate the reviewers, the candidate review now looks like this on a submitted student's required fields when I impersonate the reviewer in COMMITTEE REVIEW:

6e4fd6211dffd84c26ae82aed9063f58-huge-im

Again, we're lucky we pull numbers out and place in our warehouse. It seems to be random, but I notice multiple missing scores now that have a submitted status by the reviewer in multiple review groups.

Comments

  • Kelsey Huijgen
    Kelsey Huijgen Blackbaud Employee
    Tenth Anniversary Kudos 2 Name Dropper Facilitator 1

    Hi @Grant Harrison! I just sent you an email about this. Happy to look into it to see if we can pinpoint what's happening.

    Thanks!
    Kelsey Huijgen, Product Manager

  • Troy Burki
    Troy Burki
    Fifth Anniversary Kudos 2 Name Dropper Participant

    @Grant Harrison related to the second issue you mentioned - something very similar happened to us around 2-3 years ago.

    Basically, I had made a copy of the rating matrix we were using for committee review to experiment with something for our next admissions cycle. When I removed one of the categories in the copy of the existing matrix, it dumped all of the ratings and individual category summaries from the committee review that was using the original matrix (600+ students), leaving only the summary narrative at the end of each review. If I remember correctly, after several weeks of investigation there was essentially no resolution other than it was a glitch that wouldn't happen again.

  • Grant Harrison
    Grant Harrison
    Third Anniversary Kudos 1 Name Dropper Participant

    @Troy Burki
    Thanks, that's good information. In our case though, these ratings are from March 10, and we haven't made any changes to the ratings matrixes yet. They're just gone. And completely random too. No review group is 100% missing, and the evidence of the submitted statuses still exist in the candidate's record, the scores themselves are just gone

  • Grant Harrison
    Grant Harrison
    Third Anniversary Kudos 1 Name Dropper Participant

    @Kelsey Huijgen Thanks, Kelsey. I emailed you the details. I'll let everyone know here what you end up finding out about Blackbaud doubling scores in the rating summary.

  • Grant Harrison
    Grant Harrison
    Third Anniversary Kudos 1 Name Dropper Participant

    @Kelsey Huijgen
    Hello all,

    Wanted to provide an update I haven't heard anything yet from the ticket.

  • Grant Harrison
    Grant Harrison
    Third Anniversary Kudos 1 Name Dropper Participant

    @Grant Harrison
    for anyone following this error, I've still got an open ticket and haven't gotten an update since April (over 3 months).

    Our team will discuss options, likely using rating summaries with our in house scripting to double check what Blackbaud cannot (since we still can't even get rating summary totals), but this is not good if rating summaries are making ghost scores we can't explain. I'm guessing maybe reviewers have 2 tabs open since I'd say it's occurring at a rate of 1 in 200, and we'll advise that at next selection, but this is the kind of thing that can get us sued, so hopefully Blackbaud is taking this seriously.

Categories