Announcing the results of its 2011 state bar member satisfaction survey, the September 2011 issue of the state bar’s public relations mouthpiece, “Arizona Attorney,” trumpeted the headline, “High Satisfaction, Room for Growth.”
In its quick analytical overview of the 2011 survey conducted by Researchers Bruce Merrill, Ph.D., and Tara Blanc, Ph.D., the story written by the two statisticians boasted that the survey “found high levels of overall satisfaction with membership in the organization.”
“the foolish face of praise”
However, before all the self-congratulatory trophies are passed out, there’s another cast that ought to be put on what Emerson calls “the foolish face of praise.”
The headline is actually ‘spin’ – – – a misnomer since the reported results actually indicated that only 26% or 1,115 of the 4,291 respondents said they were “Very Satisfied” with their bar membership.
Almost twice that number or 2,188 members (51% of all respondents) reported they were “Somewhat Satisfied.” And an additional 23% said they were either “Somewhat Dissatisfied” (17%) or “Very Dissatisfied” (6%). All 21,457 bar members were asked to participate and 20% responded.
The meaning of “somewhat.”
Merriam–Webster defines “somewhat” as “in some degree or measure: slightly.” Like calling yourself, “somewhat pregnant,” what does that mean? Or said another way, if I had ever told my late sainted mother and a terrific cook, that I was only “somewhat satisfied” with one of her meals, I would have brought her to tears.
Only in the Arizona Bar’s smug universe does the descriptive moniker “somewhat” becomes synonymous with high achievement. And it’s in that same self-contented world where high satisfaction and slight satisfaction are lumped together to claim ‘victory,’ i.e., “that three-quarters (77 percent) of members” are aglow with their membership and that “Arizona lawyers view the Bar as a healthy organization that is doing a good job in serving them.”
But if the bar’s ordinal satisfaction scale that allows for comparisons of degree is to be ignored when measuring satisfaction, then the same purposeful disregard ought to be used when measuring dissatisfaction. The descriptive adverb “Somewhat” might as well be given the same meaningless gloss when placed in front of “Dissatisfied.”
Indeed, according to Levels of Measurement, in discussing consumers’ satisfaction with something just as meaningful – – – microwave ovens, “the difference between the responses “very dissatisfied” and “somewhat dissatisfied,” is probably not equivalent to the difference between “very dissatisfied” and “somewhat satisfied.” Nothing in our measurement procedure allows us to determine whether the two differences reflect the same difference in psychological satisfaction. Statisticians express this point by saying that the differences between adjacent scale values do not necessarily represent equal intervals on the underlying scale giving rise to the measurements.”
So in truth, when crowing about high achievement, it should be noted that almost as many members were “Somewhat Dissatisfied” and “Very Dissatisfied” as were “Very Satisfied.” And then, of course, there’s that big chunk in the middle that said they were only “Slightly [Somewhat] Satisfied.”
Relationship surveys for a mandatory bar.
The other thing to consider is that businesses and organizations conduct so-called “relationship surveys” to collect input from people (usually customers) that they have an ongoing relationship with. But in the case of a mandatory bar like Arizona’s, the members aren’t “customers” nor are they treated as such. So long as members need to recoup their substantial investments of time, angst and money, first, to become lawyers and then, to keep earning a living, they’re mandated to join. Or if you want to eat, there’s only one grocery store in town.
Consequently, they’re captive joiners not customers. So, in every practical way that matters, it’s nonsensical to ask members to assess their satisfaction with an organization they’re forced to belong to.
The reality then, is that even if 90% of the members surveyed were “Very Dissatisfied,” it really wouldn’t matter. Like Monty Python‘s “Tis but a scratch” Black Knight, there’s not a heck of a lot mandatory bar members can do – – – after having their limbs cut off to spite their torsos.
Improving on perfection.
“When you’re already “excellent in every area,” there’s not a heck of a lot of room for improvement short of next time calling yourself perfect. But then, that begs the question, “How do you improve on perfection?” This is what I concluded back on April 30, 2010 when I blogged that the “AZ Bar drafts up 5 year vision but misses the mark.”
But there is a happy consequence for the bar. Based on the interpretations given the last two member satisfaction surveys and this “Arizona Attorney” story, the bar now has member corroboration of its self-laudatory excellence.
Lip service to efficiency and criticisms.
The final ‘take-away’ from the bar’s survey analysis is to note how the report devalued member suggestions. Members called for greater efficiencies and cost reductions, which dovetailed precisely with member criticisms of high bar dues, overpriced CLE and of paying for services that members don’t use.
But apparently, those dots don’t connect when analyzing the results. Instead, the authors conclude: “In looking at the overall results of the survey, it is important to keep in mind the nature of a mandatory organization and its relationship with its members. Some members, for instance, indicated their resentment that they must join an organization in order to practice their profession; that underlying distaste may have played a role in other assessments members made, such as in regard to mandatory CLE. This sentiment may have overidden other responses, even when a member had positive feelings about the Bar’s programs and services.”
Or as Richard Pryor once said, “Who are you going to believe, Me? Or your lying eyes?”
But begging the question again, “Why bother with a “mandatory organization” satisfaction survey in the first place?
Or in other words, pay no attention, “It’s just a flesh wound.”