Study's Findings Deserve Clarification

The Oct. 24 issue of Credit Union Journal reported on a study that compared phone satisfaction surveys to mystery shopping. The study was, not surprisingly, conducted by Prime Performance, a phone survey vendor. Rather than focus solely on the benefits of phone surveys, the study questioned the validity of mystery shopping. As one of the country's leading providers of mystery shopping services to credit unions in the U.S. and Canada and a member of the Mystery Shopping Providers Association, I'd like to refute several misrepresentations of mystery shopping contained in the report.

The Prime Performance study's main contention was that mystery shoppers cannot gauge customer satisfaction. The fact is that mystery shopping is not intended to be a measure of customer satisfaction. The primary purpose of mystery shopping is to provide detailed feedback on employee behavior. This allows credit union managers to coach employees to improve specific behaviors that contribute to high levels of service. A satisfaction survey is rarely actionable because it relies on general perceptions of the credit union overall and does little to inform a branch manager as to how to improve the hundreds of factors that contribute to the member experience. Decades of member satisfaction surveys have failed to provide any correlation between member satisfaction and credit union growth. Credit union growth is spurred by the development of a strong sales and service culture which includes training, coaching, measurement of actual behavior, and accountability. Mystery shopping is a tool that supports those endeavors.

The Prime Performance report also contends that mystery shoppers do not represent typical customers and are too far and few between to provide levels of service over a wide range of employees, dates, and times. This may have been true of traditional shopping programs, but mystery shopping has evolved to become a reliable gauge of the member experience. For example, our company recruits and trains a credit union's own members to shop. We ensure that shoppers reflect the credit union population as a whole. Moreover, we ask members to conduct their shops as they go about their daily business so that shops capture the actual member experience and go undetected by employees. Finally, by utilizing a credit union's own members as shoppers we are able to deliver a high number of shops at a reasonable cost so that all delivery channels, functions, and employees are shopped regularly.

The fact is that all forms of research have advantages and disadvantages. For example, phone surveys cannot provide a great depth of information because most consumers are not willing to complete long surveys over the phone. In addition, likely responders to phone surveys do not necessarily represent the demographics of the credit union membership as a whole. The primary difference between phone surveys and mystery shopping, however, is that the first is usually quantitative while the second is qualitative. Both types of research have their place in a credit union's member research toolbox.

Constance Anderson, President
MemberShoppers.Com, New York City