Due to the nature of the mystery shop methodology, which relies on human judgment, the element of subjectivity must be recognized and any biasness must be minimized from the start. For example to the question, “Is the fitting room tidy?” Some mystery shoppers may interpret yes or no depending on the individual’s views. While iSN will diminish some of the biasness substantially, an individual’s preferences on what is and what is not acceptable will inherently affect the shopper’s assessment on areas such as cleanliness, music level and friendliness of employees.
Sometimes, the findings may not be representative of the actual experience on the ground. iSN found that a reliable comparison can be obtained from a well-designed study. Each mystery shopping encounter is valid since it is a snap shot of the service experience at one moment in time, and as such is not trying to represent the population of all such experiences. iSN concluded that generalizability is not an issue and the data collected can be analyzed within the context it was collected.
Several factors exist to affect the validity of the results obtained from a mystery assessment. First, the views of the mystery shopper may not match what a typical customer would notice during the service transaction. Second, the shopper may have forgotten to assess one or more items in the audit checklist or simply failed to record a segment of the service experience accurately due to the time lag between the audit and recording. Besides inaccuracy in mystery shopping by shoppers, iSN holds a quality hand on both reliability as well as validity.
In summary, mystery shopping can provide an in-depth analysis of a service experience, but that analysis is narrow since it is restricted to a professional who takes on the role of customer. iSN opined that it is not possible to make accurate comparisons between the reliability and validity of different service quality measurement methods. iSN provided accurate results especially when mystery shoppers are not well prepared and trained.