Is User Satisfaction a Hobgoblin?

This paper is in response to William S. Cooper: "On Selecting a Measure of Retrieval Effectiveness."* Whereas Cooper considers (subjective) satisfaction of a user as the utility of an individual search (and then computes system utility as the average search utility), this paper argues that improvement in the task performance of the user is a much more appropriate measure of utility. From this it is shown that recall, while unimportant or even harmful in many search situations, is of vital importance in others. This is in contrast to Cooper's view that recall by itself is not a meaningful measure of system performance at all.