Start Validating summative assessment

Validating summative assessment

Kaye frequently participates in safety-oriented evaluations of 510(k) and premarket approval (PMA) submissions, focusing his attention on the adequacy of human factors testing approaches and findings. By contrast, a heterogeneous user population—one that includes users with widely varying characteristics or for situations in which devices are used differently from one facility to the next—might warrant testing in multiple cities. If three cities are good, wouldn't 10 cities be better?

Complexities also stem from the absence of methodologically prescriptive regulations or standards for usability testing.

However, it can leave less-experienced practitioners asking lots of methodological questions in a strained attempt to conduct a proper test.

Common Questions about Usability Testing Some common how-to questions are addressed below with valuable commentary provided by FDA's Ron Kaye, a human factors specialist in the agency's Human Factors Team. The appropriate but potentially frustrating answer is, “it depends.” If the user population is homogenous, reflecting little variability in the dimensions pertinent to device use (e.g., training, native language, procedural approaches), and if patterns of use are relatively consistent, it might not provide much of an advantage to conduct summative usability testing in more than one locale.

For example, a company developing a new product for the United States and Europe might choose to conduct its test in two U. and two European cities that are known among the company's country managers to offer the greatest user diversity.4 Although it may seem that choosing cities spread far apart, such as Boston and Los Angeles, as a pair of research locations would provide different patient perspectives, in reality more closely spaced locations, say Boston and St.