The already difficult knowledge acquisition process is complicated when the expert system being developed requires interaction with multiple experts. In this tutorial for novice knowledge engineers and managers we discuss some considerations involved in using multiple experts, including (1) deciding when multiple experts may be necessary, (2) eliciting knowledge from multiple experts individually or in small groups, and (3) knowledge engineer capabilities and preparation. Next, we present three specific group-appropriate techniques to elicit knowledge during a knowledge acquisition session: brainstorming, consensus decision making, and the nominal group technique. Finally, we consider the importance and objectives of debriefing following knowledge acquisition from multiple experts
I n t r o d u c t i o n
The knowledge acquisition process is complex because it entails defining, acquiring, conceptually structuring and refining the content of a knowledge base. To accomplish these tasks, knowledge engineers must be proficient in the application of various knowledge acquisition skills (e.g. task analysis, interviewing, protocol analysis) (McGraw & Seale, 1987a). The already difficult knowledge acquisition process becomes even more complicated when the expert system being developed requires that knowledge engineers interact with multiple experts, either as individuals or in a group. In fact, if knowledge acquisition for an expert system with a single expert can be described as a bottleneck (HayesoRoth et al., 1983), acquisition from multiple experts, especially in a group setting, has the potential to become a 'log jam'.
The use of multiple experts for knowledge acquisition may often be unavoidable, however, necessitating that expert system developers confront the inherent difficulties of methodology and procedures, elicitation techniques, diverse opinions and informal knowledge verification. The purpose of this tutorial is to outline basic issues with which developers must contend when multiple domain experts may be * Dr K. McGraw currently heads Cognitive Technologies. a consulting firm in Alexandria, VA. She is on assignment with Falcon Microsystems at 1801 McCormick Dr., Landover, MD 20785.