Peter Norvig on Expert Systems - Dictionary of Arguments
Norvig I 633
Expert Systems/AI research/Norvig/Russell: Early expert system research concentrated on answering questions, rather than on making decisions. The emergence of >Bayesian networks in the late 1980s made it possible to build large-scale systems that generated sound probabilistic inferences from evidence. The addition of >decision networks means that expert systems can be developed that recommend optimal decisions, reflecting the preferences of the agent as well as the available evidence.
A system that incorporates utilities can avoid one of the most common pitfalls associated with the consultation process: confusing likelihood and importance. A common strategy in early medical expert systems, for example, was to rank possible diagnoses in order of likelihood and report the most likely. Unfortunately, this can be disastrous!
(…)a testing or treatment plan should depend both on probabilities and utilities. Current medical expert systems can take into account the value of information to recommend tests, and then describe a differential diagnosis.
Norvig I 634
Steps for an expert system for a medical treatment:
Create a causal model: Determine the possible symptoms, disorders, treatments, and outcomes. Then draw arcs between them, indicating what disorders cause what symptoms, and what treatments alleviate what disorders
Simplify to a qualitative decision model: we can often simplify by removing variables that are not involved in treatment decisions. Sometimes variables will have to be split or joined to match the expert’s intuitions.
Assign probabilities: Probabilities can come from patient databases, literature studies, or the expert’s subjective assessments.
Assign utilities: When there are a small number of possible outcomes, they can be enumerated and evaluated individually (…).
Verify and refine the model: To evaluate the system we need a set of correct (input, output) pairs; a so-called gold standard to compare against. For medical expert systems this usually means assembling the best available doctors, presenting them with a few cases (…).
Norvig I 635
and asking them for their diagnosis and recommended treatment plan.
Perform sensitivity analysis: (…) checks whether the best decision is sensitive to small changes in the assigned probabilities and utilities by systematically varying those parameters and running the evaluation again. If small changes lead to significantly different decisions, then it could be worthwhile to spend more resources to collect better data. Sensitivity analysis is particularly important, because one of the main
Norvig I 636
criticisms of probabilistic approaches to expert systems is that it is too difficult to assess the numerical probabilities required. Sensitivity analysis often reveals that many of the numbers need be specified only very approximately.
Norvig I 639
Surprisingly few early AI researchers adopted decision-theoretic tools after the early applications in medical decision (…). One of the few exceptions was Jerry Feldman, who applied decision theory to problems in vision (Feldman and Yakimovsky, 1974)(1) and planning (Feldman and Sproull, 1977)(2). After the resurgence of interest in probabilistic methods in AI in the 1980s, decision-theoretic expert systems gained widespread acceptance (Horvitz et al., 1988(3); Cowell et al., 2002)(4). >Decision theory/Norvig, >Decision networks/AI research.
1. Feldman, J. and Yakimovsky, Y. (1974). Decision theory and artificial intelligence I: Semantics-based region analyzer. AIJ, 5(4), 349–371.
2. Feldman, J. and Sproull, R. F. (1977). Decision theory and artificial intelligence II: The hungry monkey.
Technical report, Computer Science Department, University of Rochester.
3. Horvitz, E. J., Breese, J. S., and Henrion, M. (1988). Decision theory in expert systems and artificial intelligence. IJAR, 2, 247–302.
4. Cowell, R., Dawid, A. P., Lauritzen, S., and Spiegelhalter, D. J. (2002). Probabilistic Networks and Expert Systems. Springer._____________Explanation of symbols: Roman numerals indicate the source, arabic numerals indicate the page number. The corresponding books are indicated on the right hand side. ((s)…): Comment by the sender of the contribution. Translations: Dictionary of Arguments The note [Author1]Vs[Author2] or [Author]Vs[term] is an addition from the Dictionary of Arguments. If a German edition is specified, the page numbers refer to this edition.
Stuart J. Russell
Artificial Intelligence: A Modern Approach Upper Saddle River, NJ 2010