Rule-based expert systems attach a weight to each rule in order to represent uncertainty or strength of association. There are a number of schemes that are used to represent uncertainty in expert systems. Some of these methods allow the system designer to solicit either the probabilities, used to compute the weights, or to solicit the weights directly, or both .
This paper presents results that indicate that if the weights are gathered directly, rather than using probabilities, then the weights may not meet the underlying conditions of the mathematical model of uncertainty on which the weights are based or the weights may imply highly unusual behavior for the underlying probabilities and implicit utility function.
In one system it is found that there were violations of the mathematical properties of the model in over forty percent of the weights on the rules of the system. If the weights do not meet the constraints of the underlying mathematical models then such violations may yield inappropriate parameterization of other weights in order to make the model work . Further, such violations can lead to an inappropriate estimation of the probabilities of events by the system and yield inappropriate inferred weights.
In another case it was found that a system was dominated by weights that suggest highly unusual behavior for the underlying probabilities.
From an operational per spective these inconsistencies indicate the importance of the method used in gathering the weights , e.g. indirectly through the probabilities or directly through the weights. It also indicates the importance of validating and verifying the weights to ensure that the weights meet the needs of the underlying theory and do not force unusual relationships onto the underlying probabilities.