Rationality of IBE
(A philosopher’s perspective)
Introduction
Consider that someday one wake’s up and through windows sees that the grass in the lawn is wet. Assuming one could only see the grass and nothing else, the best explanation for the grass being wet that comes in our mind is that it might have rained the previous night. Since it had been raining frequently for a week, this assumption is quite intuitive which almost all of us would have made. Now consider another explanation that the gardener might have come early in the morning and watered the plants which has caused the wet grass. Though we have two equally good theories to explain the evidence of wet grass, one choses first one over the other as one knows that the gardener doesn’t usually come that early. This reasoning by which we chose the first hypothesis over the second one is known as Inference to Best Explanation or IBE. IBE is the process of choosing a hypothesis that best explains the given evidence. At a time, we might have several competing hypotheses to choose from so one must be able to reject all others before making such inference. Since we already have the evidence in the first place, this is essentially reasoning backwards.
Examples Of IBE
We are always using IBE in our daily life so it often gets unnoticed. We often shape our beliefs based on a set of hypotheses and choose one which best explains the phenomena. The idea being that from a set of competing theories, one which best explains, is more likely to be true.
Scientific Theories
Apart from daily life decisions, it is extensively used by scientists too. Major science postulates are laid based on the fact that they provide the best explanation to the observed things. Many space theories (one being deformation of Uranus) are essentially inference to the best explanation where one theory best explains the abnormalities observed. Consider Newton’s theory of gravitation. This could explain all of his observations with different objects in different environments. So he chose this as it was best explaining the motion of various objects and now we know how important it is and how extensively it is used to prove other science theories. Another is the discovery of electrons where abductive reasoning helped to arrive at the desired conclusion. Even Halley’s comet discovery is linked to IBE. When scientists were puzzled with the observation of a comet in the night sky at the gap of about 75 years, they thought it is best to assume that it is a single comet which completes its path in 75 years and thus Halley’s comet was discovered.
Philosophical Arguments
IBE is also extensively used in the field of philosophy. Existence of God is inferred to as the best explanation to explain the things in the universe. Consider the design argument used to prove the existence of God. This states that every living organism has a superb design which allows them to carry several functions. If we think about how we got this excellent design, the best explanation that arises is that there is an intelligent designer which has a mind superior to all (which allows him to design these extraordinary designs) and is hence none other than God.
Another such case from where IBE is used is the argument against cartesian scepticism. Philosophers such as Vogel argue that the RWH (Real World Hypothesis) is better than CSH (Computer Simulation Hypothesis) by means of IBE. Consider the fact that in real world no two objects can occupy the same place at same time and this is a conceptual truth. But in the case of CSH it is not conceptual truth as computers can readily place two objects at the same location, and thus it needs to be explained. Since RWH and CSH are similar in all other aspects, RWH is better as it does not need to explain this simple fact which makes it short and simple and thus better. Similar argument could be used to neglect the hypothesis which claims that we are controlled by a demon who in turn is responsible for all our experiences.
Is IBE Rational?
As seen above, we unknowingly tend to use abductive reasoning a lot in making our day to day decisions. This certainly doesn’t mean that this thought process is rational and its correctness can’t be questioned. In fact the extent to which we should rely on inference to best explanation is a bit controversial. Also there have been theories which say that people tend to overestimate the probability that the event would actually occur which is even more for simpler events as compared to the complex events. Thus a hypotheses seeming to be best actually may not even be a good one to support the evidence. Then the fact we consider IBE as rational is highly debatable.
Success
From its definition it may seem that IBE is nowhere rational and using it to prove something would always result in failures as it appears to be a somewhat intuitive idea rather than based on proper reasoning. But most of our scientific theories such as discovery of electrons, gravitational laws etc. are based on IBE itself. These theories are later used to prove more advanced results and thus they also depend on IBE indirectly. Since we have not found flaws in any major scientific theory so far, it seems that abductive reasoning can be rational in several cases and can be employed as and when required. Not only this, some philosophers even suggest that IBE is the ‘quintessential’ way of explaining scientific theories.
Failure
But we have known instances where IBE fails such as the design argument for existence of God. It was advocated that God is responsible for the exceptional design of living things which was arrived at using IBE. Later on Darwin gave a more firm theory based on natural selection which disproved the previous theory. Now consider another scenario where one has only seen gray coloured elephants and no non-gray ones. Through this observation one might infer that all elephants are gray as this seems to be the best explanation for our observations. Clearly this example shows that IBE is not always correct and is associated with some flaws.
Fundamental Flaws in IBE
Bad lot argument
The problem comes with the definition that we use for IBE which is:
“Given evidence E, and hypothesis H1, H2 .. Hn, infer the truth of that hypothesis which best explains E.”
A natural flaw in this is that there can be a case when all the hypotheses are not good and from a set of bad hypotheses, we chose one which is relatively better but actually a bad one. IBE in no way guarantees that we start with a lot containing all good hypotheses. Another way to think of this is that IBE doesn’t guarantee that the set of hypotheses we begin with is exhaustive, i.e. there can be a case that the best hypotheses is missing from our set. This implicit assumption is too strong which brings out this flaw in our reasoning.
For this definition to hold firm, the best explanation must always be included as one of the hypotheses. But we can’t always be sure that the best explanation would be there in our set of hypotheses. If this would be the case, we must consider ourselves to have some super powers to always think the best explanation as one of the candidates in our hypothesis set. But it seems that this task doesn’t require much of super powers and can easily be done as for a set of hypotheses {H1,H2..Hn} we can always add another hypothesis called the catch-all hypothesis such that all hypotheses can never be false. This hypothesis is given by:
HC=￢H1 ⋂ ￢H2 .. ⋂ ￢Hn
It seems that the flaw is removed but this method doesn’t provide us with anything better in general as the constructed hypothesis turns out to be less informative. Consider the example of wet grass where we have two hypotheses: the gardener came early and it had rained. Now our constructed hypotheses that neither gardener came early nor it rained doesn’t provide us much information on the cause of wet grass. Hence we need to modify the definition of IBE with something more affirmative. To overcome the flaw we consider the modified definition as:
“Given evidence E, and hypothesis H1, H2 .. Hn, infer the truth of that hypothesis which best explains E where the hypothesis is satisfactorily good to be used for explaining the evidence.”
But this brings in another issue where we need to decide if some hypothesis is good or not which in itself is a difficult task. Thus it appears that this definition is also not able to meet up with our requirements so ultimately we decide to use IBE to narrow down to a hypotheses which is more close to the actual hypotheses than any other hypotheses in the set rather than using it to select the best hypotheses in general. This shift in how to define IBE can be understood with the example of Helley’s comet. Scientists would observe comets after gaps of 75–80 years so they put forward a theory that these are due to a single comet only. Now this theory is based on our final definition of IBE that the hypotheses is more closer to the actual hypotheses than any other and when assumed true, this increases the probability that our evidence would occur.
Circularity of IBE
With our current definition of IBE, the hypothesis is decided based on which explains the evidence best. However the hypothesis is validated by examining it with a new set of evidence. For example, consider a hypothesis predicting coin toss based on certain occurrences of heads and tails. It is only later after new evidence is obtained we validate the hypothesis. In other words we get a certain degree of confidence in a hypothesis only when it explains new evidence well. This causes it to be circular in nature as we decide on hypotheses based on existing evidence and later use that evidence to explain it. This makes it difficult to use for reasoning.
Interpretation of “best”
Another major problem is how the term “best” is interpreted. It could mean likeliest or most plausible or loveliest, each of which will change how IBE is understood. There can be competition on which basis to choose the best hypotheses. Does it have to be shorter, or more explanatory or should it have more support from the evidence etc. This creates a whole lot of confusion in how IBE is interpreted by the people. General method is to keep a theory simple and avoid unnecessary and complicated explanations. These are used as tie breakers i.e when everything else is the same, the shorter theory is better. If that is also the same, then one with more explanatory power is better.
Conflicts with Bayesian Theory
Abductive argument in some sense violates the main principles of bayesianism and thus is criticised by several people for its rationality as bayesian theory is a well formed theory to explain probabilistic events which can be derived using axioms of probability.
IBE and Bayesian Theory
Probabilistic Approach of IBE
Though Bayesian theory and IBE appear very different from each other some philosophers such as Lipton try to combine them both. Consider IBE as follows:
P(H | E,K) = P(E | H, K) * P(H | K) / P(E | K)
where E is evidence, H is a hypothesis and K is our background knowledge. This relation shows how probability of our hypothesis given evidence and our prior knowledge depend on
- probability of evidence given knowledge and hypotheses
- Probability of hypothesis given our prior knowledge
- Probability of evidence considering our prior knowledge
This is the general basis on which we decide if some hypothesis is better than the other by simply comparing P(H1 | E,K) with P(H2 | E,K) when we experience certain evidence E with prior knowledge K.
Use of IBE in Bayesian Theory
Problem here is that to use the above formula one requires probabilities of events which are not very specific and are rather based on our choice or intuition. One such choice is to assign priors with equal probabilities. Many times we have no idea about the priors so sometimes we also assign random probabilities to the priors. IBE can be used here to assign probabilities by considering it as an approximate heuristic. Based on certain past experiences we use IBE and combine them to form a heuristic which is further used in bayesian theory.
Another way of combining them both is that we could use IBE to rule out certain hypotheses before applying bayesian theory on the remaining set of hypotheses. As abductive reasoning requires less rigour, we can easily apply it to a large set of theories and reduce the size of the set by eliminating those which appear to be least possible.
Criticism
However we can’t always do that as bayesian theory is to be applied only when certain conditions are met. One of these is that we should be able to decide the prior probabilities with a certain precision. When this is not followed, bayesian theory is generally not applied as it may not give true results. Thus there is no point in using IBE to guess prior probabilities. But according to Bayesian theory the effect of priors in deciding the outcome reduces to a great extent when new evidence is discovered. Thus in some sense if we expect new evidence, we may readily use IBE to begin with an initial set of priors as later on their effect would be mitigated. But then it seems that IBE is just an auxiliary device and cannot be used as a standalone method. Apart from this the way we use IBE to formulate prior probabilities is also not well described and is too vague to be considered as a rational method.
Conclusion
Throughout our history we have seen how IBE is used to prove various scientific and philosophical theories. We have also seen some cases where it fails and even discussed the fundamental flaws associated with it. Even with these flaws this method of reasoning is fairly popular as it is backed by the cases where it has provided success. Overall, the situation on IBE is quite unsatisfactory. On one hand it has proved some great theories and on the other it fails at simple cases. Generally it gives us the correct answer but in many cases it also results in accepting false hypotheses and in the end it completely depends on us on where to draw the line where we begin to distrust this type of reasoning.
