A quick argument for the principle of simplicity
Suppose there is a world where there are facts F1, F2, F3, ... Fn that need to be explained. Suppose further that someone advances an infinite amount of theories that aims to explain the facts. Suppose even further that all the theories presented happen to explain the facts equally well.
The first theory implies the existence of one entity, E1. The second implies the existence of E1 and E2. The third implies the existence of E1, E2 and E3. ... The N'th theory implies the existence of E1, E2, E3, ... En entities. How should one choose which theory is more likely to be correct? The intuition I have is that the first is the most plausible and the one with the most implied entities (there is no one with the most though) is the most implausible. In other words: the more entities implied, the less probably the theory is. So, if we are to formulate this as a general reasoning principle we could do it like this:
Of two equivalent theories or explanations, all other things being equal, the simplest one is to be preferred.1
More reading:
http://en.wikipedia.org/wiki/Occam%27s_Razor
http://plato.stanford.edu/entries/simplicity/
http://www.iep.utm.edu/o/ockham.htm#H2
1Common phrasing of the principle. http://en.wikipedia.org/wiki/Occam%27s_Razor#Variations