Einstein on operationally defining and measuring simplicity

http://plato.stanford.edu/entries/einstein-philscience/

But for all that Einstein's faith in simplicity was strong, he despaired of giving a precise, formal

characterization of how we assess the simplicity of a theory. In 1946 he wrote about the perspective

of simplicity (here termed the “inner perfection” of a theory):

This point of view, whose exact formulation meets with great difficulties, has played an

important role in the selection and evaluation of theories from time immemorial. The

problem here is not simply one of a kind of enumeration of the logically independent

premises (if anything like this were at all possible without ambiguity), but one of a kind

of reciprocal weighing of incommensurable qualities.… I shall not attempt to excuse the

lack of precision of [these] assertions … on the grounds of insufficient space at my

disposal; I must confess herewith that I cannot at this point, and perhaps not at all,

replace these hints by more precise definitions. I believe, however, that a sharper

formulation would be possible. In any case it turns out that among the “oracles” there

usually is agreement in judging the “inner perfection” of the theories and even more so

concerning the degree of “external confirmation.” (Einstein 1946, pp. 21, 23).

As in 1918, so in 1946 and beyond, Einstein continues to be impressed that the “oracles,”

presumably the leaders of the relevant scientific community, tend to agree in their judgments of

simplicity. That is why, in practice, simplicity seems to determine theory choice univocally.

  I wonder if any proper empirical test has been done on judgements of simplicity in theories. I can think of a simple case. Give people a data set and various functions that is supposed to result in that data set. Let them judge the simplicity of the various functions. Repeat this exercise in different age groups, different universities, different countries, different parts of the world. If there is near-universal agreement, then there mostly likely is some human universal present.   That is, unless it has something to do with using the mathematical system that is currently popular, e.g. the decimal system. We can repeat the exericse with other systems as well, even fictional systems.   Experimental/empirical math? Who wud have known? :P   After the above has been done, and it can be done quickly, that does not settle the matter that easily. Since theories are much more complex than functions alone. But we can still expand the data set and ask people to judge various multi-variable functions. This comes closer to normal theories.   The best thing about all this, is that it can be done via the internet, thus making it a very cheap experiment to run.   I wud be surprised if we did not find very high correlations between judges' ratings of simplicity.