By Paul Almond, 9 October 2005

Previous Articles in this Series

Reading of the following articles is suggested before reading this article:

Occam's Razor Part 1: What Is Occam's Razor? http://www.paul-almond.com/OccamsRazorPart01.htm.

The following article, although not part of this series, is closely related to it and is a prequel to this series:

What is a Low Level Language? http://www.paul-almond.com/WhatIsALowLevelLanguage.htm.

Introduction

This is the second article in the series about Occam's razor. The previous article Occam's Razor Part 1: What is Occam's Razor? [1] introduced the concept of Occam's razor and discussed it in general terms.

Amongst other things, this series will attempt to provide a justification for Occam's razor. This means that in this series of articles we cannot actually assume Occam's razor to be correct: we have to show how it emerges from more basic principles. What should these more basic principles be? This will be the subject of this article and the one following it. This article will deal with principles about how theories should be expressed. The next article will deal with assumptions about reality, and how theories relate to it, that are necessary to allow Occam's razor to be derived.

Some discussion of the ideas in this article has already occurred in the previous article. Any repetition is due to the need to set the main principles out in a structured way.

Principles

1. The Principle of Formal Description

A model is only valid if stated in a formal way; that is to say there must be no doubt about the meaning of a model and no subjectivity possible in its interpretation. There must be no vagueness in its predictions. Models should be expressed in formal languages.

2. The Principle of Algorithmic Description

This really follows directly from the last principle. The language used to describe a model should only allow formal description. There must be no vagueness in the predictions made by a model or in interpreting the model to determine what the predictions are. The best way of avoiding vagueness in how predictions are to be extracted from a model is not for the model merely to be a static entity, admitting to such interpretation as may seem desirable, but for it to be a dynamic thing that actually makes it predictions. This demands a great deal of flexibility from the language.

All of this demands that the languages used to describe models should be algorithmic ones. Models are algorithms that accept input data about the current state of reality and make predictions; that is to say models are like computer programs.

3. The Principle of Turing Equivalency

Turing equivalency is a property possessed by a computing language when its capabilities, regarding what can be expressed in it, are not limited. If a language is Turing equivalent then, in principle, it can be used to express any algorithm that can be expressed in any other language. Turing equivalency, as a requirement of the language used to express models, follows quite naturally from the previous principle; otherwise there would be the risk that reality may contain features that cannot be expressed as algorithms in whatever modelling language is used.

4. The Principle of Low Level Language

As I discussed in a previous article What is a Low Level Language? [2] the choice of language used to express models can introduce prejudice. One obvious cause of such prejudice is in the lengths of descriptions of models: a model that has a short description in one language may have a long description in another language. This is important because description length is likely to be relevant in any workable, formally stated version of Occam's razor.

The solution to this is to use a formal language which is as low level as possible. An objection to this would be that the idea of low level language is subjective and that we cannot really say what a low level language is. The article which I just mentioned refutes that idea and shows how the concept of low level language can be absolute: that is why it was written.

Should these requirements be enforced rigorously?

A formalized version of Occam's razor, the goal of this series of articles, will only work with formally described theories. I am not, however, arguing that these requirements should be enforced rigorously whenever we deal with theories. If we were to do this then it would be impractical for humans to do science. I am not suggesting that descriptions of scientific theories produced by scientists in human languages such as English are somehow invalid. If this is the case, why try to formalize things at all?

This issue of language is relevant when we are considering the philosophy behind Occam's razor and an understanding of it should influence the way we view theories - even when they are stated in more casual ways that do not meet these requirements, as most theories will be when being considered by humans. Having a formal Occam's razor that works on formal theories can still give us the understanding needed to assess more casually described theories and an understanding of what makes a theory desirable.

As an example even if a theory is stated in English we may intuitively see that such a theory would have certain properties, if it is stated formally and in a low-level algorithmic language, that preclude it as a reasonable choice. All of this formalization is necessary for us to get a grasp on the issue of Occam's razor.

Objections

Here are some of the more obvious objections to these principles, and my replies.

Objection 1: What if formal language is insufficient? Things could exist that are outside the scope of formal language to describe them.

Answer

Could reality have any characteristics that are outside the scope of formal language? Many people who argue for all kinds of supernatural things would say, 'Yes.' They would argue that my insistence on formal description is closed minded and that reality has characteristics that cannot be formally described. I would disagree with this. Using formal language is merely being precise about what we mean and if someone claims that something exists that needs to be described vaguely, instead of precisely, we should be suspicious.

It is rare for people making claims like this to put them beyond the scope of language in general, something which, although I would disagree with it, I would at least find more consistent. A claim which has been put beyond the scope of language in general cannot, of course, be made using language, making it difficult to tell anyone what the claim is - something which proponents of theories that allegedly fall outside the scope of formalization generally have little difficulty doing. Rather, people making claims like this often insist that they are within the scope of language when it comes to stating them while being outside the scope of precise language when it comes to actually working out what the claims mean with a view to refuting them, which is all very convenient.

The idea that formal language is not enough is not really tenable.

Objection 2: What if reality is non-computable? Algorithmic descriptions would not be sufficient then.

Answer

If formal description is enough, what about algorithmic description? I have not merely said that descriptions need to be formal. I have assumed that an algorithmic formal system which is Turing equivalent, that is to say it allows expression of any algorithm, is adequate for the expression of any model that may be required. There is an assumption here that reality can be modelled by an algorithm and does not exhibit any behaviour outside the scope of algorithms.

Is this assumption reasonable? We could say that algorithmic descriptions of models and formal descriptions of models are basically the same thing and that there is no formal model which could describe reality which could not be expressed algorithmically. We need to be careful here: the existence of formally describable but non-algorithmic characteristics of reality is exactly what has been proposed by Roger Penrose.

Penrose argues that non-computable laws of nature exist, laws which can be described formally, but cannot be simulated by an algorithm. Penrose's claim is that non-computability is necessary to account for the strange results of quantum mechanics experiments. He also states that it is necessary to account for human mental abilities. Penrose thinks that human cognition cannot be a result purely of computational processes [3]. He attempts to offer a proof to support this [4], a proof which has been contested in numerous papers, including one by me [5].

Even though I am sceptical about Penrose's claim to have proved that human cognition is non-computable, this does not resolve the general issue of whether or not non-computable physics exists. If non-computable physics can exist then the language requirements stated in this article would have to be extended to allow formal non-computable descriptions as well as algorithmic ones. This would not present too much of a problem.

I think that at the moment of this would be a distraction from the main purpose of dealing with Occam's razor. I will however admit here that the issue of possible non-computability in reality may need to be considered at some point.

Objection 3: You are trying to show why Occam's razor applies, but you are having to making so many assumptions to do it that we may as well just assume Occam's razor instead! Why not just assume Occam's razor instead of all your assumptions.

Answer

Some assumptions are clearly implied by the principles for model description stated in this article. It is assumed that any characteristic of reality can be described formally and that algorithms will suffice as descriptions (though I have left this issue open to later consideration). These assumptions, however, are not being made instead of Occam's razor. They are being made to allow Occam's razor to be formalized, simply so that we can be sure what we are talking about - a necessary step in actually showing Occam's razor to be correct. Even if we just assumed Occam's razor to be true, if we wanted to use a formal version of it we would still need to have principles like this so that models are described without ambiguity. Nothing is being assumed that does not need to be assumed anyway.

Objection 4: Why should we assume the principle of low level language? Aren't you just introducing prejudice whatever language is used?

Answer

This is a bit like the old argument, usually in relation to history that 'there are no objective points of view'. The objection is applying a similar idea to language and saying that there is no language, low level or otherwise, which provides a neutral point of view.

The principle of low level language was assumed to prevent the apparent plausibility of models being affected by features of the languages in which they are expressed. I could be asked why I think that insisting on a low level language will solve this problem. Even if the concept of low level language is meaningful, is that in itself not just another form of prejudice? Why swap the prejudices of high level languages for the prejudice of low level language?

This concern is actually quite deep, and not at all obvious: I think it would be lost on some people who think there that does not need to be any reason for favouring low level languages. When we make the decision to use low level language we are making an assumption about reality. We are assuming, in a way, that reality 'thinks' (I use the word in a non-literal sense) - whatever that might mean - that low level languages are as sensible as we do.

Someone could choose some arbitrary, high level language in which some really absurd concept can be represented by a small amount of information and then declare that language to be the 'true' language for describing reality while all other languages, including even the most low level language that can be conceived, are the 'prejudices ones'. This could be met by the obvious retort, from people like me, of 'Your language is not a low level language!' and, rather than any attempt to argue that it is, the response could simply be, 'So what?'

When we are facing a matter like this we are into quite deep ontological issues. The main defence here, and it is hardly desirable, is that this is all we can do. We have to choose some type of language to represent models, otherwise there would be nothing to distinguish models from each other: any model could be made to appear plausible by finding an appropriate language in which to represent it. Intuitively, we can see that high level languages are likely to lead to absurd results: we would hardly want to choose a language in which a single symbol in a model represented 'blue pixies from the planet Zanussi in another dimension', for example, which would make it appear plausible. Preferring low level languages is the only way out of this.

The answer that I gave to the last objection also applies here: even if we just assumed Occam's razor to be true, if we wanted to formalize it, we would still have these same language issues. Anyone who wants to assess the plausibility of things, if he/she thinks about it deeply enough is really stuck with this issue of language anyway. It is philosophically relevant even to our everyday formation of opinions about the world and it may just seem to be more of an issue here because we are explicitly discussing it.

While I can accept the need for low level language, I would prefer to deal with it in a better way. I may return to this matter later, when other, more important issues have been resolved.

Conclusion

This article has stated some requirements for languages used to represent models when applying Occam's razor. These requirements are so that Occam's razor can be employed in a way that returns useful results when comparing different models.

The requirements are:

• A language used to express models should be formal
• The language should allow algorithmic description; that is to say theories should be algorithms.
• The language should be Turing equivalent; that is to say there are no restrictions on the kinds of algorithms that can be represented in the language.
• The language should be low level so that no prejudice is introduced by its features.

This article has dealt with requirements for language when dealing with Occam's razor. The next article will deal with assumptions about reality that are necessary to allow us to obtain Occam's razor and some of the methodology that will be needed for Occam's razor to work.

References

[1] Almond, P. (2005). Occam's Razor Part 1: What Is Occam's Razor? Retrieved 22 August 2005 from http://www.paul-almond.com/OccamsRazorPart01.htm.

[2] Almond, P. (2005). What is a Low Level Language? Retrieved 17 July 2005 from http://www.paul-almond.com/WhatIsALowLevelLanguage.htm.

[3] Penrose, R. (1989). The Emperor's New Mind: Concerning Computers, Minds and the Laws of Physics. Oxford: Oxford University Press.

[4] Penrose, R. (New Edition, 1995). Shadows of the Mind: A Search for the Missing Science of Consciousness. London: Vintage. Chapter 2, pp64-76.
(Originally published:1994. Oxford: Oxford University Press)

[5] Almond, P. (2004). A Refutation of Penrose's Godel-Turing Proof that Computational Artificial Intelligence is Impossible. Retrieved 24 October 2004 from http://www.paul-almond.com/RefutationofPenroseGodelTuring.htm.