(Original Review, 1988-05-30)
Perhaps it is worthwhile clearing up a few fundamentals here. Specifically, the concept of something complicated being created as opposed to evolved. Of course, consciousness has evolved and is a characteristic of the complex arrangement of entities whose properties are understood by physics. But, going from the basic laws governing the building blocks to the complex is currently way beyond anything dreamt of in systems theory, where biological simulation is hovering around the simple swim patterns of single celled flagellum bacteria. Before attempting to build something we try and understand it. That is, what aspects of the thing to focus on. What are the essentials? For example, the planetary orbits are described by their mass relative orientations and velocities. We don’t bother about their colour, smell or aesthetic properties. This is a canonical example of the great success of predictive science. When we come to other things, broadly termed complexes then we have a completely different situation. This is because of the problem of defining what we want to predict. To progress we have to get at something simple. Like a phase transition, a gross change in behaviour. But, what are we looking for in the human mind? What do we hope to find? This is a perfectly legitimate area of investigation, but we should be cognisant of the relative fruitfulness of different approaches. I’m reminded of Nietzsche’s metaphor where a huntsman sets of in pursuit of this elusive quarry. His dogs get separated each pursuing a different scent and the hunt has to be abandoned, as each dog’s lead is equally valid. We end up knowing more and more about less and less.
I am a little suspicious on the strong statement about the relation of computation to Physics. I am not aware that computation is fundamental to physics itself. The statement that given enough time and space any physical situation can be computed seems overstated. Even at the mundane levels of chaotic systems Nature 'solves' the physics uniquely and in real-time - the problems we have arise from the fact that we cannot in real-time acquire infinitely precise and infinitely covering representations of the physics and we cannot do the fully parallel calculations required (not even with more processors - we do not have analytic methods which solve for all degrees of freedom truly simultaneously). Consider n-body attraction - we have no analytic solution which emulates Nature - we do have approximations of course, which are good enough for everyday use.
Of course Penrose has also looked at this from another angle - specifically whether computation as a formalism can deliver the measurable/observable behaviour of the human mind. This is still controversial but for the moment is quite a strong position. More heuristically, Searle's chinese room poses multiple unresolved issues for the 'mind as computation' camp - precisely because it is a very good analogue to computation and is self-evidently a nonsense.
There are plenty of reasons to doubt the universality of computation and thus our ability to simulate organic and mental processes beyond the little toy models we have currently.
No objection to the possibility of AGI, nor to the author's positing of philosophical questions as central, nor to the history given of AGI attempts so far BUT, Professor Deutsch goes wrong, wrong, wrong with his very first sentence. The human brain has no capabilities at all. The human organism however has all the capabilities of thought ascribed to its brain AND capabilities of movement, action, speech which are in no way optional accessories. The only intelligence we know of has arisen exclusively in the context of beings which act and react in and against the world. You can emulate that in Turing machines all you like; and end up with an simulation.
The computer analogy for the organism, with its implicit partner of the disembodied brain, has been and remains profoundly misleading. Following the evidence we have - ourselves - the best prospect for AGI may be in robotics. There is no reason - other than carbon chauvinism - to suppose that a machine, with the ability to act and react in the world, cannot be conscious and intelligent. There is every reason to suppose that a computer programme cannot.
[2018 EDIT: I suggest looking in to the work of Robert Rosen rather than Deutsch's; Rosen "revealed" the underlying tacit assumptions of a state/phase-based physics and showed that biology requires entailment structures that are not present in this physics. Thus it is that biological organisms are not a mere subset of current physics, but are representative of complexities that require physics to be enlarged.]