Here is an argument that I have been noodling on lately. I am not sure I agree entirely with it, but I think it is worth considering.
It is easy to believe that you are a machine, and it gets easier the more complex the machine is. If the machine can also mimic some part of our behaviour it becomes almost impossible not to fall into the metaphorical trap and assume that mind being as machine means that mind is a machine.
The shift from as to is is a small slip of the mind, but tempting for us as we are always trying to understand ourselves.
Yet, we are not machines, not even very complex ones. We are human beings.
This is not a grand statement of human exceptionalism (we should be wary of those as well), but rather a simple scientific fact. We are a part of in incompressible algorithm, evolution, that unfolds in a network of ecosystems. The machines we compare with are not.
Now, it is easy to lose sight of this when we speak of things like “artificial intelligence”, since this, and many other concepts like it, come with a silent third term – “human”. When we say “artificial intelligence”, well, we hear “artificial human intelligence” and so we get stuck in the metaphorical trap again.
What has emerged in AI today are extra-biological intelligences – but this does not mean that they are less interesting. On the contrary, they are stranger and more wondrous than we think. How come something that is so unlike us can behave so much like us? That is the question – not if we are mere machines.
And this matters for everything – philosophy, ethics, policy, economics and our overall predictions for this technology.