Tag: Artificial Intelligence

Studying the mereologies of AI

Nicklas LundbladLeave a comment

In this article Robin Hill suggests something that may seem both obvious and strange at the same time: that our artificial intelligence systems might not cut the world up in the same way we do, or that they may not use the same features to cluster concepts as we do. The example she gives is a dog – we recognize it by looking at the fur, ears, wet nose etc – those are the features we focus on – but why should the machine focus on them? Why should we assume that it divides the world into wholes and parts the way we do?

This is a deep question, one associated with the often neglected subject of mereology1 The sciences of wholes and parts, see for example https://plato.stanford.edu/entries/mereology/ – a very useful way of thinking about concepts. In mereology we recognise that wholeness of the “dog” and the parts that go into constructing that whole, but we also realise quickly that there are many, many different ways in which we could construct that particular whole. This includes lumping together or slicing things differently across a number of different dimensions, including a temporal dimension. The dog may be made of moments in space.

We could hypothesize that an all-knowing super-intelligence might actually converge to finding patterns in the particle paths through space-time, and so would recognize entirely different concepts than we do. A clustering of such paths may conjure the concept of “Wegobans” who represent certain particle paths through space time with strong commonalities that we cannot even begin to guess.

Now, the way we slice and lump (to use the evocative language that Lee Anne Fennell uses in her excellent book Slices & Lumps: Divisions and Aggregation in Law and in Life (2019)) the world is not arbitrary, it is rooted in evolution. Our concepts have evolved for a function – as Ruth Millikan has shown – and so we should expect human mereology to follow from that evolutionary path. But what if that particular mereology is not preserved in the training of large language models? What if that is a feature that is lost in the methods we currently use?

What would that mean?

We get into deep philosophy of language territory here – there seems that there is the chance that we end up in false communication patterns, where the symmetry of the way we use the signs convince us that we mean the same, but the structural composition of those signs into wholes and parts are radically different.

This is something like what Nelson Goodman suggested with his grue/bleen-experiment2 See the “new riddle of induction” here https://en.wikipedia.org/wiki/New_riddle_of_induction , where a concept can be composed in arbitrarily many different ways – or at least along the temporal dimension of the concept3 One interesting question is of course if there are general dimensions along which a mereology is constructed, and if that can be used to explore alternative mereologies?. That is in turn interesting because there will be, in any such faux communication, points where the difference in the composition of the signs lead to drastic break-downs in the ability to convey meaning to each-other. 4 A simple example may be one in which I mean “allergy” to be a medical condition that only applies before the 31st of January 2024 – a far-fetched example, but still.

And again, it is almost obvious to point out – but the study of the mereologies of artificial intelligences will be an absolutely essential piece of getting both security and safety right. This is slightly different from the study of explainability, where we look just for a mapping of systems to our own mereology so we can translate – and more fundamental: it is the view that we should look for general principles of mereological composition in large language models.

As we do so there are a number of interesting questions to consider, such as:

  • Do we believe that AI mereologies become more like human mereologies as the size of the training data set grows? Or could the relationship between human / AI mereologies be, say, U-shaped? Why?
  • Are there fundamentals in mereology that have to do with perception, and if so – does this mean that when we add sensors to AI that represent no human sensing capabilities we will end up with vastly different mereologies (cf what it may be to be a bat, as Nagel asks — what is the mereology of a bat – are the commonalities rooted in evolution in some way?)
  • What do mereological safety risks look like and how can they be addressed in the best way?
  • Is shared mereological composition a pre-condition for alignment?

And so on.

Notes

  • 1
    The sciences of wholes and parts, see for example https://plato.stanford.edu/entries/mereology/
  • 2
    See the “new riddle of induction” here https://en.wikipedia.org/wiki/New_riddle_of_induction
  • 3
    One interesting question is of course if there are general dimensions along which a mereology is constructed, and if that can be used to explore alternative mereologies?
  • 4
    A simple example may be one in which I mean “allergy” to be a medical condition that only applies before the 31st of January 2024 – a far-fetched example, but still.

Some policy problems in artificial agency (Agency and policy I)

Nicklas Lundblad1 Comment

A further step in understanding the challenges artificial intelligence will pose for public policy and regulation is understanding the role, construction and problems of artificial agency of different kinds. The models we are discussing today are mostly prompted in different ways, and so are in a way easy to regulate, since they only do what we tell them to. The step from one-prompt models to mission prompted or even unprompted or role-prompted agents is one that could introduce an interesting and qualitatively different set of questions for us.

But before we go there, we should admit that we understand only very little of what agency is overall, and how it relates to intelligence. 1 See eg Tomasello, Michael. The evolution of agency: behavioral organization from lizards to humans. MIT Press, 2022. We should probably also allow for there being many different kinds of agency and understand how they differ from each other.

What is agency, then? There is a simple answer here, and that is to essentially say that agency is intelligence in action – and if you build an intelligent system it will automatically exhibit agency as it tries to solve problems of different kinds. The system will acquire goals, and those goals will aggregate into some kind of agency. In this model of the world agency is a quality in an intelligent system, and related to specific goals – and so intelligence comes first and agency second.

But is this an accurate representation of agency? We could imagine a very different answer here – one that highlights how agency is primary and even precedes intelligence. Now, this would wreak havoc on some of the more instrumental understandings of the concept of intelligence, but it is worth exploring more in detail.

In this model, agency is something that is inherent in living systems and scales with their complexity. All living things have a will of some kind – and that will organises life in different ways. Intelligence is consequence of the organisation of life into ever more intricate patterns through agency, and not the other way around.

Here, agency is a kind of orientation to the world, a foundational relationship between life and the world, where all life seeks to situate itself in such a way as to adapt effectively. Agency, then, is rooted in adaptation and intelligence a tool for that agency. This is a not goal-seeking behaviour. It is seeking behaviour, and the notion of goals first emerges as intelligence has emerged as a response to the overall search that life engages in. 2 There is something here about a mechanistic and a biological view of the world. The idea that intelligence and agency can be broken down into separate algorithms and sliced into smaller pieces is challenged by a world in which life and intelligence may be algorithmic, but not compressible. This is a line of thinking that is found in, for example, some of Stuart Kauffman’s work. Where Penrose et al seemed to focus on whether or not there was something non-algorithmic that we can do that cannot be replicated, we could instead say that the algorithms that make us up are so complex, entangled and shifting that they are hard to replicate in any meaningful way — this would provide another kind of boundary condition for artificial intelligence.

Let’s, for the sake of argument, say that this model is right – then the order of things look like this: life – agency – intelligence – goals. We have, with our project, started to build far down in this chain and we are trying to replicating intelligence, and we confuse goals and agency. 3 Johannes Jaeger seems to be building out a criticism along these lines here: https://arxiv.org/abs/2307.07515 – his argument seems to confuse the algorithmic nature of intelligence and the complexity and primacy of agency, though – but this deserves a more careful read. The idea of algorithmic mimicry is an interesting one. Agency may be richer, less directed and more raw than goal-seeking behaviour. This model is Nietzschean in its focus on will as the primary quality, and intelligence as a secondary effect, perhaps even a regulator of will rather than anything else.4 Nietzsche’s view of the will to power, of will as a foundational life force is of course both problematic and protean, but there is a likeness here that needs to be recognised. The key to this perspective may be to focus on what the default state of living beings is. We could argue that there is a huge difference between inanimate matter and teleonomic matter5 The first person I noted using this term was David Krakauer, and he used it to provide a demarcation criterion for complexity science – the idea that complexity science deals with agency, with teleonomic matter, is fascinating. The challenge with the idea of a telos is of course that it is specific, and the telos we are exploring here may start out as much more of a tone or field. here, and that this difference is that inanimate matter is, if no forces affect it, at rest. Teleonomic matter is not, it is always moving, always seeking or directed in some way – even at “rest”. It is innately intentional. 6 This seems to bear more than a passing resemblance to the Husserlian way of thinking about the life world.

If this is the case the implications can be quite interesting. The whole project of building artificial intelligence, then, is focused on reconstruct what is really an evolutionary response devised out of agency. The real challenge, if we want to build an intelligence that can match ours, would be to construct artificial agency. But how do we even begin there? Can agency be reconstructed without also reconstructing an evolutionary setting and a selection pressure? Is agency even individual, or is it a relationship between multiple entities in an ecosystem? The idea that our will is in our heads seems suspect at this point: it may well be the case that agency is a relational force between individuals in a network.

The policy consequences here are interesting to think through, and there seems to be a whole catalogue of possible questions for us to explore and think through. Here are a few.

  • How should we regulate artificial agency? Should we assume that agency also requires consciousness and so opens the question about rights – or do we think that consciousness is a third independent quality, or one that is generate in a different way from agency? Indeed — is agency required for consciousness, and the other way around?
  • The larger problem will be hybrid, or mixed agency of different kinds. As we delegate to agents we will need to explore the forms of delegation closely. There seems to be a salient difference between missions and roles, for example. In the first case I am tasking an agent with doing something for me – and extending my agency. In the second I am asking the agent to play a role for me, and so extending my agency and my presence in interesting ways. This notion of artificial presence seems to suggest a series of interesting problems as well.
  • What is the relationship between autonomy and agency? Here we can imagine someone who has an agent that is so good as to allow that agent – or set of agents – to play roles and perform acts of different kinds in such a way that the majority of the exercised agency is artificial, even if it is anchored in the individual human agency. Is this person still autonomous? Is all that is required for autonomy an “agency anchor”? A special variant of this problem is the challenge that a very, very good agent represents: when it performs missions and roles so well as to make me feel that exercising my agency would dilute the outcomes or worsen them in some way, do I still have autonomy even if the autonomy only is to lower the quality of the acts the agent performs?
  • A lot of human law is based on a necessary fiction of agency – we cannot allow for biological defences that reduce our agency to biological functions so the law operates on the necessary fiction of complete individual agency, with a few exceptions. Do we need new exceptions? Do we need to posit stronger versions of the fiction of an agency in order for legal systems to remain robust as agency becomes more and more mixed?
  • How do we deal with collective agency of different kinds? We often think about agents as a 1:1 technology. I have an agent that does things for me, and then we figure out how to regulate that. But is that really what we should be looking at here? Or should we assume that people will have entire cabinets of agents and that there may be such a thing as the collective mixed agency of these agents and myself? What happens with identity when different cabinets, belonging to different people, interact. Where does the locus of agency and accountability reside in distributed agents systems?

And these are only some of the interesting problems that agency will present for us. It seems obvious that this will be a rich and interesting field to explore further, and that agency may play a much larger role in understanding the long term feasibility of the artificial intelligence project.

Notes

  • 1
    See eg Tomasello, Michael. The evolution of agency: behavioral organization from lizards to humans. MIT Press, 2022.
  • 2
    There is something here about a mechanistic and a biological view of the world. The idea that intelligence and agency can be broken down into separate algorithms and sliced into smaller pieces is challenged by a world in which life and intelligence may be algorithmic, but not compressible. This is a line of thinking that is found in, for example, some of Stuart Kauffman’s work. Where Penrose et al seemed to focus on whether or not there was something non-algorithmic that we can do that cannot be replicated, we could instead say that the algorithms that make us up are so complex, entangled and shifting that they are hard to replicate in any meaningful way — this would provide another kind of boundary condition for artificial intelligence.
  • 3
    Johannes Jaeger seems to be building out a criticism along these lines here: https://arxiv.org/abs/2307.07515 – his argument seems to confuse the algorithmic nature of intelligence and the complexity and primacy of agency, though – but this deserves a more careful read. The idea of algorithmic mimicry is an interesting one.
  • 4
    Nietzsche’s view of the will to power, of will as a foundational life force is of course both problematic and protean, but there is a likeness here that needs to be recognised.
  • 5
    The first person I noted using this term was David Krakauer, and he used it to provide a demarcation criterion for complexity science – the idea that complexity science deals with agency, with teleonomic matter, is fascinating. The challenge with the idea of a telos is of course that it is specific, and the telos we are exploring here may start out as much more of a tone or field.
  • 6
    This seems to bear more than a passing resemblance to the Husserlian way of thinking about the life world.

The metaphorical trap

Nicklas LundbladLeave a comment

Here is an argument that I have been noodling on lately. I am not sure I agree entirely with it, but I think it is worth considering.

It is easy to believe that you are a machine, and it gets easier the more complex the machine is. If the machine can also mimic some part of our behaviour it becomes almost impossible not to fall into the metaphorical trap and assume that mind being as machine means that mind is a machine.

The shift from as to is is a small slip of the mind, but tempting for us as we are always trying to understand ourselves.

Yet, we are not machines, not even very complex ones. We are human beings.

This is not a grand statement of human exceptionalism (we should be wary of those as well), but rather a simple scientific fact. We are a part of in incompressible algorithm, evolution, that unfolds in a network of ecosystems. The machines we compare with are not.

Now, it is easy to lose sight of this when we speak of things like “artificial intelligence”, since this, and many other concepts like it, come with a silent third term – “human”. When we say “artificial intelligence”, well, we hear “artificial human intelligence” and so we get stuck in the metaphorical trap again.

What has emerged in AI today are extra-biological intelligences – but this does not mean that they are less interesting. On the contrary, they are stranger and more wondrous than we think. How come something that is so unlike us can behave so much like us? That is the question – not if we are mere machines.

And this matters for everything – philosophy, ethics, policy, economics and our overall predictions for this technology.

Energy and innovation (Questions)

Nicklas LundbladLeave a comment

It seems clear that the ability we have as a civilization to capture energy is directly related to the space of possible inventions we can unlock. If we wanted to do a classical technology tree, we would find, at the joints, the ability to capture energy in different ways. Som inventions are much more likely, at least, in a world that has harnessed the energy of fossil fuels.

More likely, yes – but is it a strict limit? Could you imagine a world in which parts of our technology tree were unlocked without fossil fuels? What would a scenario look like where nuclear power was discovered before the fossil fuel engine? Perhaps it is not so much a question about the source of the energy as the total sum of energy available to us as a civilization.

The concept of Kardashev-classes is interesting here. It was launched by Nikolai Kardashev, Soviet astronomer, in 1964 and essentially proposed three types of civilizations:

  • Type 1. Harnesses all the energy that reaches the planet from its parent star. Some calculations has this at 4 orders of magnitude above where we are now. Some put this at around 1016 to 1017 watts.
  • Type 2. Harnesses all the energy of the parent star. We are now at 4 x 1026 watts.
  • Type 3. Harnesses the energy in its parent galaxy. 4×1037 watts)

The different types are interesting thought experiments for sure, but they also allow us to ask a question about invention and innovation. Are there some innovations that we can conceive of in a Kardashev type 1 civilization but that really requires a type 2 civilization to work? The most obvious example could be widespread, planetary artificial intelligence. In order to work persistently, evolve and manage complex systems, such a technology may need type 2-energy capacity.

There would be, then, a special kind of tragedy here – the ability to understand and design a technology without meaningfully being able to deploy it – because of the energy shortage. If so, that would be interesting to explore as a scenario.

The question here, then, would be: is there technology that requires Type 2+ energy capture to be viable large scale, but can be designed in a type 1 civilization?

Data is not like oil – it is much more interesting than that

Nicklas Lundblad1 Comment

So, this may seem to be a nitpicking little note, but it is not intended to belittle anyone or even to deny the importance of having a robust and rigorous discussion about data, artificial intelligence and the future. Quite the contrary – this may be one of the most important discussions that we need to engage in over the coming ten years or so. But when we do so our metaphors matter. The images that we convey matter.

Philosopher Ludwig Wittgenstein notes in his works that we are often held hostage by our images, that they govern the way we think. There is nothing strange or surprising about this: we are biological creatures brought up in three-dimensional space, and our cognition did not come from the inside, but it came from the world around us. Our figures of thought are inspired by the world and they carry a lot of unspoken assumptions and conclusions.

There is a simple and classical example here. Imagine that you are discussing the meaning of life, and that you picture the meaning of something as hidden, like a portrait behind a curtain – and that discovering the meaning then naturally means revealing what is behind that curtain and how to understand it. Now, the person you are discussing it with instead pictures it as a bucket you need to fill with wonderful things, and that meaning means having a full bucket. You can learn a lot from each-others’ images here. But they represent two very different _models_ of reality. And models matter.

That is why we need to talk about the meme that “data is like oil” or any other scarce resource, like the spice in Dune (with the accompanying cry “he who controls the data…!”). This image is not worthless. It tells us there is value to data, and that data can be extracted from the world around us – so far the image is actually quite balanced. There is value in oil and it is extracted from the world around us.

But the key thing about oil is that there is not a growing amount of it. That is why we discuss “peak oil” and that is why the control over oil/gold/Dune spice is such a key thing for an analysis of power. Oil is scarce, data is not – at least not in the same way (we will come back to this).

Still not sure? Let’s do a little exercise. In the time it has taken you to read to this place in the text, how many new dinosaurs have died and decomposed and been turned into oil? Absolutely, unequivocally zero dinosaurs. Now, ask yourself: was any new data produced in the same time? Yes, tons. And at an accelerating rate as well! Not only is data not scarce, it is not-scarce in an accelerating way.

Ok, so I would say that, wouldn’t I? Working for Google, I want to make data seem innocent and unimportant while we secretly amass a lot of it. Right? Nope. I do not deny that there is power involved in being able to organize data, and neither do I deny the importance of understanding data as a key element of the economy. But I would like for us to try to really understand it and then draw our conclusions.

Here are a few things that I do not know the answers to, and that I think are important components in understanding the role data plays.

When we classify something as data, it needs to be unambiguous, and so needs to be related to some kind of information structure. In the old analysis we worked with a model where we had data, information, knowledge and wisdom – and essentially thought of that model as hierarchically organized. That makes absolutely no sense when you start looking at the heterarchical nature of the how data, information and knowledge interact (I am leaving wisdom aside, since I am not sure of whether that is a correct unit of analysis). So something is data in virtue of actually having a relationship with something else. Data may well not be an _atomic_ concept, but rather a relational concept. Perhaps the basic form of data is the conjunction? The logical analysis of data is still fuzzy to me, and seems to be important when we live in a noise society – since the absolutely first step we need to undertake is to mine data from the increasing noise around us and here we may discover another insight. Data may become increasingly scarce since it needs to be filtered from noise, and the cost for that may be growing. That scarcity is quite different from the one where there is only a limited amount of something – and the key to value here is the ability to filter.

Much of the value of data lies in its predictive qualities. That it can be used to predict and analyze in different ways, but that value clearly is not stable over time. So if we think about the value of data, should we then think in terms of a kind of decomposing value that disappears over time? In other words: do data rot? One of the assumptions we frequently make is that more data means better models, but that also seems to be blatantly wrong. As Taleb and others have shown the number of correlations in a data set where the variables grow linearly in turn grows exponentially, and an increasing percentage of those correlations are spurious and worthless. That seems to mean that if big data is good, vast data is useless and needs to be reduced to big data again in order to be valuable at all. Are there breaking points here? Certainly there should be from a cost perspective: when the cost C of reducing a vast data set to a big data set are greater than the expected benefits in the big data set, then the insights available are simply not worth the noise filtering required. And what of time? What if the time it takes to reduce a vast data set to a big data set necessarily is such that the data have decomposed and the value is gone? Our assumption that things get better with more data seems to be open to questioning – and this is not great. We had hoped that data would help us solve the problem.

AlphaGo Zero seems to manage without at least human game seed data sets. What is the class of tasks such that they actually don’t benefit from seed data? If that class is large, what else can we say about it? Are key crucial tasks in that set? What characterizes these tasks? And are “data agnostic” tasks evidence that we have vastly overestimated the nature and value of data for artificial intelligence? The standard narrative now is this: “the actor that controls the data will have an advantage in artificial intelligence and then be able to collect more data in a self-reinforcing network effect”. This seems to be nonsense when we look at the data agnostic tasks – how do we understand this?

One image that we could use is to say that models eat data. Humor me. Metabolism as a model is more interesting than we usually allow for. If that is the case we can see another way in which data could be valuable: it may be more or less nutritious – i.e. it may strengthen a model more or less if the data we look at becomes part of its diet. That allows to ask complicated questions like this: if we compare an ecology in which models get to eat all kinds of data (i.e. an unregulated market) and ecologies in which the diet is restricted (a regulated market) and then we let both these evolved models compete in a diet restricted ecology – does the model that grew up on an unrestricted diet then have an insurmountable evolutionary advantage? Why would anyone be interested in that, you may ask. Well, we are living through this very example right now – with Europe a, often soundly, regulated market and key alternative markets completely unregulated – with the very likely outcome that we will see models that grew up on unregulated markets compete with those that grew up in Europe, in Europe. How will that play out? It is not inconceivable that the diet restricted ones will win, by the way. That is an empirical question.

So, finally – a plea. Let’s recognize that we need to move beyond the idea that data is like oil. It limits our necessary and important public debate. It hampers us and does not help in understanding how this new complex system can be understood. And this is a wide open field, where we have more questions than answers right now – and we should not let faulty answers distract us. And yes, I recognize that this may be a fool’s plea, the image of data like oil is so strong and alluring, but I would not be the optimist I am if I did not think we could get to a better understanding of the issues here.