We are all tulpamancers now

One of the key features of interaction is the sense of presence. We immediately feel it if someone is not present in a discusison and we often praise someone by saying that they have a great presence in the room – signalling that they are influencing the situation in a positive way. In fact, it is really hard to imagine any interaction without also imagining the presence within which that interaction plays out.

In Presence: the strange science and true stories of the unseen other (Manchester University Press 2023), psychologist Ben Alderson-Day explores this phenomenon in depth. From the presence of voices in people who suffer from some version of schizophrenia to the recurring phenomenon of presences on distant expeditions into harsh landscapes, the authors explores how presence is perceived, and to some degree also constructed. One way to think about this is to say that presence is a bit like opening a window on your virtual desktop, it creates the frame and affordances for whatever next you want to do. The ability to construct and sense presence is absolutely essential for us if we want to communicate with each-other, and it is ultimately a relational phenomenon.

Indeed, the sense of a presence in an empty space, on a lonely journey or in an empty house may well be an artefact of the mind’s default mode of existing in relationship to others. We do not have unique minds inside our heads – our minds are relationships between different people and so we need that other presence in order to think, an in order to be able to really perceive the world. So the mind has the in-built ability to create a virtual presence where no real presence exists. 

One of the most extreme examples of this is the artificially generated presence of the tibetan Tulpa. A Tulpa is a presence that has been carefully built, infused with its own life and intentions and then set free from our own minds, effectively acting as another individual, but wholly designed by ourselves. We are all, to some degree, tulpamancers – we all know how to conjure a Tulpa – since we all have the experience of imaginary friends. These imaginary friends allow us to practice having a mind with another in a safe environment, and so work as a kind of beta testing of the young mind. 

All of this takes an interesting turn with the emergence of large language models, since we now have the ability to create something that is able to have a presence – and interact with these new models as if they were intentional. An artificial intelligence is only possible if it also manages to create an artificial presence, and one of the astonishing things about large language models is that they have managed to do so almost without us noticing. The world is now full with other presences, slowly entering into different kinds of interactions with us. We are, in some sense, all tulpamancers again, building not imaginary friends, but perhaps virtual companions. 

There are many reasons to be attentive to this development, not least because we want to make sure that people do not confuse a language model for a real human being. The risks associated with such confusion are easy to see – since what it essentially would mean is that we co-create our mind with an entity that is vastly different from us. A language model has not evolved, it is not embodied and has no connection to the larger eco-system we exist in. It’s presence is separate, almost alien, but we still recognise it as a presence. 

We can compare with dogs. A dog projects presence in a home, and it seems clear that we have human/dog minds at least if we are dog owners. If you grew up with a dog you can activate that particular mode of mind when you meet a dog and it is often noticeable when people “are good with animals” or have a special rapport with different kinds of pets. This ability to mind-share in a joint presence is something humankind has honed over many, many generations of co-evolution. You could even argue that this ability now is a human character trait, much like eye color or skin tone. There are those that completely lack this ability and those that have an uncanny connection with animals and manage to co-create minds with all kinds. 

The key takeaway from this is that the ability to co-create a mind with another is an evolved capability, and something that takes a long time to work out. There are, in addition, clear mental strengths that need to be developed. Interacting with a dog requires training and understanding the pre-conditions and parameters of the mind you are co-creating. 

We can generalise this and note that our minds are really a number of different minds created in different presences, all connecting to a single set of minds that we compress into the notion of an I. This is what we mean when we say things like “I am a different person with X” or “You complete me” or cast ourselves in different roles and wearing different masks in different contexts. What is really going on is not just that we are masking an inner secret self, but we are really different with different people, the minds we co-create with them are us, but also not us. The I is secretly a set of complex we:s, and the pre-condition for creating that we is presence. 

What does this mean, then, for artificial intelligence and how we should think about language models? As these models get better, we are likely to be even more enticed to co-create minds with them and interact with them in ways that are a lot like the ways in which we interact with each-other. But we need to remember that these artefacts are really more like our imaginary friends than our real relationships – and we probably need to develop what researcher Eric Hoel calls a set of intrinsic innovations – mental skills – that help us interact with these models. 

A lot of how we think about these models now is about how we think we can fix the models so that they say nothing harmful and do nothing that is dangerous. We are treating these technologies as if they were mechanical, but they are more than that – they are intentional technologies, technologies that can create presence and a sense of intent. This means that we may need to complement our efforts on creating safety mechanisms in the machine, with creating safety mechanisms in our minds.

There is, then, an art to co-creating a mind with a language model – and it is not something we are naturally good at, since they have not been around for long. And this art reminds us of a sort of tulpamancy – the knowing construction of an artificial presence that we can interact with in different ways. A conscious and intentional crafting of an imaginary friend. One part, then, of safety research also needs to be research into the mental techniques that we need to develop to interact with artificial presences and intentional systems. And it is not just about intellectual training – it is about feeling these presences and intentional systems, understanding how they co-opt age old evolutionary mechanisms for creating relational minds and figuring out ways in which we can respond mentally to ensure that we can use these new tools. It requires a kind of mentalics to interact well with, and co-create functional and safe minds with, artificial intelligence. 

A surprising conclusion? Perhaps. But the more artificial presences and intentional artefacts we build, the more attention we need to pay to our own minds and how they work. We need to explore how we think and how we think with things, people, presences and other tools. Artificial intelligence is not a substitute for our intelligence, but a complement – and for it to really be that complement we need to develop the skills to interact with such technologies.

It is not unlike learning to ride a bike or driving a car. A lot of the training there is the building of mental constructs and mechanisms that we can draw on, and this is something we need here too. How we do that is not clear – and I do think that we need research here – but some simple starting points can be meditation, a recognition of the alien nature of the presences created by these models and conscious exploration of how the co-created minds work, where they behave weirdly and where they are helpful. It requires a skillful introspective ability to do so, and such an ability is probably useful for us overall in an evermore complex world. 

We are all tulpamancers now. 

The summarised society

One of the things that generative AI will enable is the summarisation of the growing flows of information that we all live in. This is not surprising to the reader of Herbert Simon, who suggested that with a wealth of information comes a poverty of attention and a need to allocate attention efficiently. Now, what it does help us understand is that attention allocation can be achieved in a multitude of ways. The first is to help us focus attention on the right piece of information – this is essentially what a recommendation algorithm does. The second is to focus attention on the right features, and at the right resolution, in an information set. This is what a summarising algorithm does.

Summaries have been around for ages, most of them produced by people – and so they are not new in themselves. The time it takes to summarise a field has grown, however, and today there are several fields of research and knowledge that are impossible to summarise well before they change in fundamental ways. The limits of summarisation also limit how we can update our knowledge.

Now, if we believe that the quality of our decisions depend on the quality of the information we draw on when we make those decisions, this should worry us. It seems we could make better decisions if we had access to summaries of different fields as they evolve – at least if it is true that these summaries can be made in such a way that they capture the salient features of the evolving information landscape that are relevant for the decisions that we want to make. Are such summaries possible? Is the tendency to hallucinate that generative AI has a fatal flaw in producing such summaries? This is increasingly the focus of research.

The paper “Improving Primary Healthcare Workflow Using Extreme Summarization of Scientific Literature Based on Generative AI” proposes a summarisation approach to address the challenge faced by primary care professionals in keeping up-to-date with the latest scientific literature.

The researchers employed generative artificial intelligence techniques based on large-scale language models to summarise abstracts of scientific papers. The goal was to reduce the cognitive load experienced by practitioners, making it easier for them to stay informed about the latest developments relevant to their work.The study involved 113 university students from Slovenia and the United States, who were divided into three groups. Each group was provided with different types of abstracts (full abstracts, AI-generated short abstracts, or the option to choose between the two) and use cases related to preventive care and behaviour change.

The findings in the paper suggest that AI-generated summaries can significantly reduce the time needed to review scientific literature. However, the accuracy of knowledge extraction was lower in cases where the full abstract was not available, and this is key — we need to have a sense of what the ideal resolution of the data set used to summarise really is. It seems obvious that summaries made from a set of full papers will be more costly and take longer time, than summaries made from some kind of abstracts. This in turn suggests that we should think about a hierarchy of summaries here, and that there may be an argument for forcing longer abstracts on all papers submitted, so that the abstracts are more legible for AI-models that summarise them!

The design of summaries will quickly become a key element in how we learn about things.

In another example the paper titled “Summaries, Highlights, and Action items: Design, implementation and evaluation of an LLM-powered meeting recap system” explores the application of Large Language Models (LLMs) to improve efficiency and effectiveness of online meetings. The researchers designed, implemented, and evaluated a system that uses dialogue summarisation to create meeting recaps, focusing on two types of recap representations: important highlights, and a structured, hierarchical minutes view.

The study involves seven users in the context of their work meetings and finds that the LLM-based dialogue summarisation shows promise. However, it also uncovers limitations, such as the inability of the system to understand what’s personally relevant to participants, a tendency to miss important details, and the potential for mis-attributions that could affect group dynamics – but does that matter?

What is interesting is that there may be a quite low threshold for the quality of summaries for many readers (this will vary of course) and so even summaries that have these limitations could easily be valuable if you miss a meeting and no notes were taken. To preserve privacy, we would also have to think through different kinds of limits on attribution here.

One way to think about summarisation algorithms is to suggest that they compress information – and that raises an interesting question about how much information can be compressed for different purposes. How much can a meeting be compressed? This question quickly turns very funny, since we have all been in meetings that could have been compressed not even to emails, but into the sentence “we don’t know what we are doing, really” – but there is a serious use for it as well: we should look at the information density of our different activities.

One area that is over-ripe for information compression is email. If you look at your inbox in the morning and imagine that you could summarise it into a few items – how much would then actually be lost? I often find different emails on the same subject, and would love the ability to just ask my inbox to summarise the latest on a subject with views and action items. That would allow my a “project view” of my email and I would be able to step back and track work streams. The fact that email is not already the subject of massive efforts of summarisation and compression is somewhat baffling.

You could also allow for different summarisation views – summarise on individual, on project or on a topic. Summarise on emotional content — give me all the angry emails first. There are endless opportunities. One example of this idea – in this case to summarise on topic – is found in the paper titled “An End-to-End Workflow using Topic Segmentation and Text Summarisation Methods for Improved Podcast Comprehension” where authors combine topic segmentation and text summarisation techniques to provide a topic-by-topic breakdown, rather than a general summary of the entire podcast.

Now, you may be tempted to test this by inputting your inbox headlines into a chatbot – but you shouldn’t, remember that your email is probably filled with somewhat sensitive information. But that you may be tempted show something: even just headline summaries would sometimes be helpful – or would highlight how bad we are at writing good subject headings in email.

Summarisation naturally also carries risks – summaries destroy information, and nuance – and you could imagine second and third order effects to a society that consumes summaries rather than the original thing – the lack of nuance could be disastrous for some classes of decision (legal decision making comes to mind). This is true for all shifts and changes in attention allocation, however – since society is made of attention.

Recommendation algorithms and summarisation algorithms are just two dimensions here. How we re-design, necessarily re-design, we should say, attention allocation will change society too.

Studying the mereologies of AI

In this article Robin Hill suggests something that may seem both obvious and strange at the same time: that our artificial intelligence systems might not cut the world up in the same way we do, or that they may not use the same features to cluster concepts as we do. The example she gives is a dog – we recognize it by looking at the fur, ears, wet nose etc – those are the features we focus on – but why should the machine focus on them? Why should we assume that it divides the world into wholes and parts the way we do?

This is a deep question, one associated with the often neglected subject of mereology1 The sciences of wholes and parts, see for example https://plato.stanford.edu/entries/mereology/ – a very useful way of thinking about concepts. In mereology we recognise that wholeness of the “dog” and the parts that go into constructing that whole, but we also realise quickly that there are many, many different ways in which we could construct that particular whole. This includes lumping together or slicing things differently across a number of different dimensions, including a temporal dimension. The dog may be made of moments in space.

We could hypothesize that an all-knowing super-intelligence might actually converge to finding patterns in the particle paths through space-time, and so would recognize entirely different concepts than we do. A clustering of such paths may conjure the concept of “Wegobans” who represent certain particle paths through space time with strong commonalities that we cannot even begin to guess.

Now, the way we slice and lump (to use the evocative language that Lee Anne Fennell uses in her excellent book Slices & Lumps: Divisions and Aggregation in Law and in Life (2019)) the world is not arbitrary, it is rooted in evolution. Our concepts have evolved for a function – as Ruth Millikan has shown – and so we should expect human mereology to follow from that evolutionary path. But what if that particular mereology is not preserved in the training of large language models? What if that is a feature that is lost in the methods we currently use?

What would that mean?

We get into deep philosophy of language territory here – there seems that there is the chance that we end up in false communication patterns, where the symmetry of the way we use the signs convince us that we mean the same, but the structural composition of those signs into wholes and parts are radically different.

This is something like what Nelson Goodman suggested with his grue/bleen-experiment2 See the “new riddle of induction” here https://en.wikipedia.org/wiki/New_riddle_of_induction , where a concept can be composed in arbitrarily many different ways – or at least along the temporal dimension of the concept3 One interesting question is of course if there are general dimensions along which a mereology is constructed, and if that can be used to explore alternative mereologies?. That is in turn interesting because there will be, in any such faux communication, points where the difference in the composition of the signs lead to drastic break-downs in the ability to convey meaning to each-other. 4 A simple example may be one in which I mean “allergy” to be a medical condition that only applies before the 31st of January 2024 – a far-fetched example, but still.

And again, it is almost obvious to point out – but the study of the mereologies of artificial intelligences will be an absolutely essential piece of getting both security and safety right. This is slightly different from the study of explainability, where we look just for a mapping of systems to our own mereology so we can translate – and more fundamental: it is the view that we should look for general principles of mereological composition in large language models.

As we do so there are a number of interesting questions to consider, such as:

  • Do we believe that AI mereologies become more like human mereologies as the size of the training data set grows? Or could the relationship between human / AI mereologies be, say, U-shaped? Why?
  • Are there fundamentals in mereology that have to do with perception, and if so – does this mean that when we add sensors to AI that represent no human sensing capabilities we will end up with vastly different mereologies (cf what it may be to be a bat, as Nagel asks — what is the mereology of a bat – are the commonalities rooted in evolution in some way?)
  • What do mereological safety risks look like and how can they be addressed in the best way?
  • Is shared mereological composition a pre-condition for alignment?

And so on.

Notes

  • 1
    The sciences of wholes and parts, see for example https://plato.stanford.edu/entries/mereology/
  • 2
    See the “new riddle of induction” here https://en.wikipedia.org/wiki/New_riddle_of_induction
  • 3
    One interesting question is of course if there are general dimensions along which a mereology is constructed, and if that can be used to explore alternative mereologies?
  • 4
    A simple example may be one in which I mean “allergy” to be a medical condition that only applies before the 31st of January 2024 – a far-fetched example, but still.

Invisible commons

Are there some commons that we should protect that we are just not aware of? Well, for the longest time the idea that there was a “climate” to protect was not obvious to people – and the overall discussion we now have about climate change required a concept to focus on, and climate ended up being the focal concept.

One way to think about the future is to think about possible focal concepts like this that have not been invented yet. Here are a few possible examples.

  • The Human Text – the sum total of all things written by human beings, not generated by AI. The value of this commons is still unclear to us (why is a text written by a human being in some way more worthy of protection than a generated text?) – but we may discover that there are some qualities in The Human Text that are unique for that corpus. This could of course be true for a lot of other forms of creative outputs too.
  • Social autonomy. The level of overall autonomy that we retain as a society, and the degree to which we are relying on systems to solve our problems for us. How many decisions are made by human beings and not by systems? This is a difficult one, since we usually look to the quality of the decision rather than its origin, but there may well be reasons to value human decision making not just as an artisanal quaint thing, but as better for different reasons.
  • Agency. Closely related – but more about the distribution of agency over systems, perhaps. Evolution creates behavioural complexity through pre-programming (spider’s web) or agency. We now have created a new, artificial agency – post-programming – and human agency may be squeezed in between these two forms of programming. This may make a lot of sense from evolution’s perspective – i.e. adaptive capability may increase when we revert to the ideal state of pre and post programming and avoid messy agency in the middle – but still is interesting to think about the value of agency in the middle. Dennett and others seem to argue that agency in the middle is what underpins almost all of our social institutions, even if it sometimes is a fiction. And maybe this connects to the first one: maybe there is a value in agency-produced text?
  • Fact. This is a common that we often are quite blind to – and feel uncomfortable articulating – but the depletion of facts is an interesting and hard problem to work on. Facts do – as Sam Arbesman points out – have a half-life, so there is a natural depletion – but beyond that depletion the rate at which we destroy this particular commons has to do with education, public sphere norms and many, many other social patterns. What if we started to treat fact as a commons? And yes – you could make the same argument for truth, and analyse the evolution of our relationship to truth as a commons problem.
  • Attention. While deeply individual, in some sense, attention is actually a commons we draw on to build society and democratic institutions. The way in which we manage this commons is determinative of the quality of democracy that we get.
  • Childhood. Do we still have childhoods like we used to? Is childhood a commons that underpins our civilisational growth and prosperity? What about the Global South / North divide in childhoods etc?

I am sure there are many other possible focal concepts as well. These three interest me more as examples than anything else — and the real prediction question is which focal concepts will seem obvious in the future? Animal life quality comes to mind as well, as one that many have guessed at – the idea that we eat animals may seem a lot like insulating with asbestos or torture in the future, they argue. That does not seem wholly unlikely to me, with synthetic meat on the horizon etc. These changes can be relatively swift – how long did it take for climate to emerge as an obvious commons? Arguably this took at most a decade, and then we can debate if it starts as “common heritage of mankind” or with the 1988 establishment of the IPCC, but still.

A lot of these things come back to a more fundamental commons that has remained invisible for a long time – since we believe it is deeply individual: time. As noted by many, the best measure of freedom is probably discretionary time, but that is not quite right. If you are free to use your time alone you may feel individually free – but are you really free in a more fundamental way? If there is not a certain pool or commons of shared discretionary time in a society, it falls apart. This is the argument for what we sloppily call leisure – and we probably need to think much more about this than we are currently.

We are all living in the invisible commons of time.

The role of play

Let’s say that we want to understand if a certain system or phenomenon is exhibiting human-level intelligence. What would we then look for? There are multitudes of such tests, and they look for a great variety of things. One catalogue of such tests is BIG-bench, a battery of task tests that is aimed at understanding the capabilities of different systems. 1 Srivastava, Aarohi, et al. “Beyond the imitation game: Quantifying and extrapolating the capabilities of language models.” arXiv preprint arXiv:2206.04615 (2022).See here for the paper.

BIG-bench looks at a large number of tasks that are both creative and difficult, and it is a fascinating catalogue – but it does not specifically look at if the system plays. Yet, the ability to play is probably fundamental to human intelligence – not just incidental or a kind of idling mode. 2 See for example Fein, Greta G. “Skill and intelligence: The functions of play.” Behavioral and Brain Sciences 5.1 (1982): 163-164. exploring the adaptiveness of play and the setting of intrinsic limitations.

Would it then not make sense to look at if there is a way to test for play? To see if an intelligence can play? Well, you could argue that it seems quite clear that LLMs can play with us – we can ask it to play any game where we give it the rules, and we can even ask it to make up a game about something and it will do so. But that is not quite what we are looking for. What we want to see is if a system starts playing itself – if it can be, in some sense, curious enough to set intrinsic limitations and obstacles for itself and find joy in playing a game that it has invented for itself.

This notion of motivation by curiosity also seems a little trivial, you just need to build a system that has an instruction to play games when it does nothing else, right? Not quite. When we play and how we play are not random behaviours. Play is not the equivalent of a human screensaver. Instead, play is something that is related to building models of the world, planning ahead, remembering systems of rules — it is much more fundamental.

Games may play a much larger role in intelligence than we think, and the role of play is still being explored conceptually in philosophy. 3 See e.g. Nguyen, C. Thi. “Philosophy of games.” Philosophy Compass 12.8 (2017): e12426. and the same author’s subsequent book Nguyen, C. Thi. Games: Agency as art. Oxford University Press, USA, 2020. . Maybe the study of complex systems like models and their behaviour should focus not only on problem solving, but the ways in which agency is exhibited and structured in play and games.

There is an irony here – in a way – since a lot of research has focused on how to play games and to do so until the machine can beat human players. Maybe that was almost right, and the greater question is not how to play, but when and why we play and testing for that in a different way.

There is much more to be explored here – like how our narrow understanding of rationality and intelligence obscures the adaptive value and role of things like play and games, or how games – as mental models – structure the way we learn and understand the world.

Notes:

  • 1
    Srivastava, Aarohi, et al. “Beyond the imitation game: Quantifying and extrapolating the capabilities of language models.” arXiv preprint arXiv:2206.04615 (2022).See here for the paper.
  • 2
    See for example Fein, Greta G. “Skill and intelligence: The functions of play.” Behavioral and Brain Sciences 5.1 (1982): 163-164. exploring the adaptiveness of play and the setting of intrinsic limitations.
  • 3
    See e.g. Nguyen, C. Thi. “Philosophy of games.” Philosophy Compass 12.8 (2017): e12426. and the same author’s subsequent book Nguyen, C. Thi. Games: Agency as art. Oxford University Press, USA, 2020.

Artificial Strange Intelligence

Here is a common thought model: artificial intelligence will become more and more intelligent, and reach human capability. At that point it becomes something like an artificial general intelligence and some time after that it will use that intelligence to bootstrap itself into an artificial super intelligence. The model then is one-dimensional: [AI-AGI-ASI]

Now, there is a problem with this model, and that is that it anchors on the idea that intelligence is a one-dimensional capability, a bit like height. We imagine intelligence almost like the height of a child, an adolescent and then an adult. But that surely must be a simplification?

Intelligence as a concept is tricky. There are many different definitions 1 See e.g. Legg, Shane, and Marcus Hutter. “A collection of definitions of intelligence.” Frontiers in Artificial Intelligence and applications 157 (2007): 17. – and the number of definitions have not become fewer in the last 15 years and the usefulness of the concept can be seriously questioned – even if we can hope to find working definitions that can be better than some other definitions 2 See e.g. Wang, Pei. On the working definition of intelligence. Technical Report 94, Center for Research on Concepts and Cognition, Indiana University, 1995. .

One approach here is to abandon the concept of intelligence and instead speak about capability or competence. A system can become increasingly capable or competent – and that is what we care about. When a system is as capable as a human being in a domain we have human-equivalent capability. That is enough. We do not need to call it intelligence. This also allows us to escape the metaphorical morass that follows with using the term “intelligence” since it is so closely entwined in a language game that deals with human intentionality.

This is hardly a revolutionary approach – capability language is already a key component in many discussions about artificial intelligence, so why would we make this shift? One reason is that it helps us explore a discussion about the different dimensions at play here. We can agree that systems become more capable, but we should still ask if they also develop in other dimensions.

One well-known such dimension that concerns a lot of researchers is explainability. As the systems become more capable they become less explainable and there seems to be a trade-o ff there. That brings another tricky concept into our midst – explanation – and forces us to discuss exactly what it means to explain something. This is not a trivial question, either, but a live philosophical debate that is largely undecided. We know that there are different stances / levels of explanation that we can explore – from physicalistic to intentional in Dennett’s works for example – and that explanations can be evaluated in a multitude of ways. 3See Dennett, Daniel C. The intentional stance. MIT press, 1989 but also early works like Thagard, Paul, Dawn M. Cohen, and Keith J. Holyoak. “Chemical Analogies: Two Kinds of Explanation.” IJCAI. 1989, where different kinds of explanations are introduced. The philosophical standard works include books like Von Wright, Georg Henrik. Explanation and understanding. Cornell University Press, 2004 (1971) and Apel, Karl-Otto. Understanding and explanation: A transcendental-pragmatic perspective. MIT Press, 1984.

Instead of explainability, then, we could just use another term – and one that is not just about explaining why or what a technical system is doing, but the degree to which we understand it. I would suggest that we could call that quality simply strangeness.

What emerges is a re-phrasing (but I think a useful one) where we can explore how capability and strangeness develop over time in complex systems of the type we are now building.

Now, there are different hypotheses here that we can explore – but the reason this interests me is that I wonder if systems become stranger faster than they become capable. We can then imagine different system paths across this space and discuss if there are general paths that we can expect or that would be interesting to understand. System A and system B in this picture are interesting examples:

System A goes through a period of strangeness before becoming much more capable, and system B is strange to begin with and then becomes very capable. And looking at this we can then play around with concepts like artificial general intelligence and artificial super intelligence and artificial strange intelligence:

This would be a good reminder that as systems become more complex, they do not only become more capable, but they also become more inscrutable and inaccessible to our understanding – but at the same time, a stranger in our midst can be a way for us to see ourselves anew. 4 See for example Tuan, Yi-Fu. “Strangers and strangeness.” Geographical Review (1986): 10-19, who notes that there is a kind of grace to strangeness . And the etymology here is also satisfying: “from elsewhere, foreign, unknown, unfamiliar, not belonging to the place where found” – a good reminder that we are dealing not with a replication of the familiar, ourselves, but the uncovering of something truly strange.

Notes

  • 1
    See e.g. Legg, Shane, and Marcus Hutter. “A collection of definitions of intelligence.” Frontiers in Artificial Intelligence and applications 157 (2007): 17. – and the number of definitions have not become fewer in the last 15 years
  • 2
    See e.g. Wang, Pei. On the working definition of intelligence. Technical Report 94, Center for Research on Concepts and Cognition, Indiana University, 1995.
  • 3
    See Dennett, Daniel C. The intentional stance. MIT press, 1989 but also early works like Thagard, Paul, Dawn M. Cohen, and Keith J. Holyoak. “Chemical Analogies: Two Kinds of Explanation.” IJCAI. 1989, where different kinds of explanations are introduced. The philosophical standard works include books like Von Wright, Georg Henrik. Explanation and understanding. Cornell University Press, 2004 (1971) and Apel, Karl-Otto. Understanding and explanation: A transcendental-pragmatic perspective. MIT Press, 1984.
  • 4
    See for example Tuan, Yi-Fu. “Strangers and strangeness.” Geographical Review (1986): 10-19, who notes that there is a kind of grace to strangeness

On the possibility of model ethology

In considering the potential for an ethology of artificial intelligence (AI), we are venturing into uncharted territory. The idea of observing and studying AI behavior as we would an animal’s seems, at first glance, to be an attempt to anthropo- or biomorphize technology. However, as AI systems like large language models become increasingly complex, the need to understand these systems through their behavior becomes more pressing. In fact, the complexity and opaqueness of these systems often necessitate an approach akin to ethology, the study of animal behavior.

This also goes to the heart of the question around explainability. What an explanation really is, is a surprisingly deep question. One fairly clear view in philosophy is that explanations exist at different levels – and ethology is a way to frame a nested hierarchy of explanations into a comprehensive explanation of behavior over all. An ethological explanation is composed of layers of explanations.

One formulation of this is that of Niko Tinbergen, who approaches animal behavior from four primary angles: adaptation, phylogeny, mechanism, and ontogeny. Applying these lenses to the study of AI presents unique challenges, but it also potentially offers us new ways of understanding these complex systems – a nested system of explanations.

The first question of adaptation–why the AI is performing the behavior–might seem straightforward in the case of AI, as its behaviors are generally determined by human-defined goals and objectives. However, as AI becomes more complex and capable of self-learning, this question becomes more intricate. The AI may develop behaviors that optimize its performance in unexpected ways, making the question of why it is behaving in a certain way less straightforward.

Phylogeny, or the evolutionary history of a behavior, is a concept that might seem out of place when applied to AI. Unlike biological entities, AI does not evolve through natural selection over generations. However, AI does have a form of phylogeny if we consider the iterative development and improvement of AI models as a kind of evolution. Understanding the ‘phylogeny’ of an AI system could involve tracing back its development, examining the various iterations it went through, and the changes made at each step.

Mechanism, or what triggers the behavior, is perhaps the easiest to apply to AI, as it involves the inner workings of the system. However, as AI systems grow more complex and their decision-making processes more opaque, understanding these mechanisms becomes more challenging. This is particularly the case with machine learning models where the exact decision-making process is not always easily extractable or understandable.This is where traditional explainability questions often end up.

Finally, ontogeny, or how behavior develops over an individual’s lifetime, presents another intriguing lens through which to view AI. While AIs do not have lifetimes in the biological sense, they do have operational lifespans during which their behavior can change significantly, especially in the case of learning models. Observing how an AI’s behavior changes over time, in response to new data or changes in its operating environment, could provide valuable insights into its functioning.

So, can we develop an ethology of AI? The application of Tinbergen’s questions to AI is certainly challenging, but not impossible. While some aspects of these questions require modification or reinterpretation, they provide a potentially fruitful framework for understanding the behavior of complex AI systems. In fact, they might become necessary tools given the increasing complexity of these systems.

Refusing this approach, we risk confining ourselves solely to mechanistic explanations of AI behavior. Such a narrow view would limit our understanding of AI and potentially blind us to significant aspects of their operation and implications. By embracing an ethological approach, we can gain a more holistic and nuanced understanding of these complex systems, aiding us in their development, management, and integration into society. As we continue to create and interact with increasingly sophisticated AI, the need for such understanding will only grow more urgent.

Now, we can also take this another level and study man-machine complex systems, and the resulting ethology of technology in use here. This will allow us to find even better ways of understanding risks and weaknesses in the systems. Our focus on understanding how the technical systems work is misplaced when the reality is that it is the technologies in human use we need to understand – and when we defer to systems, and how they can be designed to avoid that we defer to them when we – and they – have the least understanding of what is going on. 1 See this intriguing argument for models stating they do not know when their confidence levels sink below a certain threshold.

This mix of anthropology and ethology could well develop into a key skill in the very near future.

  • 1
    See this intriguing argument for models stating they do not know when their confidence levels sink below a certain threshold.

A note on the pace of technical development

One of the most common, yet unexamined, concepts in technology policy is the idea of a “pace of technical development”. Usually, it is described as “fast” and taken to mean that we need to make interventions in a timely manner. This is all good and well, but what does it mean to say that the pace of innovation or technical development is fast? Relative to what? One way to approach that question is to say that we should look at two kinds of speed, and we can do that by employing a very simple mental model: that of the alphabet. 1 This notion derives from a talk by Ricardo Hausman on knowledge and economy.

The alphabet is essential in developing textual space – the space of texts that we have actually written and that can be used for different purposes.2 There is an important difference between the texts that could be written and the texts we have actually written. When we speak of the pace of textual development (the analogue here of technical development), we immediately realise that it depends on two different things: how many actual letters there are to combine and how many combinations we have made. The invention of a new symbol or letter allows us to create more words, and hence more texts.

The textual space, then, develops through capability and combination.

This is not unlike biological evolution – the idea of speciation speed and the emergence of new biological capabilities captures something like this as well. This is – grossly simplified – the idea behind the Cambrian evolution.

Systems that develop in this way are tricky to study, and we often miss the sum total pace of development because we are only looking at the pace with which new letters are added to the alphabet, and not the number of new combinations that this makes possible or the number of actual combinations that are produced.

As we add new letters the number of possible words grows fast – combinatorially – and the textual space then expands at the pace we try new texts and words and combinations.

There is no reason to think that the pace of technical development is any different. It is also, very likely, a composite speed where capabilities and combinations unfold the technical space. Think about the Internet as an example: it added a whole bucket of letters to our alphabet and suddenly there are a lot of new things we can do to explore the adjacent possible – and a lot of the progress we have seen since has been fuelled by that, but progress in combinations rather than pure capabilities. And look at AI now – the way that the field is growing is Cambrian in nature, for the moment. 3 See here for example.

There are other interesting aspects of this mental model that also are worth thinking more about.

Given a certain alphabet that vastly exceeds our ability to explore the textual space that can be generated by it – how fast are new words and new texts added? It seems likely that we produce more new words today than before, but not per English language speaker. So what does this tell us about the pace of textual development?

[ Alphabet – dictionary – text ] as a model could be translated to capabilities, tools, usages in technical development – but there are surely other mappings as well, so what do they look like?

In order to be able to compare the pace of development of different systems, we can assume a sort of general model where they are all languages, with alphabets, dictionaries and texts. What can we then say about law and technology? Is law developing slower than technology? It would seem that way, but why is that? Is it because of a lack of letters in the alphabet? A lack of words in the dictionary? Or is there something about the production of text in some systems that is harder? This seems to suggest that there is a role in the mental model for writing / producing text and the costs associated with that.

Can systems, equally, be slowed down by having too many words in a dictionary? Or too many letters?

At the very least it seems prudent to think of technical development as a two dimensional process – with capabilities and combinations as a first approximation – alphabets and words. It is probably even better described by some n-dimensional analogy, but economy of description also matters.

Notes

  • 1
    This notion derives from a talk by Ricardo Hausman on knowledge and economy.
  • 2
    There is an important difference between the texts that could be written and the texts we have actually written.
  • 3
    See here for example.