Umeå University's logo

The notion of intelligence is relevant to several fields of research, including cognitive and comparative psychology, neuroscience, artificial intelligence, and philosophy, among others. However, there is little agreement within and across these fields on how to characterise and explain intelligence. I put forward a behavioural, operational characterisation of intelligence that can play an integrative role in the sciences of intelligence, as well as preserve the distinctive explanatory value of the notion, setting it apart from the related concepts of cognition and rationality. Finally, I examine a popular hypothesis about the underpinnings of intelligence: the capacity to manipulate internal representations of the environment. I argue that the hypothesis needs refinement, and that so refined, it applies only to some forms of intelligence.

Cognitive representations are typically analysed in terms of content, vehicle and format. While current work on formats appeals to intuitions about external representations, such as words and maps, in this paper we develop a computational view of formats that does not rely on intuitions. In our view, formats are individuated by the computational profiles of vehicles, i.e., the set of constraints that fix the computational transformations vehicles can undergo. The resulting picture is strongly pluralistic, it makes space for a variety of different formats, and is intimately tied to the computational approach to cognition in cognitive science and artificial intelligence.

Fairness is central to the ethical and responsible development and use of AI systems, with a large number of frameworks and formal notions of algorithmic fairness being available. However, many of the fairness solutions proposed revolve around technical considerations and not the needs of and consequences for the most impacted communities. We therefore want to take the focus away from definitions and allow for the inclusion of societal and relational aspects to represent how the effects of AI systems impact and are experienced by individuals and social groups. In this paper, we do this by means of proposing the ACROCPoLis framework to represent allocation processes with a modeling emphasis on fairness aspects. The framework provides a shared vocabulary in which the factors relevant to fairness assessments for different situations and procedures are made explicit, as well as their interrelationships. This enables us to compare analogous situations, to highlight the differences in dissimilar situations, and to capture differing interpretations of the same situation by different stakeholders.

Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Despite their potentially transformative impact, these new capabilities are as yet poorly characterized. In order to inform future research, prepare for disruptive new model capabilities, and ameliorate socially harmful effects, it is vital that we understand the present and near-future capabilities and limitations of language models.

To address this challenge, we introduce the Beyond the Imitation Game benchmark (BIG-bench). BIG-bench currently consists of 204 tasks, contributed by 442 authors across 132 institutions. Task topics are diverse, drawing problems from linguistics, childhood development, math, common-sense reasoning, biology, physics, social bias, software development, and beyond. BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models. We evaluate the behavior of OpenAI's GPT models, Google-internal dense transformer architectures, and Switch-style sparse transformers on BIG-bench, across model sizes spanning millions to hundreds of billions of parameters. In addition, a team of human expert raters performed all tasks in order to provide a strong baseline. Findings include: model performance and calibration both improve with scale, but are poor in absolute terms (and when compared with rater performance); performance is remarkably similar across model classes, though with benefits from sparsity; tasks that improve gradually and predictably commonly involve a large knowledge or memorization component, whereas tasks that exhibit "breakthrough" behavior at a critical scale often involve multiple steps or components, or brittle metrics; social bias typically increases with scale in settings with ambiguous context, but this can be improved with prompting. 

Computational perspectivalism has been recently proposed as an alternative to mainstream accounts of physical computation, and especially to the teleologically-based mechanistic view. It takes physical computation to be partly dependent on explanatory perspectives and eschews appeal to teleology in helping individuate computational systems. I assess several varieties of computational perspectivalism, showing that they either collapse into existing non-perspectival views or end up with unsatisfactory or implausible accounts of physical computation. Computational perspectivalism fails, therefore, to be a compelling alternative to perspective-independent theories of computation in physical systems. I conclude that a teleologically-based, non-perspectival mechanistic account of physical computation is to be preferred.

This is a task accepted in July 2021 as part of Google’s “Beyond the Imitation Game collaborative benchmark for measuring and extrapolating the capabilities of language models”. It is published at https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/conceptual_combinations. Links to the collection of queries are below, followed by the ReadMe file that explains the task, its justification, and its performance with existing AI language models.

Debate on the nature of representation in cognitive systems tends to oscillate between robustly realist views and various anti-realist options. I defend an alternative view, deflationary realism, which sees cognitive representation as an offshoot of the extended application to cognitive systems of an explanatory model whose primary domain is public representation use. This extended application, justified by a common explanatory target, embodies idealisations, partial mismatches between model and reality. By seeing representation as part of an idealised model, deflationary realism avoids the problems with robust realist views, while keeping allegiance to realism.

An influential view in (philosophy of) cognitive science is that computation in cognitive systems is semantic, conceptually depending on representation: to compute is to manipulate representations. I argue that accepting the non-semantic teleomechanistic view of computation lays the ground for a promising alternative strategy, in which computation helps to explain and naturalise representation, rather than the other way around. I show that this computation-based approach to representation presents six decisive advantages over the semantic view. I claim that it can improve the two most influential current theories of representation, teleosemantics and structural representation, by providing them with precious tools to tackle some of their main shortcomings. In addition, the computation-based approach opens up interesting new theoretical paths for the project of naturalising representation, in which teleology plays a role in individuating computations, but not representations.

I analyze a tension at the core of the mechanistic view of computation generated by its joint commitment to the medium independence of computational vehicles and to computational systems possessing teleological functions to compute. While computation is individuated in medium-independent terms, teleology is sensitive to the constitutive physical properties of vehicles. This tension spells trouble for the mechanistic view, suggesting that there can be no teleological functions to compute. I argue that, once considerations about the relevant function-bestowing factors for computational systems are brought to bear, the tension dissolves: physical systems can have the teleological function to compute.