Umeå University's logo

In constitutional theory, the requirement of necessity is an integral part of a wider proportionality assessment in the limitation of constitutional rights. It fulfils a function of sorting out measures that restrict rights beyond what is required to fulfil the intended purpose. Within data protection, the requirement varies in strictness and interpretation—from ‘ordinary’ necessity to ‘strict necessity’. Recently, the European Court of Justice (ECJ) has introduced what appears to be an even stricter requirement of ‘absolute necessity’ relating to the processing of biometric information under the EU Law Enforcement Directive (LED). In practice, however, the implications of those respective levels of strictness tends to vary, from a strict ‘least restrictive means’ test, to an analysis of whether a measure is necessary for a more effective or a more efficient fulfilment of the intended purpose. In this contribution the principle of necessity as applied by the ECJ is analysed as it pertains to the LED and the Charter, more specifically in the context of implementing AI supported analysis of biometric data. The gradual development of the interpretation of necessity is traced in the data protection case law of the ECJ. The study shows the increased emphasis placed on proportionality over time, highlighting both strengths and potential weaknesses of the requirement in relation to the use of AI supported decision-making in the law enforcement context.

This contribution examines the relationship between automation, discretion, and proportionality. It argues that automation efforts in public administration need to be further discussed and analyzed in relation to requirements of proportionality flowing from both national and European law, as the principle carries important implications for both the implementation of automated systems and the responsibilities of decision-makers within those systems. The different facets of proportionality flowing from, inter alia, constitutional, and human rights law, administrative law, and data protection law are explored, with four distinct stages of proportionality analysis identified: legislative, system, decision, and ex post proportionality. These stages all carry different implications for discretion and the prospects of automation. Through the requirements in these different stages, the authors conclude that proportionality ought to act as another driver of keeping human oversight of automated systems. This human oversight will however, in relation to proportionality, require further contextual awareness and control of correct output proportionality, a role which may be significantly more demanding than a more limited oversight implied by current legal discussions on "humans in the loop".

The Rule of Law is a foundational value of the modern democratic state. It contains within its most influential conceptualizations key legal principles to reduce arbitrariness in the exercise of power, and in doing so promoting legal certainty and individual autonomy, while protecting against despotism. A growing body of research points to the potential for AI to undermine Rule of Law values, especially when implemented in the exercise of public power. This contribution attempts to approach these concerns through four critical dichotomies. These dichotomies are intended to assist in underpinning a theoretical foundation for a critical analysis of AI, assisting in structuring previously diverse explorations into Rule of Law concerns along more fundamental analytical lines.

This contribution situates the rule of law in the context of automated decision-making by using leading rule of law theories to distill some key underlying values of the rule of law which carry particular significance for automated decision-making; dignity, autonomy, and democracy. While disagreement may exist surrounding specific components of the rule of law, general agreement seems to exist surrounding these core values underpinning the concept. By adopting these underlying values of the rule of law, issues with automation, such as the foreseeability of law, access to justice, or non-discrimination, can also be conceptualized as tensions between achieving automation and maintaining respect for human dignity, autonomy of the individual, and a connection to democratic processes. These tensions are finally explained and illustrated through the analogy of normative refraction occurring as automated decision-making requires legal intent to be mediated not only by the language of the law, but through additional refractive layers of code and data. While existing rule of law principles are adapted for managing the refraction added by the language of law, new approaches may be needed to manage the challenges emerging from the additional layers of code and data.

Public and private organizations are increasingly implementing various algorithmic decision-making systems. Through legal and practical incentives, humans will often need to be kept in the loop of such decision-making to maintain human agency and accountability, provide legal safeguards, or perform quality control. Introducing such human oversight results in various forms of semi-automated, or hybrid decision-making – where algorithmic and human agents interact. Building on previous research we illustrate the legal dependencies forming an impetus for hybrid decision-making in the policing, social welfare, and online moderation contexts. We highlight the further need to situate hybrid decision-making in a wider legal environment of data protection, constitutional and administrative legal principles, as well as the need for contextual analysis of such principles. Finally, we outline a research agenda to capture contextual legal dependencies of hybrid decision-making, pointing to the need to go beyond legal doctrinal studies by adopting socio-technical perspectives and empirical studies.

Författarna beskriver utmaningar med att använda AI samt att digitalisera och automatisera beslutsfattande. Rättsstatliga principer som legalitet, proportionalitet och likabehandling riskerar att inte uppfyllas i såväl utformningen av systemen som i tillämpning och uppföljning av beslut som fattas maskinellt. Exempel från svenska myndigheters användning av digitala beslutsstöd visar att brister kan få långtgående konsekvenser.

To efficiently ensure that EU law will have an impact on the national level, the EU is dependent on the loyal cooperation of national administrative authorities, which has been described as a distributed administration. To further this, the CJEU has developed certain responsibilities flowing from EU law ensuring that national administrative authorities will give full effect to EU law within their respective areas of responsibility. This includes a responsibility that in many states would be otherwise reserved for courts – namely, to disapply national legislation when it conflicts with requirements under EU law. In this post, which is based on our recently published article ‘Administrative independence under EU law: Stuck between a rock and Costanzo?’ in European Public Law, we will illustrate how this may create a tension between the hierarchical structure of Member State administrations on the one hand, and the mandates and responsibilities provided to administrative authorities on the other. It may force a subordinate administrative authority to override their own government. Similar to courts, national administrative authorities thus have the invidious position of serving two masters at once, while not enjoying the same structural or legal independence towards the national government.

EU law places a number of requirements on administrative authorities that puts them in potentially invidious positions; while EU law today does not require institutionally independent administrative authorities or provide protection for the independence of authorities beyond the field of data protection, it does require administrative authorities to act independently through the loyal and effective enforcement of EU law. This requirement of acting independently without institutional independence raises certain implications for the role of administrative authorities acting within the hierarchical administrative orders of Member States. Using the case of Sweden – a Member State where administrative authorities enjoy significant constitutionally protected independence in the application of law and decision of cases – this article argues that the effect of EU law obligations of effectiveness and loyalty is a weakening of the hierarchical influence of the government over its own authorities, with a resulting shift of influence towards the legal arena through the provision of politically expedient interpretations of EU law. The invidious position of administrative authorities within the scope of EU law is likely to make them vulnerable to such influence, which may ultimately interfere with the effective administration of EU law.