Umeå University's logo

Contextual language processing concerns how information from the surrounding media environment, such as accompanying texts or interaction signals, is represented and used in automated language processing systems. Incorporating contextual information enables models to derive interpretations that go beyond isolated textual analysis and capture aspects of meaning grounded in real-world situations. Despite substantial research in language modeling, effectively processing contexts remains an appealing challenge, particularly in balancing the expressive power, interpretability, and adaptivity of the model, and its alignment with human understanding.

This thesis investigates contextual language processing from both methodological and applied perspectives, focusing on two related domains: natural language processing and contextual advertising. On the methodological side, the thesis formalizes the grammatical inference of finite-state transducers as a sequential decision-making problem. By integrating structured transducer models with reinforcement learning, it demonstrates how contextual languages can be learned dynamically while retaining interpretability. These learning-based approaches offer a systematic way to adapt language transformations to shifting contexts. Such adaptivity can, for instance, lead to more effective decisions in automated advertising auctions. On the applied side, the thesis examines contextual communication in real-world, user-facing systems through empirical studies in contextual advertising. These studies reveal a gap between the computational level of contextual relevance that is often optimized by automatic metrics, and human-perceived relevance as experienced by end users. The findings show that increased algorithmic precision or model complexity does not necessarily translate into improved user experiences, and that factors such as transparency and trust play a central role in contextual effectiveness.

Through theoretical, empirical, and conceptual analyses, this thesis demonstrates that effective contextual language processing necessitates bridging multidisciplinary perspectives, including algorithmic and perceptual ones. It frames contextual language processing as an interpretable, adaptive, and user-centered interaction process. The insights discussed in this work offer implications not only for algorithmic optimization, but also for modern online advertising and other user-facing applications where contextual understanding, practical constraints, and privacy considerations are critical.