Umeå University's logo

In the realm of data-to-text generation tasks, the use of large language models (LLMs) has become common practice, yielding fluent and coherent outputs. Existing literature highlights that the quality of in-context examples significantly influences the empirical performance of these models, making the efficient selection of high-quality examples crucial. We hypothesize that the quality of these examples is primarily determined by two properties: their similarity to the input data and their diversity from one another. Based on this insight, we introduce a novel approach, Double Clustering-based In-Context Example Selection, specifically designed for data-to-text generation tasks. Our method involves two distinct clustering stages. The first stage aims to maximize the similarity between the in-context examples and the input data. The second stage ensures diversity among the selected in-context examples. Additionally, we have developed a batched generation method to enhance the token usage efficiency of LLMs. Experimental results demonstrate that, compared to traditional methods of selecting in-context learning samples, our approach significantly improves both time efficiency and token utilization while maintaining accuracy.

Current clinical approaches fail to fully utilize unstructured data in managing prostate cancer (PCa) and kidney cancer (KC), leading to inefficiencies in patient care and increased costs. Effective diagnostics and treatments depend on integrating multimodal data, yet progress is hampered by limited data accessibility and a lack of collaborative validation between clinicians and computer scientists. To address these challenges, the EU-funded COMFORT project aims to develop commercially viable, data-driven multimodal decision support systems. These systems will improve clinical prognostication, patient stratification, and personalized treatment while also assessing the trust that healthcare professionals and patients place in AI-driven tools.

Few-shot Named Entity Recognition (NER) methods have shown initial effectiveness in flat NER tasks. However, these methods often prioritize optimizing models with a small annotated support set, neglecting the high-quality data within the unlabeled query set. Furthermore, existing few-shot NER models struggle with nested entity challenges due to linguistic or structural complexities. In this study, we introduce Retrieving high-quality pseudo-label Tuning, RiTNER, a framework designed to address few-shot nested named entity recognition tasks by leveraging high-quality data from the query set. RiTNER comprises two main components: (1) contrastive span classification, which clusters entities into corresponding prototypes and generates high-quality pseudo-labels from the unlabeled data, and (2) masked pseudo-data tuning, which generates a masked pseudo dataset and then uses it to optimize the model and enhance span classification. We train RiTNER on an English dataset and evaluate it on both English nested datasets and cross-lingual nested datasets. The results show that RiTNER outperforms the top-performing baseline models by 1.67%, and 3.04% in the English 5-shot task, as well as the cross-lingual 5-shot tasks, respectively.

The drug recommendation task aims to provide effective and safe prescription decision support for clinical treatment based on patients' past Electronic Health Records (EHR). However, the prevalent phenomenon of missing views in multi-source heterogeneous EHR data may cause performance degradation. This is due to the lack of sufficient information and increased learning difficulties, which limit the practical effectiveness of drug recommendation models in medical applications. In this paper, we emphasize the problems of incompleteness in practical drug recommendation and propose the Incomplete Multi-View Drug Recommendation model via Multi-Level Representation Learning and Curriculum Learning named IMDR. In particular, IMDR employs a Multi-Level Representation Learning architecture equipped with a Medical Code-Level Drug Knowledge Infusion Module and a Visit-Level Cross-View Information Module for patient representation learning to overcome the information loss caused by incomplete data. And then, a Gaussian-guided curriculum learning strategy is proposed to assist the learning process of IMDR with a novel difficulty measure to achieve effective progressive learning under missing medical views. Systematic evaluation on two large-scale real-world medical datasets, MIMIC-III and MIMIC-IV, demonstrates that IMDR reduces the Drug-Drug Interaction (DDI) rate by 2.97% compared to existing state-of-the-art drug recommendation baselines, while achieving significant improvements of 3.29% and 1.97% in Jaccard similarity scores and F1 score, respectively. Furthermore, compared to advanced incomplete multi-view learning (IML) models, IMDR's advantages in Jaccard similarity scores and F1 score further expand to 4.03% and 2.41%.

Nested entities are prone to obtain similar representations in pre-trained language models, posing challenges for Named Entity Recognition (NER), especially in the few-shot setting where prototype shifts often occur due to distribution differences between the support and query sets. In this paper, we regard entity representation as the combination of prototype and non-prototype representations. With a hypothesis that using the prototype representation specifically can help mitigate potential prototype shifts, we propose a Prototype-Attention mechanism in the Contrastive Learning framework (PACL) for the few-shot nested NER. PACL first generates prototype-enhanced span representations to mitigate the prototype shift by applying a prototype attention mechanism. It then adopts a novel prototype-span contrastive loss to reduce prototype differences further and overcome the O-type's non-unique prototype limitation by comparing prototype-enhanced span representations with prototypes and original semantic representations. Our experiments show that the PACL outperformed baseline models on the 1-shot and 5-shot tasks in terms of F1 score. Further analyses indicate that our Prototype-Attention mechanism is a simple but effective method and exhibits good generalizability.

Large language models (LLMs) encounter challenges when addressing few-shot nested named entity recognition (NER) tasks. Traditional LLM-based approaches typically either prompt the model to generate entity words or types in sentences based on entity categories or word spans, or directly extract all entities of specific types present in the sentences. These methods often suffer from issues such as low query efficiency or suboptimal accuracy. This paper introduces an innovative framework, SynNER, which synergizes small and large language models to address these limitations. Initially, a small language model identifies low-confidence word spans, which are then refined and refined by a large language model. To simultaneously ensure recognition accuracy and improve the query efficiency of the LLM, we propose a Batch-Prompt strategy and an Entity Indexing method. These techniques enable the LLM to process multiple test instances simultaneously while maintaining precise correction results. Experimental results demonstrate that our method achieves significant performance gains on benchmark datasets, offering a cost-effective solution for few-shot nested NER tasks.

Examining boiler failure causes is crucial for thermal power plant safety and profitability. However, traditional approaches are complex and expensive, lacking precise operational insights. Although data-driven approaches hold substantial potential in addressing these challenges, there is a gap in systematic approaches for investigating failure root causes with unlabeled data. Therefore, we proffered a novel framework rooted in data mining methodologies to probe the accountable operational variables for boiler failures. The primary objective was to furnish precise guidance for future operations to proactively prevent similar failures. The framework was centered on two data mining approaches, Principal Component Analysis (PCA) + K-means and Deep Embedded Clustering (DEC), with PCA + K-means serving as the baseline against which the performance of DEC was evaluated. To demonstrate the framework’s specifics, a case study was performed using datasets obtained from a waste-to-energy plant in Sweden. The results showed the following: (1) The clustering outcomes of DEC consistently surpass those of PCA + K-means across nearly every dimension. (2) The operational temperature variables T-BSH3rm, T-BSH2l, T-BSH3r, T-BSH1l, T-SbSH3, and T-BSH1r emerged as the most significant contributors to the failures. It is advisable to maintain the operational levels of T-BSH3rm, T-BSH2l, T-BSH3r, T-BSH1l, T-SbSH3, and T-BSH1r around 527 °C, 432 °C, 482 °C, 338 °C, 313 °C, and 343 °C respectively. Moreover, it is crucial to prevent these values from reaching or exceeding 594 °C, 471 °C, 537 °C, 355 °C, 340 °C, and 359 °C for prolonged durations. The findings offer the opportunity to improve future operational conditions, thereby extending the overall service life of the boiler. Consequently, operators can address faulty tubes during scheduled annual maintenance without encountering failures and disrupting production.

While Named Entity Recognition (NER) is a widely studied task, making inferences of entities with only a few labeled data has been challenging, especially for entities with nested structures commonly existing in NER datasets. Unlike flat entities, entities and their nested entities are more likely to have similar semantic feature representations, drastically increasing difficulties in classifying different entity categories. This paper posits that the few-shot nested NER task warrants its own dedicated attention and proposes a Global-Biaffine Positive-Enhanced (GBPE) framework for this new task. Within the GBPE framework, we first develop the new Global-Biaffine span representation to capture the span global dependency information for each entity span to distinguish nested entities. We then formulate a unique positive-enhanced contrastive loss function to enhance the utility of specific positive samples in contrastive learning for larger margins. Lastly, by using these enlarged margins, we obtain better margin constraints and incorporate them into the nearest neighbor inference to predict the unlabeled entities. Extensive experiments on three nested NER datasets in English, German, and Russian show that GBPE outperforms baseline models on the 1-shot and 5-shot tasks in terms of F1 score.

Few-shot Named Entity Recognition (NER) aims to identify named entities using very little annotated data. Recently, prompt-based few-shot NER methods have demonstrated significant effectiveness. However, most existing methods employ multi-round prompts, which significantly increase time and computational costs. Furthermore, current single-round prompt methods are mainly designed for flat NER tasks and are not effective in handling nested NER tasks. Additionally, these methods do not to fully utilize the semantic information of entity labels through prompts. To address these challenges, we propose a novel Label-Prompt-based few-shot nested NER method named LPNER, which not only handles nested NER tasks but also efficiently extracts semantic information of entities through label prompts, thereby achieving more efficient and accurate NER. LPNER first designs a specialized prompt based on a span strategy to enhance label semantics and effectively combines multiple span representations using special mark to obtain enhanced span representations integrated with label semantics. Then, entity prototypes are constructed through prototype network for classifying candidate entity spans. We conducted extensive experiments on five nested datasets: ACE04, ACE05, GENIA, GermEval, and NEREL. In 1-shot and 5-shot tasks, LPNER’s F1 scores mostly outperform baseline models.

Accurate forecasting and analysis of emerging pandemics play a crucial role in effective public health management and decision-making. Traditional approaches primarily rely on epidemiological data, overlooking other valuable sources of information that could act as sensors or indicators of pandemic patterns. In this paper, we propose a novel framework, MGLEP, that integrates temporal graph neural networks and multi-modal data for learning and forecasting. We incorporate big data sources, including social media content, by utilizing specific pre-trained language models and discovering the underlying graph structure among users. This integration provides rich indicators of pandemic dynamics through learning with temporal graph neural networks. Extensive experiments demonstrate the effectiveness of our framework in pandemic forecasting and analysis, outperforming baseline methods across different areas, pandemic situations, and prediction horizons. The fusion of temporal graph learning and multi-modal data enables a comprehensive understanding of the pandemic landscape with less time lag, cheap cost, and more potential information indicators.