Umeå University's logo

umu.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Metadata assisted finetuning with largepre-trained language models forabstractive text summarization: Multi-task finetuning with abstractive text summarization and categoryclassification
Umeå University, Faculty of Science and Technology, Department of Computing Science.
2023 (English)Independent thesis Advanced level (degree of Master (Two Years)), 20 credits / 30 HE creditsStudent thesis
Abstract [en]

Text summarization is time-consuming for humans to complete but is still required in many areas. Recent progress in machine learning research, especially in the natural language domain, has produced promising results. The introduction of the Transformer and the increased popularity of pre-trained language models have driven this improvement and led to a human-comparable performance in the automatic text summarization domain. However, the finetuning of pre-trained language models with multitask learning to increase the model’s performance is still new. This potential performance increase raises the question. How can Multitask finetuning affect the summarization performance of large pre-trained language models? To answer this, we finetune two pre-trained models in 3 variants each, one model as a benchmark and two models incorporating multitask finetuning with category classification as a complementary task to the abstractive text summarization. The results indicate decreased performance with multitask finetuning. However, extended finetuning of the models shows a more negligiblem difference between standard and multitask approaches, opening up for further hyperparameter tuning and a potential benefit from the multitask approach.

Place, publisher, year, edition, pages
2023. , p. 28
Series
UMNAD ; 1426
National Category
Computer and Information Sciences
Identifiers
URN: urn:nbn:se:umu:diva-211380OAI: oai:DiVA.org:umu-211380DiVA, id: diva2:1780144
External cooperation
Websystem
Educational program
Master of Science Programme in Computing Science and Engineering
Supervisors
Examiners
Available from: 2023-07-06 Created: 2023-07-05 Last updated: 2023-07-06Bibliographically approved

Open Access in DiVA

No full text in DiVA

By organisation
Department of Computing Science
Computer and Information Sciences

Search outside of DiVA

GoogleGoogle Scholar

urn-nbn

Altmetric score

urn-nbn
Total: 249 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf