Umeå University's logo

umu.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
From precision to perception: user surveys in the evaluation of keyword extraction algorithms
Umeå University, Faculty of Science and Technology, Department of Computing Science.ORCID iD: 0009-0004-0580-6270
Malmö University, Malmö, Sweden.
Umeå University, Faculty of Science and Technology, Department of Computing Science.ORCID iD: 0000-0001-8503-0118
2025 (English)In: HILDA '25: Proceedings of the Workshop on Human-In-the-Loop Data Analytics, ACM Digital Library, 2025, article id 2Conference paper, Published paper (Refereed)
Abstract [en]

Stricter regulations on personal data are causing a shift towards contextual advertising, where keywords are used to predict the topical congruence between ads and their surrounding media contexts - an alignment shown to enhance advertising effectiveness. Recent advances in AI, particularly large language models, have improved keyword extraction capabilities but also introduced concerns about computational cost. This study conducts a comparative, survey-based evaluation experiment of three prominent keyword extraction approaches, emphasising user-perceived accuracy and efficiency. Based on responses from 552 participants, the embedding-based approach emerges as the preferred method. The findings underscore the importance of human-in-the-loop evaluation in real-world settings.

Place, publisher, year, edition, pages
ACM Digital Library, 2025. article id 2
Keywords [en]
Data analysis, Human evaluation, Keyword extraction, Language models, Statistical methods, Word embeddings
National Category
Computer Sciences
Identifiers
URN: urn:nbn:se:umu:diva-244865DOI: 10.1145/3736733.3736741ISI: 001546434700002Scopus ID: 2-s2.0-105012238681ISBN: 9798400719592 (electronic)OAI: oai:DiVA.org:umu-244865DiVA, id: diva2:2002999
Conference
HILDA '25: Workshop on Human-In-the-Loop Data Analytics, Berlin, Germany, June 22-27, 2025
Available from: 2025-10-02 Created: 2025-10-02 Last updated: 2025-10-02Bibliographically approved

Open Access in DiVA

fulltext(1603 kB)75 downloads
File information
File name FULLTEXT01.pdfFile size 1603 kBChecksum SHA-512
a1f62a67635073f637c9198ce79aa29389d265f018e63bc8689590e6f23aee7a4cf3ff57f8243f87a4bc7ee0765e380d561983e04da3cf1de5b42e5b6dcfde76
Type fulltextMimetype application/pdf

Other links

Publisher's full textScopus

Authority records

Cai, JingwenBjörklund, Johanna

Search in DiVA

By author/editor
Cai, JingwenBjörklund, Johanna
By organisation
Department of Computing Science
Computer Sciences

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 550 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf