Umeå universitets logga

umu.sePublikationer
Ändra sökning
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • apa-6th-edition.csl
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
From promise to practice: a study of common pitfalls behind the generalization gap in machine learning
Umeå universitet, Teknisk-naturvetenskapliga fakulteten, Institutionen för datavetenskap. (Machine Learning)
Umeå universitet, Teknisk-naturvetenskapliga fakulteten, Institutionen för datavetenskap. (Machine Learning)
Umeå universitet, Medicinska fakulteten, Institutionen för diagnostik och intervention.ORCID-id: 0000-0002-6321-8117
Umeå universitet, Medicinska fakulteten, Institutionen för diagnostik och intervention.ORCID-id: 0000-0002-8971-9788
Visa övriga samt affilieringar
2025 (Engelska)Ingår i: Transactions on Machine Learning Research, E-ISSN 2835-8856Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

The world of Machine Learning (ML) offers great promise, but often there is a noticeable gap between claims made in research papers and the model's practical performance in real-life applications. This gap can often be attributed to systematic errors and pitfalls that occur during the development phase of ML models. This study aims to systematically identify these errors. For this, we break down the ML process into four main stages: data handling, model design, model evaluation, and reporting. Across these stages, we have identified fourteen common pitfalls based on a comprehensive review of around 60 papers discussing either broad challenges or specific pitfalls within ML pipeline. Moreover, Using the Brain Tumor Segmentation (BraTS) dataset, we perform three experiments to illustrate the impacts of these pitfalls, providing examples of how they can skew results and affect outcomes. In addition, we also perform a review to study the frequency of unclear reporting regarding these pitfalls in ML research. The goal of this review was to assess whether authors have adequately addressed these pitfalls in their reports. For this, we review 126 randomly chosen papers on image segmentation from the ICCV (2013-2021) and MICCAI (2013-2022) conferences from the last ten years. The results from this review show a notable oversight of these issues, with many of the papers lacking clarity on how the pitfalls are handled. This highlights an important gap in current reporting practices within the ML community. The code for the experiments is available at https://github.com/SG-Azar/BraTS-ML-Pitfalls-Experiments.

Ort, förlag, år, upplaga, sidor
Transactions on Machine Learning Research , 2025.
Nationell ämneskategori
Data- och informationsvetenskap
Identifikatorer
URN: urn:nbn:se:umu:diva-233898OAI: oai:DiVA.org:umu-233898DiVA, id: diva2:1926398
Forskningsfinansiär
Barncancerfonden, MT2021-0012Lions Cancerforskningsfond i Norr, LP 22-2319Lions Cancerforskningsfond i Norr, LP 24-2367Tillgänglig från: 2025-01-10 Skapad: 2025-01-10 Senast uppdaterad: 2025-02-04Bibliografiskt granskad

Open Access i DiVA

Fulltext saknas i DiVA

Övriga länkar

Article on Open ReviewJournal website

Person

Ghanbari Azar, SaeidehTronchin, LorenzoSimkó, AttilaNyholm, TufveLöfstedt, Tommy

Sök vidare i DiVA

Av författaren/redaktören
Ghanbari Azar, SaeidehTronchin, LorenzoSimkó, AttilaNyholm, TufveLöfstedt, Tommy
Av organisationen
Institutionen för datavetenskapInstitutionen för diagnostik och intervention
I samma tidskrift
Transactions on Machine Learning Research
Data- och informationsvetenskap

Sök vidare utanför DiVA

GoogleGoogle Scholar

urn-nbn

Altmetricpoäng

urn-nbn
Totalt: 260 träffar
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • apa-6th-edition.csl
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf