Umeå University's logo

umu.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
LRP-based Policy Pruning and Distillation of Reinforcement Learning Agents for Embedded Systems
Nanjing University of Science and Technology, School of Computer Science and Engineering, Nanjing, China.
Umeå University, Faculty of Science and Technology, Department of Applied Physics and Electronics.
Umeå University, Faculty of Science and Technology, Department of Applied Physics and Electronics.ORCID iD: 0000-0003-4228-2774
Nanjing University of Science and Technology, School of Computer Science and Engineering, Nanjing, China.
Show others and affiliations
2022 (English)In: 2022 IEEE 25th International Symposium on Real-Time Distributed Computing, ISORC 2022, Institute of Electrical and Electronics Engineers (IEEE), 2022Conference paper, Published paper (Refereed)
Abstract [en]

Reinforcement Learning (RL) is an effective approach to developing control policies by maximizing the agent's reward. Deep Reinforcement Learning (DRL) uses Deep Neural Networks (DNNs) for function approximation in RL, and has achieved tremendous success in recent years. Large DNNs often incur significant memory size and computational overheads, which greatly impedes their deployment into resource-constrained embedded systems. For deployment of a trained RL agent on embedded systems, it is necessary to compress the Policy Network of the RL agent to improve its memory and computation efficiency. In this paper, we perform model compression of the Policy Network of an RL agent by leveraging the relevance scores computed by Layer-wise Relevance Propagation (LRP), a technique for Explainable AI (XAI), to rank and prune the convolutional filters in the Policy Network, combined with fine-Tuning with Policy Distillation. Performance evaluation based on several Atari games indicates that our proposed approach is effective in reducing model size and inference time of RL agents.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2022.
Keywords [en]
embedded systems, Knowledge Distillation, Policy Distillation, Reinforcement Learning
National Category
Computer Sciences
Identifiers
URN: urn:nbn:se:umu:diva-198614DOI: 10.1109/ISORC52572.2022.9812837ISI: 000863009700006Scopus ID: 2-s2.0-85135377522ISBN: 9781665406277 (electronic)OAI: oai:DiVA.org:umu-198614DiVA, id: diva2:1693944
Conference
25th IEEE International Symposium on Real-Time Distributed Computing, ISORC 2022, Västerås, Sweden, 17-18 May, 2022.
Available from: 2022-09-08 Created: 2022-09-08 Last updated: 2023-09-05Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Luan, SiyuGu, Zonghua

Search in DiVA

By author/editor
Luan, SiyuGu, Zonghua
By organisation
Department of Applied Physics and Electronics
Computer Sciences

Search outside of DiVA

GoogleGoogle Scholar

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 144 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf