umu.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Federated Averaging Deep Q-NetworkA Distributed Deep Reinforcement Learning Algorithm
Umeå University, Faculty of Science and Technology, Department of Computing Science.
2018 (English)Independent thesis Advanced level (degree of Master (Two Years)), 20 credits / 30 HE creditsStudent thesis
Abstract [en]

In the telecom sector, there is a huge amount of rich data generated every day. This trend will increase with the launch of 5G networks. Telco companies are interested in analyzing their data to shape and improve their core businesses. However, there can be a number of limiting factors that prevents them from logging data to central data centers for analysis.  Some examples include data privacy, data transfer, network latency etc.

In this work, we present a distributed Deep Reinforcement Learning (DRL) method called Federated Averaging Deep Q-Network (FADQN), that employs a distributed hierarchical reinforcement learning architecture. It utilizes gradient averaging to decrease communication cost. Privacy concerns are also satisfied by training the agent locally and only sending aggregated information to the centralized server. We introduce two versions of FADQN: synchronous and asynchronous.

Results on the cart-pole environment show 80 times reduction in communication without any significant loss in performance. Additionally, in case of asynchronous approach, we see a great improvement in convergence.

Place, publisher, year, edition, pages
2018. , p. 40
Series
UMNAD ; 1139
National Category
Engineering and Technology
Identifiers
URN: urn:nbn:se:umu:diva-149637OAI: oai:DiVA.org:umu-149637DiVA, id: diva2:1223266
External cooperation
Ericsson
Educational program
Master of Science Programme in Computing Science and Engineering
Supervisors
Examiners
Available from: 2018-06-25 Created: 2018-06-25 Last updated: 2018-06-25Bibliographically approved

Open Access in DiVA

fulltext(972 kB)84 downloads
File information
File name FULLTEXT01.pdfFile size 972 kBChecksum SHA-512
2917491e7f9d00bdd154cf6f2f0b322716a5a36d2f230110cef1fb30bef77678490b96a401ab30ff7b99fc6b0134b810e187fcd87034768c71b6240760c54c1f
Type fulltextMimetype application/pdf

By organisation
Department of Computing Science
Engineering and Technology

Search outside of DiVA

GoogleGoogle Scholar
Total: 84 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 108 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf