Cryptocurrencies are gaining prominence among individuals and companies alike, resulting in the growing adoption of so-called cryptocurrency wallet applications, as these simplify transactions. These wallets are available in a myriad of different forms and specifications. All of them are susceptible to various ways the attacker can exploit the vulnerabilities and steal money from victims. Cryptocurrency wallets create a unique field as they combine features of password managers, banking applications, and the need to keep their users and their transactions anonymous. We collect the findings from previous literature to provide an overview of the different attack surfaces, possible countermeasures, and further research. Existing literature focused on one of the features mentioned before, while we considered all of them. Our systematic study shows that there is a considerable variety of attack vectors, which we have divided into six subcategories, (i) Memory and Storage, (ii) Operating Systems, (iii) Software Layer, (iv) Network Layer, (v) Blockchain Protocol, and (vi) Others. We have found a large gap between the possible countermeasures and their actual adoption. Therefore, we provide a list of possible directions for future research to tackle this gap.