The Swedish public system of elderly care is highly relevant for studying gender relations, specifically when male care workers are more frequently seen within this female-coded field of practice. In this article, qualitative interviews with male and female care workers, elderly women and men, and care managers are analysed to discover how they talk about care work and how gender is expressed, both implicitly and explicitly, in the materialization of care. By illuminating the dynamics of how gender is constructed and negotiated in the intersection of the different actors' perspectives, the paradoxes of gender appeared. The care workers' moral responsibility seemed to undermine equality between male and female care workers, and the elderly clients' gendered expectations and representations created inequality in care work. Furthermore, the gender-neutral assessments made by the care managers came to favour elderly men. Thus, the results suggest the importance of capturing the different perspectives in society's institutions, such as elderly care, in order to understand the complexities of gendered processes.