Limitations to full social participation in physical activity—both during leisure time and in physical education—persist among young individuals with cerebral palsy. This study explores the experiences of physical education and physical leisure activities among youth with cerebral palsy living in Sweden. It examines how these youths navigate contextual barriers and facilitators to participation, focusing on norms of function and gender. Individual interviews with 13 youth, aged 15–18 years, were analysed using Qualitative Content Analysis resulting in a main theme: Being in a continuum between exclusion and empowerment, with three categories: 1) Excluded and denied of support; 2) Resisting prejudice and discrimination; and 3) Empowered, with appropriate support. Most participants had experienced not being given equal opportunity in particularly in physical education but also in leisure activities and had encountered limiting norms and preconceptions based on gender and function. The findings indicate that norm-consciousness, respect and involvement in decision-making are key for participation and empowerment.