This article explores intersecting notions of leisure among middle-aged people with intellectual disabilities in the setting of the Swedish welfare state. The participants are recipients of long-term disability services and have experienced the changing ideological frameworks of the welfare effort, which has recently focused on normalisation, inclusion and participation. Structured activities are arranged by disability services in order to normalise living conditions and provide recreation for disabled people. However, the range of activities is constrained by financial resources, by notions of gender and age and by an institutionalised emphasis on the work ethic – leading to constructions of leisure partly as ‘time beside’ where ‘free time’ activities should not interfere with the duties of the working week. The participants’ limited resources and their lack of a strong voice limit their ability to demand their legal rights and leave many of them with ‘too much time with too little to do’.