In recent years, involving users in the design of Internet of Things (IoT)-based solutions for everyday settings has received significant attention in Human-Computer Interaction (HCI) studies. This paper aims to contribute to that line of research by proposing an interaction design approach, which extends the scope of user-driven design beyond concept generation to support interaction design activities within the entire task-artifact cycle. We analyze key challenges for creating IoT-based solutions for domestic environments, and argue that dealing with such challenges requires adopting an approach characterized by (a) a focus on users’ activity spaces rather than individual objects and tasks, (b) support and assessment of the actual, and sufficiently extended, use experience of new solutions, (c) augmenting existing everyday objects, to transform them into smart ones, rather than developing entirely new smart objects, and, last but not least, (d) combining IoT prototyping and digital fabrication to ensure physical and esthetical integration of new designs into concrete everyday settings.