Driving is a highly social activity with various interactions between the different actors involved in traffic. Autonomous vehicles will pose several challenges to the social fabric of traffic and, as most new technology, will lead to changes in human behavior. This will surely hold for interactions between autonomous vehicles and actors \emph{outside} the vehicle. However and importantly, autonomous vehicles will also alter existing and introduce new social situations and interactions for those \emph{inside} the vehicle, which appears to be an under-researched topic. This paper will focus on the passengers of autonomous vehicles. We will discuss some of the implications and expected changes in the relationship between a car and those inside it, and highlight some of the open issues of being enclosed and driven by a highly complex, largely black-box AI system on wheels.