Umeå University's logo

Wearable cognitive assistance (WCA) is an important emerging class of applications that are both bandwidth intensive and latency sensitive. This well published genre of edge native applications was originally introduced in 2014 and has since been extensively studied [2, 7]. These applications provide assistance with real world tasks to users who are wearing smart glasses. Here, we focus on an important subclass of WCA applications that provide step-by-step guidance through physical assembly tasks. Applications in this subclass carry out a back and forth process with the user. First, the application gives the user an instruction for completing a step of the task. The user then completes this step, and then the application gives the user the next instruction. The camera on the glasses captures image frames of a user's progress through the task. The application processes these frames using deep neural network (DNN)-based computer vision, to determine when a task step has been completed. If the DNN misclassifies a frame, the user might not be given the next instruction at the correct time, or the user may be given an incorrect instruction.