Building from a survey specifically developed to address the rising concerns of highly autonomous virtual assistants; this paper presents a multi-level taxonomy of accountability levels specifically adapted to virtual assistants in the context of Human-Human-Interaction (HHI). Based on research findings, the authors recommend the integration of the variable of accountability as capital in the development of future applications around highly automated systems. This element inserts a sense of balance in terms of integrity between users and developers enhancing trust in the interactive process. Ongoing work is being dedicated to further understand to which extent different contexts affect accountability in virtual assistants.