This study used multiple methods to investigate the stigma experienced by visually impaired people (VIP) when using a camera-based assistive mobile application. Initial investigations, including semi-structured interviews with VIP and a formalised expert conversation with academics and designers, highlighted the role of design and cooperation in mitigating stigma issues. The semi-structured interviews, formalised expert conversation, and co-design workshop were conducted to explore further the multifaceted nature of stigma and its impact on technology use. The findings reveal the complexity of stigma and its effects on the attitudes and abilities of VIP to use technology. Four key themes were identified as the design considerations, namely i) prioritising user autonomy, ii) fostering trust in communication, iii) designing for daily tasks, and iv) involving users in the design process. The developed design guidelines as a result of the study provide references for creating smart assistive mobile apps.