The impact of surface features on choice of (in)secure answers by Stackoverflow readers

van der Linden, Dirk, Williams, Emma, Hallett, Joseph and Rashid, Awais (2020) The impact of surface features on choice of (in)secure answers by Stackoverflow readers. IEEE Transactions on Software Engineering. ISSN 0098-5589

09072521.pdf - Published Version
Available under License Creative Commons Attribution 4.0.

Download (711kB) | Preview
Official URL:


Existing research has shown that developers will use StackOverflow to answer programming questions: but what draws them to one particular answer over any other? The choice of answer they select can mean the difference between a secure application and insecure one, as the quality of supposedly secure answers can vary. Prior work has studied people posting on Stack Overflow—a two-way communication between the original poster and the Stack Overflow community. Instead, we study the situation of one-way communication, where people only read a Stack Overflow thread without being actively involved in it, sometimes long after a thread has closed. We report on a mixed-method study including a controlled between-groups experiment and qualitative analysis of participants' rationale (N=1188), investigating whether explanation detail, answer scoring, accepted answer marks, as well as the security of the code snippet itself affect the answers participants accept. Our findings indicate that explanation detail affects what answers participants reading a thread select (p<0.01), while answer score and acceptance do not (p>0.05)—the inverse of what research has shown for those asking and answering questions. The qualitative analysis of participants' rationale further explains how several cognitive biases underpin these findings. Correspondence bias, in particular, plays an important role in instilling readers with a false sense of confidence in an answer through the way it looks, regardless of whether it works, is secure, or if the community agrees with it. As a result, we argue that StackOverflow's use as a knowledge base by people not actively involved in threads'when there is only one-way-communication—may inadvertently contribute to the spread of insecure code, as the community's voting mechanisms hold little power to deter them from answers.

Item Type: Article
Subjects: G400 Computer Science
Department: Faculties > Engineering and Environment > Computer and Information Sciences
Depositing User: Ellen Cole
Date Deposited: 24 Sep 2020 19:29
Last Modified: 31 Jul 2021 13:36

Actions (login required)

View Item View Item


Downloads per month over past year

View more statistics