In the area of socio-technical aspects of security, we consider the question: “How well is our science doing in creating reliable and practically relevant results?” “What could possibly go wrong?” “What difference will that make for practitioners?” Based on a systematic literature review on user studies in security and privacy, we make observations on reliability of results in our own field of socio-technical aspects of security. As we dive into topics that have not received much attention in our own field yet, we consider problems that other fields have faced in their endeavour to deliver rigorous and effective science. We will also discuss the solutions with which the sciences responded to those problems.

Dr. Thomas Gross is a Reader (eq. Associate Professor) in System Security at the School of Computing of Newcastle University. He is the director of the Newcastle Academic Centre of Excellence in Cyber Security Research (ACE-CSR). Thomas is the Principal Investigator of a European Research Council (ERC) grant on Confidentiality-Preserving Security Assurance (CASCAde). He is a member of the UK Research Institute in the Science of Cyber Security (RISCS), in which he leads the approach to scientific methods on the Institute’s scientific board. Thomas’ research interests include applied cryptography as well as evidence-based investigations of security, especially on human dimensions of security. Before joining Newcastle, Thomas was a research staff member at IBM Research - Zurich. At IBM Research, he was responsible for research in identity federation and privacy-enhancing technologies as well as cloud security assurance. For instance, Thomas led IBM Research’s efforts to establish anonymous credential systems on electronic ID cards. Thomas is a member of the EPSRC Review College. Thomas has published more than 60 peer-reviewed papers in security and privacy, including top venues such as CCS, CRYPTO or PETS. He has filed more than 20 patent applications, 12 of which have been granted to date.