How to do and report survey studies robustly: A helpful mnemonic <scp>SURVEY</scp>

Abstract

Surveys are one of the most used research designs. The results of survey studies can add context around a topic, quantify the extent of an issue or suggest future research questions1, 2 However, if done poorly, with little rigour, they offer little insight, can be misleading and simply burden research participants unnecessarily.2 On top of that, poorly performed survey studies provide less rigorous and often biased results, the conclusions of which can be questioned. These studies can be considered research waste3 and should not be supported. Surveys are popular amongst health care professionals. In this journal alone, many submitted research papers use a surveys design. Unfortunately, many of these submitted manuscripts face a desk rejection or are rejected by the reviewers with the overall argument that the design, methods and reporting of the survey study was inadequate. It is important to be clear in the wording. A survey is the research design of the study, and a questionnaire is the instrument used to conduct the survey study and to collect data. Survey studies are most commonly cross-sectional, conducted at one point in time, but can be done longitudinally, where surveys are administered over a period of time.1, 4 In this editorial, we have developed a simple mnemonic—SURVEY—to guide and remind clinicians and researchers about the key issues to consider when undertaking and reporting surveys studies. The instrument is a questionnaire and should always be included as an electronic supplementary file in any submitted manuscript. This is important as it allows the reviewers to see and assess the questionnaire and relate this back to the overall reporting of the methods and results of the study. If the manuscript is accepted and published, it allows the readers to use your questionnaire and replicate the study in a similar or different context (obviously after seeking permission from the corresponding author). The proposed analysis of your survey study must be clearly reported, along with the type of data. If inferential statistics are being used, the rationale for this, the statistical test used, and the level of significance need to be reported. Inferential data analysis may be appropriate to compare different groups or different levels of education or experience, and you must justify in the methods whether parametric or non-parametric tests were used. However, in some survey studies, this is not appropriate or feasible. In this case, descriptive statistics (percentages) or distribution, such as mean and SD or median and inter-quartile range, must be reported anyway.5 It is essential to report the response rates in a survey study whenever possible, both the exact figures and the percentage. The exceptions might be if the denominator is unknown; for example, the questionnaire was distributed via social media. If response rates are known, the target response rate should ideally be >70% of your sample.6 Anything less risks introducing a large bias and may indicate a poor questionnaire or inadequate reminders. Anything less than 50% will be almost impossible to draw conclusions upon. Unless you are using a questionnaire that has been validated in your specific population, you need to establish, at the very minimum, face and content validity of the instrument.7 This is achieved by pilot testing the questionnaire in a small sample of your intended (or very similar) survey population to ensure the questions are clear and are asking what you think you are asking.8 This process needs to be reported in the methods section of your manuscript for clarity, such as how many people it was piloted on, and whether were any changes made in the questionnaire before it was used in the main study. If a translation of the questionnaire from or to another language has been done, the accepted process of translation, cultural adaptation, and validation must be reported including the steps of both forward and back translation and testing.8 While developing a new questionnaire after reviewing the literature on a topic, you may consider sending the draft survey to an expert panel for their assessment of the readability, content and feasibility.9 Ideally, these would be a small group of experts in the field, not necessarily at your local institution, and must include experts who belong to the intended respondents of the questionnaire. This process should be undertaken before piloting and can further add to the face validity of the questionnaire. The presentation of your results should be clear and reported only in the results section of a manuscript. Tables and graphs should be used where possible to save words in the text and present the results clearly. There are many limitations of the survey method, including the self-report nature of the method, which may not tell us what actually happens in practice. This is important to note, because your conclusions must be realistic and reflect these limitations, especially if your response rates are <70%, limited claims can be made. As a final recommendation, the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) Network has published reporting guidelines of all study designs (https://www.equator-network.org/). Specifically for survey studies, there are several guidelines to consider. For example, if you are reporting an online survey, you should write your manuscript according to the “Improving the quality of Web surveys: The Checklist for Reporting Results of Internet E-Surveys (CHERRIES)” guideline.10 If your survey study used a traditional paper questionnaire, you can use the recently published guideline: A Consensus-Based Checklist for Reporting of Survey Studies (CROSS).11 In summary, surveys can be a useful research method, but they need to be undertaken with the same rigour that is applied to other clinical research studies. It can be said that surveys are the most widely abused form of research because of their perceived ease of undertaking. It is important to recognize the significant limitations of survey designs and to acknowledge these in the limitations and ensure that your conclusions are justified and not “over claimed.” Finally, we hope this proposed new mnemonic “SURVEY” will help guide potential authors to maximize their chances that their survey study will be accepted for publication.

Topics & Concepts