How to write a methodology?¶
After the abstract, the methodology is a good place to start writing a paper. The methodology clearly explains how you plan to answer your research questions or test your hypothesis. Methodologies and supporting guidelines are available here. In this article I’ll focus on the rigour of your methodology and how to evaluate your findings.
Provide a justification for your choice of methodology: Justify your choice of methodology by 1) providing citations supporting your choice, 2) briefly stating why alternative approaches are insufficient, and 3) providing a logical argument that explains why that methodology is appropriate. Find similar papers as your own and cite them as evidence that your choice of methodology is valid.
Use data to design your research instruments: Research instruments are the tools and instruments you use to collect data. In software engineering instruments include questionnaires, interview questions, scripts, and monitoring applications. Justify your instruments by citing existing papers that use that instrument or create your instruments based on data. For example, investigate Github repositories to gain evidence of a gap which then informs survey questions. This approach is known as mixed methods and is helpful for obtaining triangulation.
Validate your findings as you go: If you create instruments from scratch then include an explanation of how you evaluated your instrument. Common ways to do this include a) pilot studies, b) statistical validations, and c) a rigours design process. As you write up the methodology provide evidence of how iteratively evaluated your findings. For example, manually inspect Github repositories in your corpus to refine your exclusion criteria. After writing up a draft of your methodology ask yourself “Would I trust the findings from this study?”
A methodology is like a cake recipe: When writing up your methodology provide a summary of the key steps especially a) how you created the instruments, b) the number of samples/participants, and 3) how you evaluated the findings. Add a figure at the start of the methodology section showing the core steps. As you write your methodology think of a recipe that you would follow to bake a cake. All of the ingredients and the steps required are included in the recipe. Your methodology should enable another researcher to replicate your study and obtain the same findings as you. Include links to your research instruments hosted online.
Methodology vs Approach¶
Two confusing terms that appear in research papers are methodology and approach. The methodology is the research framework that describes how the research is conducted it defines what you will do. However, the approach describes how your solution works. Approach sections are common in tool papers that describe a software artefact, sometimes the approach section is named after the tool being presented. For example, RAPPT presents the approach section under the heading RAPPT. A tool paper that contains both a methodology and an approach section will describe how the research for the tool was conducted in the methodology and the how the tool functions in the approach section.