Pilot studies

Valve

Before rushing off to take the final leap into the swimming-pool of the main data gathering exercise, it is usually advisable to conduct a quick reality-check. In some form or other, a short pilot study, which samples just a small part of each data gathering method, is a useful activity at this stage. Depending on the diversity of the selected data-gathering methods to be used in the main study, it could mean asking 3 or 4 people to complete a questionnaire, or trying out the interview questions on a few “volunteers”, or perhaps conducting a trial run of a bench experiment, just to make sure that things progress in practice as smoothly as they have been envisioned in theory. Either way, a pilot study can do several things. In the first place, it allows the supervisor to observe just how much thought, care, and background research has been already conducted in the formulation of the research methodology of this study. There may be some opportunity for improving the methods, or there might simply be a reassurance that things have been well-planned… so far. Feedback at this design stage may avoid making elementary mistakes, or designing a method which will lead to incorrect or misleading results.

For the research student, the pilot study can have multiple benefits. The reassurance of the supervisor is useful, but the feedback from the pilot participants can be even more critical. This is the time when slightly ambiguous questions can be reworded, and research methods can then be tweaked to make sure that they do what it is hoped that they should do. If a participant reports that the wording of a question is difficult to understand, or that there is no relevant category of response, this suggests that other people in the larger study will encounter the same difficulties. The error created will become multiplied when the full study progresses, and may become significant. The fault in the misunderstanding lies with the researcher, not with the participants being questioned. It is up to the researcher to construct questions which are unbiased, not leading towards a particular response, and are clearly understandable by participants in the sample population. Similarly, with experimental design, if the experiment has a fault in its design, it is much better to find out at this stage through a short pilot study, than to run the experiments several hundred times before finding out that there is a problem.

Writing up a description of the pilot study is an integral part of the methodology chapter in the dissertation. If there were changes made to improve the design of the main research survey, (and even if not) then this is a good place to note the changes, justify them, and demonstrate that the researcher has not simply woken up one morning and plucked a research design idea from thin air. Demonstrate that thought and care has been invested in this. Even the experience of codifying and analysing a few results from the pilot study might give the researcher (and the supervisor) a good sense of the ease (or difficulty) which the final main data-set will present, and allow for a simplification or clarification as appropriate. It is a huge mistake to seek a “short-cut” by avoiding pilot studies!

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s