The things other people say…

Some light-hearted relief over the summer months, and still on the topic of PhD supervision, here are a few blogs that are worth dipping into:

Get a Life, PhD (just what it says!)

The Thesis Whisperer (source of lots of good advice)

Good, practical tips (from someone who has been through it)

And, last but not least, just to illustrate that there is always someone worse off than yourself, take a look at these comments which academics have had from reviewers of their article submissions. Some of them are very, very funny…

Pilot studies


Before rushing off to take the final leap into the swimming-pool of the main data gathering exercise, it is usually advisable to conduct a quick reality-check. In some form or other, a short pilot study, which samples just a small part of each data gathering method, is a useful activity at this stage. Depending on the diversity of the selected data-gathering methods to be used in the main study, it could mean asking 3 or 4 people to complete a questionnaire, or trying out the interview questions on a few “volunteers”, or perhaps conducting a trial run of a bench experiment, just to make sure that things progress in practice as smoothly as they have been envisioned in theory. Either way, a pilot study can do several things. In the first place, it allows the supervisor to observe just how much thought, care, and background research has been already conducted in the formulation of the research methodology of this study. There may be some opportunity for improving the methods, or there might simply be a reassurance that things have been well-planned… so far. Feedback at this design stage may avoid making elementary mistakes, or designing a method which will lead to incorrect or misleading results.

For the research student, the pilot study can have multiple benefits. The reassurance of the supervisor is useful, but the feedback from the pilot participants can be even more critical. This is the time when slightly ambiguous questions can be reworded, and research methods can then be tweaked to make sure that they do what it is hoped that they should do. If a participant reports that the wording of a question is difficult to understand, or that there is no relevant category of response, this suggests that other people in the larger study will encounter the same difficulties. The error created will become multiplied when the full study progresses, and may become significant. The fault in the misunderstanding lies with the researcher, not with the participants being questioned. It is up to the researcher to construct questions which are unbiased, not leading towards a particular response, and are clearly understandable by participants in the sample population. Similarly, with experimental design, if the experiment has a fault in its design, it is much better to find out at this stage through a short pilot study, than to run the experiments several hundred times before finding out that there is a problem.

Writing up a description of the pilot study is an integral part of the methodology chapter in the dissertation. If there were changes made to improve the design of the main research survey, (and even if not) then this is a good place to note the changes, justify them, and demonstrate that the researcher has not simply woken up one morning and plucked a research design idea from thin air. Demonstrate that thought and care has been invested in this. Even the experience of codifying and analysing a few results from the pilot study might give the researcher (and the supervisor) a good sense of the ease (or difficulty) which the final main data-set will present, and allow for a simplification or clarification as appropriate. It is a huge mistake to seek a “short-cut” by avoiding pilot studies!

Getting research ethics approval


Teaching research ethics is almost impossible. Teaching someone about ethics is a different matter, but unless a person actually understands why ethical standards are essential, then everything else is fruitless. It is relatively straightforward to present examples of good ethical practice (and what happens when this practice is ignored) but this simply underpins the implementation of the ethical standards, not the need for them. Fortunately, there are lots of detailed guidelines and professional codes describing the expectations of ethical behaviour, many of them readily available on the web. I say “lots” because the ethical standards vary widely in content and detail, dependent on the subject discipline, the research methods employed, the level of study, and several other factors. This might sound vague, but think about it. There will be a different level of scrutiny required if a researcher seeks access to the confidential medical files of patient, rather than simply asking patients to respond to a few verbal questions. There will be different standards again if the researcher plans to work with animals, or children, or vulnerable adults with diminished responsibility. There is also an ethical code for internet-mediated research, although this is new, variable, and highly contextual, so it is an evolving set of guidelines. Despite these differences, the purpose of research ethics is the same in each case – namely to prevent causing harm to the participants, to preserve their dignity (for example their right to anonymity) and to enable them to withdraw from the study without any undue pressure or penalty.

For these reasons, there is a crucial stage between deciding on what research methods are to be adopted for a study, and the commencement of data collection. This crucial stage is where the researcher submits the details of the design, methodology, and any issues relating to the collection and storage of data, for approval by the university ethics committee. Only after ethical clearance has been approved can the student begin to collect data. Failure to obtain approval before data is collected may result in the university deciding that this data is not admissible for inclusion in the study. If there have been any severe breaches of ethical responsibility, the study may be terminated or the student de-registered. For this reason, the ethical approval of a student research project is a gate-keeper stage of every study.

Fortunately, most research projects have fairly straightforward ethical requirements which are easily satisfied in full. A lot of the ethical safeguards might be regarded as “simply common sense” (and so they are) but you might be surprised by the number of times people say “Oh, there are no ethical issues with my research!” This is almost certainly wrong. Even the issue of whether the researcher with half-formed ideas should be “wasting” the time of an interviewee who almost certainly has something better, perhaps crucial, to do, is an ethical issue. For these reasons, seeking ethical approval for research should be a serious matter, but not something to fret unduly about, if the researcher has properly thought through the research design. Once the ethical approval has been obtained, the researcher is able to jump out of the starting blocks to engage with data collection, and this is where the real fun part starts.

What methods will help to answer the research question?


This is where it gets hard, not simply because the research student is venturing out into the unknown, but also because selecting the methods through which the research will be conducted will differ hugely between cultures, between disciplines, and between subjects within disciplines. There is no one-size-fits-all template which will allow a pick-and-choose approach to selecting the most appropriate methods. In one sense, this is an easy step, because it will probably be pretty obvious from the outset what methods will be needed in order to answer the research question(s). Almost all academic research methods will involve reading, either to follow-up on what has already been said about the topic or to put it into a wider context. After that, the methods might include interviews, experiments, observations, questionnaires, focus groups, and a host of other activities which will change in emphasis from discipline to discipline. Getting the “correct” mixture of these methods is what will determine the methodology, that is, the system of methods for further research.

Here is where high technology can come in. I say “high” technology because even using a pen-and-paper or driving a car to conduct an interview is using technology, but of course we generally mean computer-based technology. In educational circles you will frequently hear the assertion that “the technology should never lead!”. This is certainly true, to an extent, but not entirely. For instance, if there are two (or more) ways to record research data, and one way entails using a high-technology solution which makes it easier, more flexible and/or more secure, then surely most sensible people would vote for the use of the technology. Examples might include, the use of RefME to compile the dissertation reference list and store it on the cloud; using Mendeley to store the articles online; the use of SurveyMonkey to conduct a questionnaire online rather than face-to-face, giving time-flexibility, wider geographic coverage, and the ability to utilise automatic data analysis and presentation tools; the use of a free voice-recorder smartphone app to record interviews… The list could go on and on.

A crucial factor in all of this is to consider carefully – right at the start – how these methods will allow you to analyse and hopefully make sense of the data which will be gathered. It makes little sense jumping off a high-point without knowing, even approximately, where you might land. Similarly, it makes little sense to gather mountains of data without any ideas how to begin to make sense of it. The supervisor should be able to give some clear directions, but ultimately each situation, each carefully worded question, is slightly different, and will have different constraints on time, resources, and abilities, so the student will need to be fully comfortable with the methodology before even starting the research. Prior studies in a similar area can help to provide some direction, but the precise mixture needs to be decided for each individual research project.

Writing the methodology


When starting a PhD, there is often a great mystique surrounding the selection (and writing-up) of the proposed methodology. It is important to remember that the term “methodology” means more than simply describing the methods that are intended to be used for the collection of research data, it is the constructed system of methods proposed, and how they interact. Importantly, in order to understand the data which might be generated by the research, it is critical to first understand the rules which govern the various research methods selected, their strengths and their limitations. The selection of a variety of methods will enable the researcher to gather different types of data, and to look at the research area from complementary angles. As always, it is the role of the supervisor to help the research student put together the best methodology for the research project, that is to say, the best combination of methods through which the student proposes to gather new data on the topic. In most circumstances the supervisor will already have an established preference for one or more methods. It might be necessary to include a second, or third, supervisor who has expertise in a complementary a different set of methods, particularly for multi-disciplinary research.

There are many ways of gathering research data, but broadly they can be divided into three major methodological approaches; these are quantitative, qualitative, and mixed methods. I do not propose to go into much more detail here – there are whole volumes written on even the specific sub-categories of these approaches – but briefly, quantitative research explores through the measurement of phenomena, while a qualitative researcher looks for the emergence of themes or patterns in the evidence provided. A “mixed-methods” approach is not simply a randomly constructed “a-bit-of-one-and-a-bit-of-the-other” style, but it does use both qualitative and quantitative analysis to provide complementary perspectives on the same research topic.

The reason that so much early attention is given to establishing the methodology of the proposed research project is partly because the confirmation of the methodology will determine how the researcher looks at the world emerging through the data; partly, also it will condition the forms of analysis, the reliability, and the compatibility of the research data produced. Any fool can go out and collect data, but getting hold of the type of data which will allow reasonably reliable conclusions to be established is a different matter. In some cases, the choice will be easy. There may be a very limited number of tried-and-tested ways in which an experiment can be constructed, or there might be a very similar study already published, the replication of which to the new subject area might facilitated a useful extension and comparison of knowledge. The supervisor may even have pioneered a particular combination of methods over a long research career and therefore be in a position to give the research student advice on very practical issues, as well as the theory. The literature review is, of course, one element of the methods of research, and the published academic records will likely reveal a quite precise range of options to follow. In any event, it is worth thinking hard right at this stage, in order to avoid false starts and perhaps false data later on.

Where can new data be discovered?


Having prioritised what information the PhD student needs to know in order to make progress towards answering the research question, the next step is to consider where new data can be found. An initial role of the supervisor is to direct the student towards existing data, then to discuss what sort of data might be required to build upon this prior knowledge in order to give new insights. This might not be quite so simple as it at first appears. In an idealised view of research, the problem is articulated, the types of information needed to answer the problem are identified, and then the researcher goes out and collects that data to be analysed. In the real world, there are several problems to be addressed. Firstly, the data that the researcher needs might be hidden, unavailable, or simply difficult to get. Secondly, even if it is accessible, the data needs to be collected in a way that is impartial, systematic, and allows subsequent analysis. Thirdly, there may be problems with the design of the data collection methods, such as obtaining ethical approval, or enabling cross-comparison with previous data, which need to be resolved before the primary research activities can proceed.

The role of a good supervisor is to help smooth the path of the research student without actually doing the data collection work for them. This certainly entails casting a critical eye over the research design and giving friendly feedback. It may require the supervisor to provide a covering letter of introduction for the student, to open doors and archives and to confirm that the student is a serious researcher worth giving some time to. In some situations it may be that the student is directed to existing data sets, either online or in archives, which can be used to provide preliminary analyses. Perhaps the supervisor has already done some research on the subject already, so there are practical tasks which s/he can advise on – the selection of data-gathering methods, the construction of questionnaires or interview schedules, and of course ensuring that any ethical issues relating to the proposed research are adequately covered.

In considering what sort of data are needed to answer the research question, and where this data might be found, the research student and the supervisor have a common interest to ensure that sufficient thought goes into the pre-planning process. Thinking carefully in advance about the possible obstacles involved in collecting robust new data to explore the research topic, is time well-spent. Knowing what to ask will be critical for the study, but knowing who to ask could be more important still. Depending on the subject discipline, and the nature of the study, the identification of key contacts, or an appropriate population of study participants, could make the difference between a perspective which gives a blinding flash of the obvious versus an exciting and innovatory research discovery. It will not be possible to foresee every possible angle of the research process, but having a clear idea of where data can be found, or with whom, is a big step in the research project design.

What do you need to know?

Big Low.JPG

This is another difficult question when starting out on a PhD. If you knew the answer to that, then perhaps you would not need to undertake the PhD in the first place. It gets more complicated still (and yet, at the same time, more straightforward!) Obviously, the initial broad area of investigation will get parsed down into a more specific, more manageable topic, with a particular “research question” or perhaps “research problem” which the student then attempts to explore in detail. The PhD, however, is more than simply a research project looking for a clever answer (although, it can be that as well). The most important component of the PhD is to demonstrate that the doctoral candidate fully understands the process of advanced research at this level. In the context of the research project, this means knowing (or learning) when to explore a new area (or branch of academic literature) and when to narrow the options and explore in greater depth. It means learning when to stop reading and start gathering new data; when to start writing and when (and what) to re-visit, edit, and revise. As the student becomes more deeply embedded in the research topic, this becomes more important, because a fondness for the topic might obscure critical judgment, leading to an attempt to have a comprehensive investigation of everything, rather than developing a speciality which advances knowledge about the subject.

Part of this learning process is also about timing. Students frequently complain that they “don’t have enough time” to complete the project successfully, but this is often the case of trying to squash a six-month project into three months. Bad planning means that something has to be sacrificed. It is always difficult to be deterministic about how long it will take to do each stage of a PhD, because each person and each research problem is different. As a rough guide, however, it is not uncommon that over three-years of full-time PhD research, the student will spend the first nine months or so just getting to grips with the literature. During this time they will perhaps complete the draft of a literature review, giving a narrative on the main events and key articles on the research topic to date. Towards the end of this period, the student will be developing a more intimate feel for the methodological approach and the methods of gathering new data that they wish to adopt. The methodology chapter can probably be drafted quite quickly at this point, although it will usually be necessary to revisit it later to ‘tweak’ the proposal to reflect what was actually done! Having established the preferred research methods, ethical permission to conduct the research can now be applied for and hopefully speedily approved.

The PhD now enters the really interesting point – the ‘meat’ in the sandwich – where the student now begins to conduct the research and gather new data. This period (subject to the above caution that every project is different) may last from the end of the first year (of full-time study) until the beginning of the third year. The student will be writing drafts of the results chapter towards the end of this period, and the final six months or so will usually be spent on writing the discussion/conclusions/recommendations chapter and tidying up the dissertation. Do not underestimate how long it will take to check all the spelling, grammar, citations, references, figure numbers, diagrams, and general formatting! A part-time PhD will obviously take longer than the three-year full-time project (normally 5-6 years) but this rough time-line can be adapted to suit. With a clear initial research question, and a careful approach to each subsequent stage in the process, “what you need to know” usually emerges from the academic mist!

e-learning, networking, and the UHI


ponderings from the world of educational technology in HE

3E Education

Keith Smyth blogging on education, learning, technology, inclusion

Beyond the Horizon

Commentary and Sustainability Policy Analysis from Dr Calum Macleod


Opening Educational Practices in Scotland

Stephen's Web ~ OLDaily

e-learning, networking, and the UHI

The corridor of uncertainty

e-learning, networking, and the UHI

The Ed Techie » The Ed Techie

e-learning, networking, and the UHI

Learning with 'e's

e-learning, networking, and the UHI