Tag Archives: supervision

Starting to get results – Building the picture

Ch5

Take a step backwards. Perhaps the most confusing part of any research project is when some results start to emerge – but this is also one of the most exciting stages. There are three main reactions to the early arrival of research data; firstly, delight that results are finally coming through, as evidence that progress is being made; secondly anxiety that the “correct” information is emerging; thirdly trepidation, if not outright confusion, in attempting to make some sense of the results. All of this happens in quick succession, perhaps even all at once! Hopefully, the cautionary approach to the main data-gathering phase by way of a short pilot study should at least give the research student some confidence that the right research questions were being asked. There may remain doubts that enough data has been gathered – enough interviews generated, enough experiments conducted, enough field investigations made – but the answer to this question will only appear when the research study runs its full course. Despite the temptation to gather AS MUCH data as possible, the experienced supervisor will caution the research student of two hidden dangers that lie in the shallows. One is to remember that gathering the data is only the first part of the story, and the more that is gathered the more there is to be recorded, collated, analysed, interpreted, archived, and all the other ancillary tasks that need to be accomplished in order to secure a robust research project. Secondly, is to recognise the obvious, but often neglected, reality that the quality of the data collected is much more important that the quantity. Gathering a huge mass of data is not much good if the wrong questions have been asked, if important considerations have been missed, or mistaken assumptions made at the earlier stages.

Assuming that the methodology is appropriate, and that the data-gathering methods were systematic, robust, and effective, then every researcher – whether engaged on a small project or a mammoth one – is faced with the same question. “So what does it all mean?” There often comes a natural limit when collecting research data – a point at which it becomes apparent that simply collecting more and more data is not going to substantially change the conclusions. A point of diminishing returns on effort expended. At this point, the “So what?” factor kicks in. It might be necessary to back-track and do some fine-tuning, perhaps to look at some small specific areas in greater detail, or to conduct some follow-up research to fill in some gaps. Perhaps there is a need to explore some adjacent research questions which are tangential to the main research question, but will hopefully provide a better context in which to consider it. Sometimes it helps to simply present the results, devoid of attempts at interpretation, to a few trusted colleagues such as supervisors, to obtain some feedback and get some reassurance on “Do these findings make any sense?” Or perhaps it is time to draw the data-gathering to a halt – even temporarily – and begin to re-assemble the results to piece together what pictures emerge. This is the time when simply getting all the research results down in a systematic, logical, readable form is the main task, and hopefully this will provide a new platform to analyse what the results actually mean.

Advertisements

Editing – deciding what to keep

edits

Deciding what actually needs to be in the final version of the text in a dissertation or a journal paper can be a tough job. Some people do revision after revision, chopping and changing, cutting and adding, re-working the text until they are satisfied. Others (and I am one of them) usually think the subject through, then write the complete text straight off, only making minor changes later before submitting the final version. Whatever way works best for the author is the correct approach. The most important thing to remember is that whatever topic, the dissertation should tell a logical story to the reader. The role of the supervisor is often crucial at this stage, because the writer can frequently get so immersed in the subject matter that it can be difficult to see the wood for the trees. At worst, the writer wants to include everything that they know about the topic – just to be on the safe side. At the other extreme, the writer assumes that the readers understand how they are thinking, and tends to skip on the details, leading to ambiguity or misunderstanding by the readership. Having a “fresh pair of eyes” read over the text can be of immense value – whether it is a friend, a partner, or a supervisor, just having a colleague giving an unbiased view can help to iron out any possible areas for future confusion. Listen to them, and try not to be too defensive: if they have the courage to question you, listen to their opinions. Try not to be pedantic – verbosity and clarity rarely go well together – so consider carefully if your sentence actually contributes towards understanding the text, or is it just padding?

Usually, when writing something as chunky as a 100,000 word PhD dissertation, I would advise that each chapter or section should be drafted, then parked, until the general structure of the full text becomes more clear. Before starting to write the final chapter – the conclusions and any recommendations of the research – the author should pause, go back to the very start of the text, and re-read everything that they have written – making final amendments. Constructing complex narratives, such as dissertations or academic articles, need not be written in a completely linear fashion (i.e. from page one all the way through to the end) so re-visiting the advance draft gives an opportunity to shift paragraphs around, or add/delete information, and generally tidy up the text. This is also a good time to check that all the relevant citations to referenced evidence are included, as well as inserting accurate place-holders for tables, diagrams, and images. The advantage of pausing before starting the last chapter and finalising the earlier text, is that the “story” of the narrative is now fresh in the memory (it may have been a very long time since the author wrote the first few paragraphs of the dissertation). Basically, it is in the best interests of the writer that any readers can follow clearly and understandably the points that are being made. Spelling errors, overly-long sentences, clumsy wording, ambiguous statements, and a lack of referenced evidence all serve to make it more difficult for the reader to understand, and ultimately this reflects badly on the appreciation of the text. A happy reader means a happy examiner, and a better chance that the work will be more widely read and esteemed.

Giving feedback

Feedback sheets

For the supervisor, feedback is perhaps the most difficult aspect of the whole supervision process. The intention of feedback is to enable the recipient to benefit from critically helpful comments and suggestions, but a balance is often difficult to find. To put it simply, the supervisor wants to provide the student with helpful advice to enable the student to improve their performance, but to stop short of actually doing the work for the student. Viewed in this context, any feedback should consist of three parts; a note of what the student has done well; the identification of what can be improved; and suggestions for making these improvements. It is not sufficient to say that, “Your citations are terrible” without explaining how they can be improved. Simply listing the faults can be demoralising and is not sufficiently helpful for learning. Personally, I am not a big fan of the trend to include a “Feed-forward” paragraph, because I belong to the tradition that good feedback always includes within the commentary some instructions on how to make future work even better, so the need for a separate “feed-forward” section is redundant.

That is the broad context, but the level of detail that a student can expect to receive, and the timeliness of such feedback, can be very much case-by-case, and diverse according to different supervisors. When I receive the first pieces of writing from a student, as they complete drafts of individual chapters, I like to give a detailed root-and-branch response. Normally I do this by using track-changes, to insert every missing comma, correct spelling or clumsy grammar, and place annotations in the margin to query or compliment relevant sentences and diagrams. I appreciate that not every supervisor considers this to be part of their role, but I take the view that it is my job to set the benchmark of quality for the student in the presentation of their dissertation. To do this, I can only give them an idea of the standard of writing that I personally would be comfortable with if this was my own presentation. I do not tell the student what to write, but I encourage them by example to present this in the best and most appropriate manner. I work on the (possibly naïve) idea that every student wants to exert themselves to the highest standards possible, and therefore when I make suggestions on how to improve the work, these suggestions are made with the best intention to benefit the student. I leave the decision on whether or not to accept my changes and comments to the wisdom of the student. If s/he feels that their original version is better, that is their decision, but if the External Examiner demands the same changes that I have suggested, I know that it is not because the student has not been given that advice by me, merely that they have not chosen to heed it.

Timing is another variable issue. At my current university we are required that “normally” (a wonderful word) we are expected to return feedback to students within ten working days of the submission deadline, and I think this is fair. The purpose, after all, of feedback is to help the student to improve their future work, and this is best done while the submitted work is relatively fresh in their memory, and before the student starts make similar mistakes in the next piece of work to be submitted for assessment. In practice, with research students, ongoing feedback can be given in a variety of ways – written or verbal – using a diversity of media, including text, telephone support, chats in the corridor, and formal sessions either face-to-face or using video-chat. It is wise to explore very early in the supervision process what works best for the individual student and the individual supervisor.

Setting the tone of academic writing

Evaluation

There is a lot of nonsense talked about “academic writing” in some circles. A central myth is that it needs to be “complex”. In fact, exactly the reverse is the case! In writing an academic text, the author needs to be aware of some of the same key issues as any author, whether the writing is fact or fiction, science or humanities. Firstly, the text needs to convey information to the readership. Even complex ideas and intricate research can be conveyed as a story which captivates the reader’s attention and (hopefully) helps their understanding. So good academic writing is not simply about the message, it is also, to some extent, about the style. A well-written chapter or article will be a pleasure to read and will stimulate the interest of the reader, even if they may not follow (or even agree with) everything that you claim. For this reason, it is just as important to pay close attention to spelling, grammar, and the structure of an academic article as it is for a good piece of journalism.

An academic article requires another couple of essentials, however, and these are ‘evidence’ and ‘analysis’. The main reason for writing an academic article (or PhD chapter) is to make known to the readership some new ideas – perhaps the results of a new experiment (or the confirmation by repetition of an earlier experiment) or perhaps simply bringing together scattered information to present a new way of thinking about the topic. Either way, the ‘story’ that is written will probably build upon earlier work, perhaps quoting some examples, or statistics, attempting to construct a picture of how the new information was obtained. In this synthesis, it is imperative that the writer identifies the sources of evidence which are being referred to – even in passing – in the construction of the storyline. This sometimes gives academic writing a bit of a staccato appearance, with frequent interruptions e.g. (Rennie and Smyth, 2017) to the flow of sentences that would be the norm for a non-academic article. Nevertheless, these citations to the sources of evidence are absolutely essential in order to place the new piece of writing within the context of what is already known about the topic. Remember, the purpose of research, and the PhD in particular, is to make an original contribution to knowledge, by extending what is known into an area which is less well known, and by definition extending the sum total of our knowledge of the discipline. There are different conventions on how to draw attention to the sources of evidence which are used  to give support, reliability, confidence, to the new ideas being expressed, and these citation styles – such as Harvard, Vancouver, APA – will vary with different academic disciplines. Students should check with their supervisors on what is most appropriate (sometimes the required styles will vary between different journals).

With respect to the ‘analysis’ component of the writing, this will vary between different academic levels, and occasionally even within the same piece of academic work. For instance, early-stage undergraduates may be allowed to be more descriptive in their writing, but late-stage undergraduates are expected to be more highly analytical, rather than purely descriptive. By the stage of embarking on a research degree, the student is expected to understand the importance of critical analysis, (and practice it) so that although a literature review chapter may in broad terms describe the state of current knowledge about the research topic, the reviews of the individual sources of evidence should not be solely descriptive, and should critically evaluate the strengths and possible weaknesses of the source publications.

For this reason, I try to give a particularly thorough feedback on the early work of any research student that I am supervising. I use “track changes” to comment on every missing comma, typographic error, lack of citation, or inappropriate style format. If the supervisor can quickly and clearly set the tone required for the relevant level of the student’s work, a benchmark can be established, and thereafter the student should be clear about the quality, style, conventions, and expectations required for the final product. At least, that is the theory…

Setting a routine

cycle

I think it was Graham Greene who used to say that he aimed to write 500 words every day. The novels were soon created. This might not sound like a lot of words, but there are two great advantages to this method. Firstly, 500 words every single day, even when some of the words are later amended or discarded, soon builds up to a substantial narrative. This narrative can then be edited, refined, extended or reduced. Secondly, and perhaps more importantly, the routine act of writing down 500 words each day cultivates a mind-set which develops with constant practice, so that it becomes easier to express your ideas in writing. For some people, it may never become easy, but it does become easier. It helps if the writer is also a regular reader. To become familiar with the way other writers express themselves in text, even if their language or the style is unfamiliar or even disliked, is a useful skill because it enables the writer to understand their own style, and how to capture in words what they want to say.

Most academic writing has a different appearance in style to other forms of literature, because there is a different purpose behind it. As a scientist, I am the first one to agree that scientific writing can also be creative, but analytical writing for an academic purpose – whether this is for science, arts, or the humanities – demands that the text is anchored in such things as theories, concepts, and evidence. Most non-academic writing (apart from things such as biographies or popular histories) do not require citations (e.g. “(Rennie, 2017)” but these citations are essential for academic work to provide the sources of the evidence on which your subsequent ideas are based.

In order to get into a routine which suits your own working style and personality, you need to experiment a little. Some people, like Graham Greene, prefer to set-aside some time each day to write. Others only write when the mood takes them, when they feel inspired, or when a deadline looms over them. Personally, I find writing very easy to do, and I enjoy it, so I have different behaviour patterns for different situations. I know that I can sit down and produce something very quickly if I need to (like a report of work done), but for deeper and more complex work (such as a journal article or research paper) I like to start off with a working title and some headings to give the article a bit of structure. With the general ‘story-line’ in my head, I will then sit down to write the various sections when I think I know what I want to say in each section. I build the article up, then leave it a couple of days, read it again, and make any minor changes. I rarely re-write anything substantial unless I obtain new information or get feedback from a reviewer to expand upon some point of explanation. So, my routine is to establish what I want to say, build up the article as a story, then tweak the final draft until I am satisfied that I have expressed what I want to convey. Other writers will write, re-write, and re-re-write as their ideas change and the article evolves. A key point in all of this is that the finished piece of text, whether it is a research paper or a dissertation, should be enjoyable for the reader, so try to avoid long, cumbersome sentences and clearly signpost the direction of your discussion. Numbered headings and spell checking is also important, so make sure that you develop your own routine to check and double-check each stage as you progress with your text.

Useful webpages:

https://www.timeshighereducation.com/features/essential-phd-tips-10-articles-all-doctoral-students-should-read

Getting research ethics approval

waysigns

Once the research question(s) and the probable method(s) of gathering new data have been established, the next stage in any research process is to ensure that the following stage(s) of the research project will be ethically appropriate for the task(s) in hand. In actual fact, the ethical considerations will have been made at a general level much earlier in the research planning, but it is only now, when there is a greater certainty about the methodology and the details of the data-gathering methods, that the researcher will be sure of the complete nature of the ethical issues which might be involved in the research.

It is not uncommon at this stage for researchers, especially novice researchers, to declare that their particular research project “does not really have any ethical issues”. This is almost always untrue. Whether it is a simple survey with a clipboard and pen, or a more complex set of interviews, questionnaires or focus groups, there will always be issues relating to the nature of the information intended to be gathered, from whom it will be gathered, or how the subsequent data might be stored and made available to other people. Research projects involving patients, animals, children, or vulnerable adults will require especially stringent ethical codes of practice. Laboratory experiments, fieldwork, or simply re-working the data already gathered by a previous researcher, will all have their own distinct and necessary ethical guidelines to consider. At the very, very least, there are ethical questions which need to be considered about who is funding the research, who decides what research gets a priority, and what do the funding organisations receive in return for their financial support?

The supervisor has a crucial role in talking through with the student all of the possible ethical issues which might impact upon the research. Frequently, the ethical issues can be resolved very easily, and the research can proceed, but the simple fact of working through the range possible issues which might arise, and sharing experience on how they can be satisfactorily dealt with, is an important part of the professional training required by the student.

Every university, and most professional associations which come into contact with research, will have codes of conduct and formal procedures for scrutinising and approving research ethics proposals for research projects with which they are involved. This will require specific application forms, a scrutinising committee, and a formal code of research behaviour with which researchers are required to conform. Some procedures to gain ethical approval for research are particularly detailed, for example anything requiring contact with the health service, such as patient notes, contact with patients, or any engagement with either health staff or medical procedures, is likely to involve substantial detail and very careful research design.

Key concerns in all research ethics matters are to avoid causing harm, to respect confidentiality, and to maintain high standards of moral integrity. The latter, for instance, might refer to a very wide range of “common sense” standards such as to refrain from cheating, plagiarism, falsifying results, vested interests, and so on. Though they may seem “common sense” to most of us, we tend to forget that many of these issues are perceived differently by different cultures, and influenced by pressures which might be applied – internal and external to the institution – to “encourage” researchers to produce favourable results one way or another. It is for these reasons that gaining the approval of the research ethics committee is a fundamental gateway for any research project before it can be seriously undertaken.

Data analysis

dataset

The data analysis stage is one of the main research areas where the supervisory team can really make a significant contribution to assisting the research student. Obviously, the student still needs to do the work for themselves, but this is a stage in the PhD process where the depth and breadth of experience of the supervisors should shine through and help the student to make sense of a complex set of tasks, and make them a bit simpler to complete. Having collected a mass of data, perhaps even coded and categorised this data according to exacting and laborious protocols and methods of analysis, the student needs to understand what this data is actually saying. This might be a simpler task for some projects than for others, according to the amount of data collected, the form in which it was collected, how detailed or exact the observations or calculations are, or what methods for codifying or interpreting the raw data have been employed in the research methodology.

To the beginner, this might seem straightforward, but there is no “one-way” to analyse data, because there are many, many different forms of data. This data might be collected at different levels of granularity, different levels of accuracy, and embody different assumptions and methodological approaches. At some early stage of the analysis it is usually a good idea for the research student to sit down with the supervisory team, spread the collected data out on a table, and look at it together. The student needs to identify a number of key attributes of this data, such as what does this data actually indicate? How robust is the data? What is its accuracy and what are its limitations? Are there any correlations (positive or negative) and so on, leading to the penultimate question, which is what does this data actually tell us? Hopefully there will be a new insight, or a discovery which, at least in part, will address the initial research question(s). The final question is then, how should I present this information so that it understandable to others who have not been involved in this research, and how transferrable is this knowledge to other (present and future) researchers in this general subject area? The ability to repeat, re-combine, and re-use data (and the results of its analysis) is a particularly useful feature to enable contrast and comparison with other projects in similar or related subject areas.

It will seem a bit ironic, or a perhaps a bit of a paradox, given that we are often seeking to reveal hitherto unknown facts, and “answers” to high-level research questions, but usually it is better to be less ambitious in the interpretation of the conclusions, but to be absolutely sure of the reliability and credibility of our data, rather than to propose over-ambitious conclusions (exciting thought they may seem) which are based on sketchy evidence and correlations which are skating on thin ice. It the research can be shown to produce solid, incontrovertible evidence, regardless of how large or small the breakthrough, then this small advance can be built-upon by subsequent researchers. If the conclusions are like a house built on thin ice, then there is always a doubt about the credibility of the data, or the results, and therefore the value of the research output is devalued. With data analysis, it always pays to check, cross-check, consider the constraints and limitations, then double-check each stage before you venture to draw conclusions to share with a wider audience.