Starting to get results – Building the picture

Ch5

Take a step backwards. Perhaps the most confusing part of any research project is when some results start to emerge – but this is also one of the most exciting stages. There are three main reactions to the early arrival of research data; firstly, delight that results are finally coming through, as evidence that progress is being made; secondly anxiety that the “correct” information is emerging; thirdly trepidation, if not outright confusion, in attempting to make some sense of the results. All of this happens in quick succession, perhaps even all at once! Hopefully, the cautionary approach to the main data-gathering phase by way of a short pilot study should at least give the research student some confidence that the right research questions were being asked. There may remain doubts that enough data has been gathered – enough interviews generated, enough experiments conducted, enough field investigations made – but the answer to this question will only appear when the research study runs its full course. Despite the temptation to gather AS MUCH data as possible, the experienced supervisor will caution the research student of two hidden dangers that lie in the shallows. One is to remember that gathering the data is only the first part of the story, and the more that is gathered the more there is to be recorded, collated, analysed, interpreted, archived, and all the other ancillary tasks that need to be accomplished in order to secure a robust research project. Secondly, is to recognise the obvious, but often neglected, reality that the quality of the data collected is much more important that the quantity. Gathering a huge mass of data is not much good if the wrong questions have been asked, if important considerations have been missed, or mistaken assumptions made at the earlier stages.

Assuming that the methodology is appropriate, and that the data-gathering methods were systematic, robust, and effective, then every researcher – whether engaged on a small project or a mammoth one – is faced with the same question. “So what does it all mean?” There often comes a natural limit when collecting research data – a point at which it becomes apparent that simply collecting more and more data is not going to substantially change the conclusions. A point of diminishing returns on effort expended. At this point, the “So what?” factor kicks in. It might be necessary to back-track and do some fine-tuning, perhaps to look at some small specific areas in greater detail, or to conduct some follow-up research to fill in some gaps. Perhaps there is a need to explore some adjacent research questions which are tangential to the main research question, but will hopefully provide a better context in which to consider it. Sometimes it helps to simply present the results, devoid of attempts at interpretation, to a few trusted colleagues such as supervisors, to obtain some feedback and get some reassurance on “Do these findings make any sense?” Or perhaps it is time to draw the data-gathering to a halt – even temporarily – and begin to re-assemble the results to piece together what pictures emerge. This is the time when simply getting all the research results down in a systematic, logical, readable form is the main task, and hopefully this will provide a new platform to analyse what the results actually mean.

Advertisements

Editing – deciding what to keep

edits

Deciding what actually needs to be in the final version of the text in a dissertation or a journal paper can be a tough job. Some people do revision after revision, chopping and changing, cutting and adding, re-working the text until they are satisfied. Others (and I am one of them) usually think the subject through, then write the complete text straight off, only making minor changes later before submitting the final version. Whatever way works best for the author is the correct approach. The most important thing to remember is that whatever topic, the dissertation should tell a logical story to the reader. The role of the supervisor is often crucial at this stage, because the writer can frequently get so immersed in the subject matter that it can be difficult to see the wood for the trees. At worst, the writer wants to include everything that they know about the topic – just to be on the safe side. At the other extreme, the writer assumes that the readers understand how they are thinking, and tends to skip on the details, leading to ambiguity or misunderstanding by the readership. Having a “fresh pair of eyes” read over the text can be of immense value – whether it is a friend, a partner, or a supervisor, just having a colleague giving an unbiased view can help to iron out any possible areas for future confusion. Listen to them, and try not to be too defensive: if they have the courage to question you, listen to their opinions. Try not to be pedantic – verbosity and clarity rarely go well together – so consider carefully if your sentence actually contributes towards understanding the text, or is it just padding?

Usually, when writing something as chunky as a 100,000 word PhD dissertation, I would advise that each chapter or section should be drafted, then parked, until the general structure of the full text becomes more clear. Before starting to write the final chapter – the conclusions and any recommendations of the research – the author should pause, go back to the very start of the text, and re-read everything that they have written – making final amendments. Constructing complex narratives, such as dissertations or academic articles, need not be written in a completely linear fashion (i.e. from page one all the way through to the end) so re-visiting the advance draft gives an opportunity to shift paragraphs around, or add/delete information, and generally tidy up the text. This is also a good time to check that all the relevant citations to referenced evidence are included, as well as inserting accurate place-holders for tables, diagrams, and images. The advantage of pausing before starting the last chapter and finalising the earlier text, is that the “story” of the narrative is now fresh in the memory (it may have been a very long time since the author wrote the first few paragraphs of the dissertation). Basically, it is in the best interests of the writer that any readers can follow clearly and understandably the points that are being made. Spelling errors, overly-long sentences, clumsy wording, ambiguous statements, and a lack of referenced evidence all serve to make it more difficult for the reader to understand, and ultimately this reflects badly on the appreciation of the text. A happy reader means a happy examiner, and a better chance that the work will be more widely read and esteemed.

Giving feedback

Feedback sheets

For the supervisor, feedback is perhaps the most difficult aspect of the whole supervision process. The intention of feedback is to enable the recipient to benefit from critically helpful comments and suggestions, but a balance is often difficult to find. To put it simply, the supervisor wants to provide the student with helpful advice to enable the student to improve their performance, but to stop short of actually doing the work for the student. Viewed in this context, any feedback should consist of three parts; a note of what the student has done well; the identification of what can be improved; and suggestions for making these improvements. It is not sufficient to say that, “Your citations are terrible” without explaining how they can be improved. Simply listing the faults can be demoralising and is not sufficiently helpful for learning. Personally, I am not a big fan of the trend to include a “Feed-forward” paragraph, because I belong to the tradition that good feedback always includes within the commentary some instructions on how to make future work even better, so the need for a separate “feed-forward” section is redundant.

That is the broad context, but the level of detail that a student can expect to receive, and the timeliness of such feedback, can be very much case-by-case, and diverse according to different supervisors. When I receive the first pieces of writing from a student, as they complete drafts of individual chapters, I like to give a detailed root-and-branch response. Normally I do this by using track-changes, to insert every missing comma, correct spelling or clumsy grammar, and place annotations in the margin to query or compliment relevant sentences and diagrams. I appreciate that not every supervisor considers this to be part of their role, but I take the view that it is my job to set the benchmark of quality for the student in the presentation of their dissertation. To do this, I can only give them an idea of the standard of writing that I personally would be comfortable with if this was my own presentation. I do not tell the student what to write, but I encourage them by example to present this in the best and most appropriate manner. I work on the (possibly naïve) idea that every student wants to exert themselves to the highest standards possible, and therefore when I make suggestions on how to improve the work, these suggestions are made with the best intention to benefit the student. I leave the decision on whether or not to accept my changes and comments to the wisdom of the student. If s/he feels that their original version is better, that is their decision, but if the External Examiner demands the same changes that I have suggested, I know that it is not because the student has not been given that advice by me, merely that they have not chosen to heed it.

Timing is another variable issue. At my current university we are required that “normally” (a wonderful word) we are expected to return feedback to students within ten working days of the submission deadline, and I think this is fair. The purpose, after all, of feedback is to help the student to improve their future work, and this is best done while the submitted work is relatively fresh in their memory, and before the student starts make similar mistakes in the next piece of work to be submitted for assessment. In practice, with research students, ongoing feedback can be given in a variety of ways – written or verbal – using a diversity of media, including text, telephone support, chats in the corridor, and formal sessions either face-to-face or using video-chat. It is wise to explore very early in the supervision process what works best for the individual student and the individual supervisor.

Setting the tone of academic writing

Evaluation

There is a lot of nonsense talked about “academic writing” in some circles. A central myth is that it needs to be “complex”. In fact, exactly the reverse is the case! In writing an academic text, the author needs to be aware of some of the same key issues as any author, whether the writing is fact or fiction, science or humanities. Firstly, the text needs to convey information to the readership. Even complex ideas and intricate research can be conveyed as a story which captivates the reader’s attention and (hopefully) helps their understanding. So good academic writing is not simply about the message, it is also, to some extent, about the style. A well-written chapter or article will be a pleasure to read and will stimulate the interest of the reader, even if they may not follow (or even agree with) everything that you claim. For this reason, it is just as important to pay close attention to spelling, grammar, and the structure of an academic article as it is for a good piece of journalism.

An academic article requires another couple of essentials, however, and these are ‘evidence’ and ‘analysis’. The main reason for writing an academic article (or PhD chapter) is to make known to the readership some new ideas – perhaps the results of a new experiment (or the confirmation by repetition of an earlier experiment) or perhaps simply bringing together scattered information to present a new way of thinking about the topic. Either way, the ‘story’ that is written will probably build upon earlier work, perhaps quoting some examples, or statistics, attempting to construct a picture of how the new information was obtained. In this synthesis, it is imperative that the writer identifies the sources of evidence which are being referred to – even in passing – in the construction of the storyline. This sometimes gives academic writing a bit of a staccato appearance, with frequent interruptions e.g. (Rennie and Smyth, 2017) to the flow of sentences that would be the norm for a non-academic article. Nevertheless, these citations to the sources of evidence are absolutely essential in order to place the new piece of writing within the context of what is already known about the topic. Remember, the purpose of research, and the PhD in particular, is to make an original contribution to knowledge, by extending what is known into an area which is less well known, and by definition extending the sum total of our knowledge of the discipline. There are different conventions on how to draw attention to the sources of evidence which are used  to give support, reliability, confidence, to the new ideas being expressed, and these citation styles – such as Harvard, Vancouver, APA – will vary with different academic disciplines. Students should check with their supervisors on what is most appropriate (sometimes the required styles will vary between different journals).

With respect to the ‘analysis’ component of the writing, this will vary between different academic levels, and occasionally even within the same piece of academic work. For instance, early-stage undergraduates may be allowed to be more descriptive in their writing, but late-stage undergraduates are expected to be more highly analytical, rather than purely descriptive. By the stage of embarking on a research degree, the student is expected to understand the importance of critical analysis, (and practice it) so that although a literature review chapter may in broad terms describe the state of current knowledge about the research topic, the reviews of the individual sources of evidence should not be solely descriptive, and should critically evaluate the strengths and possible weaknesses of the source publications.

For this reason, I try to give a particularly thorough feedback on the early work of any research student that I am supervising. I use “track changes” to comment on every missing comma, typographic error, lack of citation, or inappropriate style format. If the supervisor can quickly and clearly set the tone required for the relevant level of the student’s work, a benchmark can be established, and thereafter the student should be clear about the quality, style, conventions, and expectations required for the final product. At least, that is the theory…

Setting a routine

cycle

I think it was Graham Greene who used to say that he aimed to write 500 words every day. The novels were soon created. This might not sound like a lot of words, but there are two great advantages to this method. Firstly, 500 words every single day, even when some of the words are later amended or discarded, soon builds up to a substantial narrative. This narrative can then be edited, refined, extended or reduced. Secondly, and perhaps more importantly, the routine act of writing down 500 words each day cultivates a mind-set which develops with constant practice, so that it becomes easier to express your ideas in writing. For some people, it may never become easy, but it does become easier. It helps if the writer is also a regular reader. To become familiar with the way other writers express themselves in text, even if their language or the style is unfamiliar or even disliked, is a useful skill because it enables the writer to understand their own style, and how to capture in words what they want to say.

Most academic writing has a different appearance in style to other forms of literature, because there is a different purpose behind it. As a scientist, I am the first one to agree that scientific writing can also be creative, but analytical writing for an academic purpose – whether this is for science, arts, or the humanities – demands that the text is anchored in such things as theories, concepts, and evidence. Most non-academic writing (apart from things such as biographies or popular histories) do not require citations (e.g. “(Rennie, 2017)” but these citations are essential for academic work to provide the sources of the evidence on which your subsequent ideas are based.

In order to get into a routine which suits your own working style and personality, you need to experiment a little. Some people, like Graham Greene, prefer to set-aside some time each day to write. Others only write when the mood takes them, when they feel inspired, or when a deadline looms over them. Personally, I find writing very easy to do, and I enjoy it, so I have different behaviour patterns for different situations. I know that I can sit down and produce something very quickly if I need to (like a report of work done), but for deeper and more complex work (such as a journal article or research paper) I like to start off with a working title and some headings to give the article a bit of structure. With the general ‘story-line’ in my head, I will then sit down to write the various sections when I think I know what I want to say in each section. I build the article up, then leave it a couple of days, read it again, and make any minor changes. I rarely re-write anything substantial unless I obtain new information or get feedback from a reviewer to expand upon some point of explanation. So, my routine is to establish what I want to say, build up the article as a story, then tweak the final draft until I am satisfied that I have expressed what I want to convey. Other writers will write, re-write, and re-re-write as their ideas change and the article evolves. A key point in all of this is that the finished piece of text, whether it is a research paper or a dissertation, should be enjoyable for the reader, so try to avoid long, cumbersome sentences and clearly signpost the direction of your discussion. Numbered headings and spell checking is also important, so make sure that you develop your own routine to check and double-check each stage as you progress with your text.

Useful webpages:

https://www.timeshighereducation.com/features/essential-phd-tips-10-articles-all-doctoral-students-should-read

Starting to write-up

writing

Sorting the structure

When is the best time to start writing the PhD dissertation? This really depends on what you have to say that is of any importance. Some people try to start too early – before they have done any primary research, really – while others attempt to put the writing off for as long as possible! Like any skill, writing gets better with practice, so the usual advice is to start early – perhaps within the first couple of months – but be prepared to edit, revise, and if necessary throw away, your early attempts. As a supervisor of research students, I like to see the early attempts at writing in order to give some constructive feedback, and help the student “set the tone” or the correct level of the academic writing. For me, this early writing should come in fully constructed sentences – not bullet-points – and there should be a logical narrative which sets the scene of the research activities. It should be spell-checked, fully referenced, and grammatically correct. I give detailed comments and changes using the ‘track-changes’ function, to give the student a clear idea of the standard required of the final text (the student is then free to accept or ignore this advice – and that response tells me a lot about the professionalism of the student). This level of feedback might not be true for all supervisors. In fact, I know of some fully-fledged academics whose grammar and sentence construction lets them down badly, so they are unlikely to be very keen or useful critics of the fine nuances of the English language, however good they are in their own subject area. My view is that the quality and impact of my writing is a reflection of me, and by implication, the work of one of my students is an indirect reflection of me.

The quality of writing is important to me, because the production of a written dissertation – usually a maximum or 100,000 words in the UK – is the central work of the PhD, around which everything else hangs. The brilliance of the research, the care and skill in crafting the research process, the novelty of the solutions and conclusions – even the defence in the viva – are dashed to nothing if the student is not able to communicate clearly and engagingly. In a nutshell, if the dissertation is laborious and difficult to read, if it contains silly errors, lack of references to evidence, or simply is written in a tortuous style or in ambiguous language, the readers (including the examiners) will become frustrated, confused, and perhaps overly-critical. If they look for faults in your work, they will surely find them. On the other hand, if the dissertation is a pleasure to read, if it is well constructed and well presented, the reader might skip over any minor faults in their enthusiasm to follow the story. The examiners might look kindly on minor issues and even suggest how these could be easily improved. The skill is to construct a narrative which guides the reader through the research story, in much the same way as a novel, or a detective story, in which each chapter leads smoothly and logically into the next. Like most skills, writing is improved by training and practice, so starting with a skeleton list of the dissertation contents might be useful, allowing the student to break up the text into shorter sections and sub-sections, which can be edited and linked together in an ongoing process. The PhD dissertation does not need to be written in a totally linear manner, and it is quite common to double-back to add, modify, or delete earlier sections of writing as new facts become available or new academic articles are discovered. A key requirement is to be organised, and to approach the write-up methodically, but in easy stages.

Book review

http://blogs.lse.ac.uk/lsereviewofbooks/2016/11/11/book-review-a-survival-kit-for-doctoral-students-and-their-supervisors-traveling-the-landscape-of-research-by-lene-tanggaard-and-charlotte-wegener/

Getting research ethics approval

waysigns

Once the research question(s) and the probable method(s) of gathering new data have been established, the next stage in any research process is to ensure that the following stage(s) of the research project will be ethically appropriate for the task(s) in hand. In actual fact, the ethical considerations will have been made at a general level much earlier in the research planning, but it is only now, when there is a greater certainty about the methodology and the details of the data-gathering methods, that the researcher will be sure of the complete nature of the ethical issues which might be involved in the research.

It is not uncommon at this stage for researchers, especially novice researchers, to declare that their particular research project “does not really have any ethical issues”. This is almost always untrue. Whether it is a simple survey with a clipboard and pen, or a more complex set of interviews, questionnaires or focus groups, there will always be issues relating to the nature of the information intended to be gathered, from whom it will be gathered, or how the subsequent data might be stored and made available to other people. Research projects involving patients, animals, children, or vulnerable adults will require especially stringent ethical codes of practice. Laboratory experiments, fieldwork, or simply re-working the data already gathered by a previous researcher, will all have their own distinct and necessary ethical guidelines to consider. At the very, very least, there are ethical questions which need to be considered about who is funding the research, who decides what research gets a priority, and what do the funding organisations receive in return for their financial support?

The supervisor has a crucial role in talking through with the student all of the possible ethical issues which might impact upon the research. Frequently, the ethical issues can be resolved very easily, and the research can proceed, but the simple fact of working through the range possible issues which might arise, and sharing experience on how they can be satisfactorily dealt with, is an important part of the professional training required by the student.

Every university, and most professional associations which come into contact with research, will have codes of conduct and formal procedures for scrutinising and approving research ethics proposals for research projects with which they are involved. This will require specific application forms, a scrutinising committee, and a formal code of research behaviour with which researchers are required to conform. Some procedures to gain ethical approval for research are particularly detailed, for example anything requiring contact with the health service, such as patient notes, contact with patients, or any engagement with either health staff or medical procedures, is likely to involve substantial detail and very careful research design.

Key concerns in all research ethics matters are to avoid causing harm, to respect confidentiality, and to maintain high standards of moral integrity. The latter, for instance, might refer to a very wide range of “common sense” standards such as to refrain from cheating, plagiarism, falsifying results, vested interests, and so on. Though they may seem “common sense” to most of us, we tend to forget that many of these issues are perceived differently by different cultures, and influenced by pressures which might be applied – internal and external to the institution – to “encourage” researchers to produce favourable results one way or another. It is for these reasons that gaining the approval of the research ethics committee is a fundamental gateway for any research project before it can be seriously undertaken.

e-learning, networking, and the UHI

Hebrides Writer

Arts. Culture. Opinion.

Jon Dron's home page

Aggregated musings, news and stories, mostly about learning and technology

howsheilaseesIT

ponderings from the world of educational technology in HE

3E Education

Keith Smyth blogging on education, learning, technology, inclusion

Beyond the Horizon

Commentary and Sustainability Policy Analysis from Dr Calum Macleod

OEPScotland

Opening Educational Practices in Scotland

The corridor of uncertainty

e-learning, networking, and the UHI

The Ed Techie

e-learning, networking, and the UHI

Learning with 'e's

e-learning, networking, and the UHI