# Using statistics in research

For me, the golden, unbreakable rule is, never use any statistics if you do not really understand what they mean! This might seem obvious, but it is surprising how frequently statistics get misused or misinterpreted to support an argument that actually has no real basis in fact. The famous saying that there are three types of lies – “lies, damned lies, and statistics” is attributed to Prime Minister Disraeli, but when properly used, statistics can be clear, unequivocal, and very supportive in communicating complex data simply. The main problem(s) are often started by people deliberately selecting the information that they want to hear and then seeking for statistical back-up which looks impressive and difficult to challenge. Secondary problems occur when people either do not properly understand what the real statistics are saying, or when people choose to deliberately select or twist the statistical information which seems to support their preferred point of view. Ultimately, all three problem areas both devalue the use of statistics and also take us even further away from a clearer understanding of the situation that we are trying to interpret and communicate.

A good “rule of thumb” is to stick to the simplest means of statistically expressing the results that you want to communicate. Percentages, pie-charts, and histograms might look fairly unadventurous if you are trying to impress an examiner, but they have the advantage that they are quick to produce, clear to interpret, and easy to understand. Fancy calculations may look more impressive, but they are frequently harder to produce, more difficult to fully understand, and have a greater chance of either the creator or the reader making errors of interpretation. A common error is to quote percentages rather than give simple numerical values for small population samples. If 7 out of 12 of your interviewees agree, say “7 out of 12 agree that…” rather than “58.33%…” A difference of 1 person immediately gives an 8% error and is clumsier. Keeping it simple gives both a truer impression of the data and an easier comparison with other results. Similarly, I have seen some very impressive and complex diagrams, complete with 3-D shading and vector trends, which actually do not tell us very much at all because the detail is lost in the artistic flamboyance. They look fancy but add nothing to the discussion.

Quite often, certain disciplinary areas will have their own conventions as to which statistical procedures are common, or preferred, and how they are presented. The supervisors should be able to advise on these common standards, and the benefit is usually that the new research data can both make use of earlier research results, and can be easily contrasted and compared with already published data in the discipline. Some statistical procedures can look complicated to calculate, but are actually quite straightforward to use. All universities will have opportunities for research students to attend courses on statistical methods which are appropriate for different subject areas of research, so students should get training early in the research to avoid any subsequent false starts. There are lots of self-guide short courses on the web, and of course in text-books, but self-tuition is also open to misunderstanding, so it is always best, in the first exploration at least, to work through the procedure(s) alongside someone who is already very familiar with the statistical technique(s). Bear in mind, contrary to some current political rhetoric, there are no “alternative facts” simply facts that you acknowledge and facts that you might prefer to ignore. Research is about improving knowledge, not picking just the bits that you like.

# Setting a routine

I think it was Graham Greene who used to say that he aimed to write 500 words every day. The novels were soon created. This might not sound like a lot of words, but there are two great advantages to this method. Firstly, 500 words every single day, even when some of the words are later amended or discarded, soon builds up to a substantial narrative. This narrative can then be edited, refined, extended or reduced. Secondly, and perhaps more importantly, the routine act of writing down 500 words each day cultivates a mind-set which develops with constant practice, so that it becomes easier to express your ideas in writing. For some people, it may never become easy, but it does become easier. It helps if the writer is also a regular reader. To become familiar with the way other writers express themselves in text, even if their language or the style is unfamiliar or even disliked, is a useful skill because it enables the writer to understand their own style, and how to capture in words what they want to say.

Most academic writing has a different appearance in style to other forms of literature, because there is a different purpose behind it. As a scientist, I am the first one to agree that scientific writing can also be creative, but analytical writing for an academic purpose – whether this is for science, arts, or the humanities – demands that the text is anchored in such things as theories, concepts, and evidence. Most non-academic writing (apart from things such as biographies or popular histories) do not require citations (e.g. “(Rennie, 2017)” but these citations are essential for academic work to provide the sources of the evidence on which your subsequent ideas are based.

In order to get into a routine which suits your own working style and personality, you need to experiment a little. Some people, like Graham Greene, prefer to set-aside some time each day to write. Others only write when the mood takes them, when they feel inspired, or when a deadline looms over them. Personally, I find writing very easy to do, and I enjoy it, so I have different behaviour patterns for different situations. I know that I can sit down and produce something very quickly if I need to (like a report of work done), but for deeper and more complex work (such as a journal article or research paper) I like to start off with a working title and some headings to give the article a bit of structure. With the general ‘story-line’ in my head, I will then sit down to write the various sections when I think I know what I want to say in each section. I build the article up, then leave it a couple of days, read it again, and make any minor changes. I rarely re-write anything substantial unless I obtain new information or get feedback from a reviewer to expand upon some point of explanation. So, my routine is to establish what I want to say, build up the article as a story, then tweak the final draft until I am satisfied that I have expressed what I want to convey. Other writers will write, re-write, and re-re-write as their ideas change and the article evolves. A key point in all of this is that the finished piece of text, whether it is a research paper or a dissertation, should be enjoyable for the reader, so try to avoid long, cumbersome sentences and clearly signpost the direction of your discussion. Numbered headings and spell checking is also important, so make sure that you develop your own routine to check and double-check each stage as you progress with your text.

Useful webpages:

# Getting research ethics approval

Once the research question(s) and the probable method(s) of gathering new data have been established, the next stage in any research process is to ensure that the following stage(s) of the research project will be ethically appropriate for the task(s) in hand. In actual fact, the ethical considerations will have been made at a general level much earlier in the research planning, but it is only now, when there is a greater certainty about the methodology and the details of the data-gathering methods, that the researcher will be sure of the complete nature of the ethical issues which might be involved in the research.

It is not uncommon at this stage for researchers, especially novice researchers, to declare that their particular research project “does not really have any ethical issues”. This is almost always untrue. Whether it is a simple survey with a clipboard and pen, or a more complex set of interviews, questionnaires or focus groups, there will always be issues relating to the nature of the information intended to be gathered, from whom it will be gathered, or how the subsequent data might be stored and made available to other people. Research projects involving patients, animals, children, or vulnerable adults will require especially stringent ethical codes of practice. Laboratory experiments, fieldwork, or simply re-working the data already gathered by a previous researcher, will all have their own distinct and necessary ethical guidelines to consider. At the very, very least, there are ethical questions which need to be considered about who is funding the research, who decides what research gets a priority, and what do the funding organisations receive in return for their financial support?

The supervisor has a crucial role in talking through with the student all of the possible ethical issues which might impact upon the research. Frequently, the ethical issues can be resolved very easily, and the research can proceed, but the simple fact of working through the range possible issues which might arise, and sharing experience on how they can be satisfactorily dealt with, is an important part of the professional training required by the student.

Every university, and most professional associations which come into contact with research, will have codes of conduct and formal procedures for scrutinising and approving research ethics proposals for research projects with which they are involved. This will require specific application forms, a scrutinising committee, and a formal code of research behaviour with which researchers are required to conform. Some procedures to gain ethical approval for research are particularly detailed, for example anything requiring contact with the health service, such as patient notes, contact with patients, or any engagement with either health staff or medical procedures, is likely to involve substantial detail and very careful research design.

Key concerns in all research ethics matters are to avoid causing harm, to respect confidentiality, and to maintain high standards of moral integrity. The latter, for instance, might refer to a very wide range of “common sense” standards such as to refrain from cheating, plagiarism, falsifying results, vested interests, and so on. Though they may seem “common sense” to most of us, we tend to forget that many of these issues are perceived differently by different cultures, and influenced by pressures which might be applied – internal and external to the institution – to “encourage” researchers to produce favourable results one way or another. It is for these reasons that gaining the approval of the research ethics committee is a fundamental gateway for any research project before it can be seriously undertaken.

# Storing and archiving data

When I was doing my own PhD, I had a filing cabinet with three or four drawers, and even then I had hundreds of photocopies of academic papers stacked in small piles according to theme and relevance to the section that I was writing about next. My raw research data, however, was compactly contained in electronic format in the form of tables and graphs; row after row of numbers on spreadsheets which could be tabulated and correlated in any format that I desired. When I left the department, the files were archived for a few years, and then I suspect they were all dumped when the department moved to another building on another campus.

Now, when I generate research data, it is almost entirely in electronic format, and it is automatically stored in several places. I have my personal space in the memory banks of the university computing system, and this space is automatically backed-up overnight. I also usually back-up to my own cloud-space, so that I can access the data wherever and whenever I want. Usually, I also store data for individual projects on a separate memory stick or portable hard-drive. The digital age means that after two or three clicks, I can be assured that copies of my data are safely held in four or five independent locations. Research students can simultaneously share data with a colleague or supervisor in a different part of the world without even leaving their own desk.

This is only the tip of the iceberg, however, because the production of digital data raises almost as many questions as it provides innovative opportunities. There needs to be an early discussion in the supervisory team, for instance, about not simply which data will be stored, but where will it be stored, for how long, and who will have access to it? This is not simply an issue of security, although security, confidentiality, and appropriate use of the data will certainly figure in the discussion. There is a growing awareness that when public money is used to fund research, there needs to be a transparent return on public interest. Initially this has meant that research results, reports, and journal articles, should be made freely available to the public. This is being extended in the next Research Excellence Framework in the UK to insist that if the journal article is not already published as an open resource, it needs to be added as an open source on the digital repository of the relevant institution. But there’s more.

The argument has been extended to include the research data generated by the public funding, so the datasets themselves are trending to become open and shared property. Whether the data is numbers, interviews, audio recordings, photographs, or other recordable results, the likelihood is that the data being gathered by a researcher today, is probably going to be a shared resource tomorrow. It will be possible for other researchers, in subsequent years, to access your raw data, perhaps combine it with other raw data, and re-analyse, re-interpret, and publish their conclusions. It now begins to matter a great deal more seriously exactly who can gain access to your research data, and for what purposes. As the law currently stands, a bona fide researcher can have access to open datasets for up to ten years after they have been deposited. But here is the catch – if a researcher accesses this data after nine years, the open-access clock is automatically re-set for a further ten years. This ensures the certainty that data which is being collected and digitally stored just now, might be still openly available long after the initial researcher has moved on from that research topic, perhaps changed institutions, changed careers, maybe even passed away. The raw data of open access digital resources is now guaranteed a lifetime longer than the career-span of many individual researchers. So think carefully about what you gather, how you organise and store it, and what your legacy of research data will be!

# Recording data

Firstly, I’m aware that I have broken the first ‘rule’ of blogging, which is to keep the posts short, and keep them coming regularly, but I had a bit of a hiatus due to other interests and demands over the summer. Hopefully, now to get back on track

Starting to record the new data which is being gathered as part of a research project, whether a long-term study like a PhD, or a quick toe-in-the-water project, is the most crucial, but perhaps the subtlest stage of the research. If you gather too little data, the project may flounder even before it gets started; too much data, and a metaphoric mountain of results can be generated by cross-correlation and individual analysis, which can paralyse a project almost as quickly as having no data at all. Then there is the question of what is the “right” data? How will I know it when I see it? In reality, it is as likely to be different for every individual project as the diversity of methods of data gathering. The correct procedure, of course, is to recognise that recording the correct data is integrally dependant on selecting the correct research methodology, and in carefully selecting how the data will be collected, coded, and stored in the future.

One of the most impressive records of research data that I can remember, is from a scientist who was studying birds of prey, and his handwriting in an old notebook recorded what seemed to me to be almost every conceivable factor which might influence nesting success, including several factors that I, personally, would never have begun to consider relevant. He was of course correct, for it is often the correlations with hidden, and often apparently spurious, information which leads to the really stunning breakthroughs in research projects. There are many different ways of the recording research data that you might collect, and there is no one-size-fits-all solution. If you are interviewing people, there is a choice between taking notes, audio recording, or video recording; all these methods have their advantages and disbenefits. Taking notes is less obtrusive, but also can be distracting for the researcher. Audio recording can be done easily with a digital recorder, or a suitable app on a smart-phone, but some people may be more guarded in their responses when they are being recorded, and there is also the problematic issue of what to do with all the data you have gathered. Gathering a huge mass of data can be attractive, but it needs to be proportionate to the scale of the project, because there is little point in generating a mountain of data if 80% is left unanalysed and unused. Great care needs to be taken to strike a balance between collecting a good data-set which provides rich possibilities for future analysis, against de-motivating your participants by presenting them with huge questionnaire or over-long interviews. Similar constraints apply when conducting laboratory experiments, fieldwork, or desk-top studies.

Finally, in addition to having to consider your recording requirements in terms of how you propose to codify and analyse the potential results (there is little point in collecting data so randomly that it cannot be interrogated effectively) there are the issues of long-term storage and access to the data. The research supervisor has a crucial role here, not simply in helping to shape what the research students proposes to gather, or how that might be analysed and interpreted, but in providing the continuity which may extend over several decades and overlap with numerous related research student projects. In an increasingly digital and open educational society, not simply the research results, but also the raw research data is also becoming more open and accessible. It is becoming more possible and more likely that scholars coming after you will read not just your conclusions, but also your original data recording notes, so think carefully about what you collect and how you record it!

# Using Skype for research supervision

Over the past few years my colleagues and I have been experimenting with the use of videoconferencing for conducting tutorial discussions with PhD students. There are several reasons for this. Firstly, in a geographically distributed institution like the University of the Highlands and Islands, we are not all conveniently located in the same building, or even the same part of the country. Both staff and students who are participating in the tutorial might be at widely dispersed locations and may rarely meet face-to-face. It used to be the presumption in most universities that the research student would be based in a room just along the corridor, or somewhere convenient within the department, convenient that is for the main supervisor. With the increasing number of part-time research students and the benefits of communications technology, I would argue that this is no longer necessary, and possibly no longer even desirable.

The advantages of using videoconferencing are several, whether it is the high-definition system which the UHI is available at the UHI, or the quick-and-easy Jabber connections for less formal meetings. The use of Skype and Facetime is also common, and can be extended into non-work activities. Firstly, although it is not always imperative to see the person to whom you are talking, the ability to see facial cues does give an extra quality that is not available in simple telephone conversations. In the same way that co-location in the same room allows speakers to see the body-language of their audience, the video presence enables participants to see their colleagues smile, nod their head in agreement, or simply watch their eyes glaze over! I have found this very useful to observe when people actually realise when I am joking and when I am not!

Secondly, probably the most convenient advantage of vc is the ability to connect people from almost anywhere. A regular meeting between the main supervisor and the research student at a distant location can be joined by another supervisor at a third location. This provides the best opportunities for networked support, regardless of where the expertise is based. Meetings can be a highly structured discussion with a formal agenda, or a quick, ten-minute focus on a specific point of deliberation. The participants can join from home, or work, or even from the field, and the media is sufficiently simple and easy-to-use that even short, ad hoc, meetings to discuss the wording of a single paragraph, can be arranged at the drop of a hat.

Thirdly, most video communications services have the ability to record the meeting. This is probably not going to be very useful on every occasion, but for key presentations, or for intense sessions of very complex discussions, the participants have the advantage of being able to replay the meeting, analyse the dialogue, and take notes at their convenience.

In many institutions, whatever the official rhetoric, the contact time between the research students and the main supervisors can be precious little, not to say sporadic. The ability to video-link with the supervision team at prearranged times, wherever they are in the world, is a great tool to give meaningful and networked support to the research student, and to provide quality time when it is most needed.

# More on getting started

The highly flexible nature of the internet means that websites appear and disappear every day. Thanks to Donald MacLean for reminding me that another currently popular university-managed site containing useful resources for prospective PhD students is http://cloudworks.ac.uk/

This is a freely-available, open-access site, although you will need to register to obtain access. Once you have entered the site, search for “PhD” to find a link to “research skills required by PhD students”. The supporting text has short articles on a wide range of issues such as what is meant by ‘critical thinking’, how to select and justify your research methods, and tips on how to organise and present your work so that other people can appreciate your work.