Research methods in innovation.  

Types of research – a deep dive into case studies.

Quantitative

Quantitative research uses a statistical interpretation of, among others, observational data using predetermined methods. It is the type of research that is most published by medical journals but is by no means the only one.

Qualitative

Qualitative research is more used in the social sciences but is also relevant in medicine, especially to innovative projects. With qualitative methods, the questions are more open-ended, and data arrives in a less structured format. Interview data, documents, and audio-visual data are all used to bring out themes, patterns, and interpretations. As a clinician, you can immediately see how qualitative research is relevant to medical projects. Patients are more than a list of blood test parameters or diagnoses. This type of data represents not patients, but people with fluctuating disease, positions in society and their families, and a reality that we know is fundamental to understand for any therapeutic intervention to succeed. When innovative projects fail, it is rarely to do with the technology itself, but because these “non-clinical” factors are not taken into account.

One of the research methods used in qualitative research which you may have come across is case studies. In 2006 Flyvbjerg published an interesting paper about case studies and popular misconceptions or even negative attitudes towards them.1. I will go over these misunderstandings because you may find you also had these misconceptions without realising them.

  1. Theoretical knowledge is more valuable than practical knowledge. This refers to how people learn. Yet experts become so by working on many case studies. A fact recognised by Harvard since the 1980s when case study based learning was introduced. Kuhn argues that “a scientific discipline without a large number of thoroughly executed case studies is a discipline without systemic production of exemplars…and a discipline without exemplars is an ineffective one”. Thus, Kuhn concludes that social sciences need more good case studies. I would argue that this also extends to innovation in digital health and medicine generally.
  2. One can not generalise from a single case, the single case study cannot contribute to scientific development. To refute this argument, we need go no further than Galileo’s rejection of Aristotle’s law was based on a conceptual experiment and only later, if at all, on a practical one. Even if he did actually carry out the experiment, it is controversial that he used the leaning tower of Pisa to demonstrate that objects fall at the same speed, independently of their size, as Aristotle had previously argued. No RCTs were carried out to prove this. Case studies can be used to generalise the type of test that Popper called ‘falsification’, a clear-cut test he illustrated with the ‘All swans are white’ example. Popper said that if only one swan were to be black, the proposition would be false and needed further investigation. Flyvbjerg uses this to argue against the misunderstanding that case studies can not be used to generalise by identifying case studies as a method of identifying ‘black swans’ because of the in-depth nature of case studies.
  3. The case study is most useful for the generation of hypotheses – other methods are more suitable for hypotheses testing and theory building. This misunderstanding is based on the previous premise that you can’t generalise. So, if that is invalidated, this misunderstanding too is invalidated. Flyvberg maintains that although useful for generating hypotheses, case studies can also be used for other parts of the research process. It is also true that atypical or extreme cases often reveal more information.
  4. The case study contains a bias towards verification. Understood as a tendency to confirm the researcher’s preconceived notions, the study, therefore, becomes of doubtful scientific value. Bias is something inherent to all researchers, no matter what method they use. Even Charles Darwin developed his own method of recognising this ‘I had . . . during many years followed a golden rule, namely, that whenever a published fact, a new observation or thought came across me, which was opposed to my general results, to make a memorandum of it without fail and at once; for I had found by experience that such facts and thoughts were far more apt to escape from the memory than favorable ones. Owing to this habit, very few objections were raised against my views, which I had not at least noticed and attempted to answer.’ Case studies have their own but different rigour of methods which means researchers using the case study method have been led to change their initial hypotheses. Indeed, in other methods, the subjective bias may be across many cases rather than one without being challenged.
  5. Often difficult to summarise specific cases studies. This final misunderstanding which has been used as a criticism or limitation, is for the fans of case studies a sign that the study has uncovered a particularly rich problematic for which a summary is not desirable. Nietzsche explains, ‘Above all,’ he says about doing science, ‘one should not wish to divest existence of its rich ambiguity’ (emphasis in original).

The choice to use a case study is often because it allows multiple and emergent data collection methods.2

Mixed methods – the best of both worlds!

Mixed methods use both predetermine and emerging methods from a pragmatic point of view. Multiple forms and levels of data are used which include both statistical analysis and text analysis. Critical interpretation across databases is obtained, bringing together the different research traditions to benefit the patient. So, if you have an idea for a research project or your innovation throws up a question that doesn’t fit neatly into a box or method, perhaps it doesn’t have to. Mixed methods are, for me, the best way of looking into the reality of patients as people and, therefore, what makes them tick.

Lessons to be learnt from qualitative research methods and the social sciences.

  1. The importance of categories.

Although we may be aware of obvious cut off points such as age which has led to many trials excluding a large proportion of the population, there are other aspects to the categories you may choose when researching innovation. Applying a category gives power to the person deciding the categories, especially if it is imposed. Choosing to associate with a category may give back the power to the participant, but you have to be clear that this inherently gives legitimacy to certain viewpoints. And silences. Putting someone in a category is an action of power, but so is the ability to remove a person from a category. If you think about yourself, you might not want to be always classed as an iPhone user, or as an iPhone user to all the people you meet. It is a frivolous yet serious example if you start thinking about other categories you are placed in, such as hypertension, marital status, and weekly alcohol intake.

Age is a tightly coupled category, but some categories may be more transient, such as mental status and have real implications on access to certain services or stigma. When you want to capture or even market to innovation users, taking into account the categories you choose and the people you silence is vital.

2) Logics of inquiry

There are generally three logics of inquiry or ways of thinking about the research you wish to undertake or have been undertaken. Again, being aware of these will make you recognise your own biases and find the line of inquiry best suited to your innovation.

An inductive line of inquiry starts bottom-up with the people’s own subjective interpretations. These lead to questions and hypotheses to investigate, which in turn are confirmed or refuted. The end product of an inductive inquiry is a theory. You may well find yourself taking this approach based on your experience with an innovative product and experience.

A deductive approach is top-down, that is existing empirical work gives a set of statements. The researcher then applies the hypotheses and generates a new hypothesis or question.

Finally, the abductive line of inquiry is driven by a social phenomenon. It tries to explain why something is happening now and in what way. Here the researcher looks to find common processes.

How do you know which study is good?

Most medics are used to doing critical appraisals of scientific journal articles. Everyone has their own method, but for those who aren’t used to doing this, the Oxford-based CASP team have a checklist you can work through. If you are interested in hearing how clinicians debate these papers and the questions, try the Resus room papers podcasts—all links are in the podcast notes.

Suppose you are thinking of presenting research or presenting information. In that case, you can assess your own work before going in front of the lions with the PROMPT criteria in which presentation, relevance, objectivity, method, provenance and timeliness are all looked at. The PROMPT criteria are as relevant for researchers as for marketers and innovators coming up with a new idea. If you know the criteria your audience or buyers are using, you can prepare yourself better.

Finally, when you get involved in a discussion about research, recognise when what you see is “ad hominem”. A critique is directed at the person producing the research rather than the work itself, attacking the messenger, not the message. “Ad hominem” attacks are especially relevant as regards COVID research in areas such as vaccinations and face coverings where emotions run high.

What sort of research is needed in innovation?

Why?

Research is due diligence by another name, especially in the early stages. Further down the line, it can be seen as auditing or market research systematically. You are probably already doing it but perhaps not with the best tools.

Who by?

Yourself. As just mentioned, you are probably involved in some sort of research, whether you realise it or not. You can also outsource to people who have already published on the matter. They may be interested in collaborating, and at the very least, can direct you to others in the field. A quick search in PubMed can lead you to the people you need. Perhaps, you need to think outside the box and go to social scientists rather than traditional medical researchers. Social scientists may have a research approach more suited to the world of innovation in digital health and other spheres where mass RCTS or randomised controlled trials can’t be carried out. You can find these researchers in the sociology department of any university, and many will already be crossing over with psychology and other health sciences. There is a loss of knowledge when people research in geographically different spaces. You only need to look at the canons of Arabic medicine being reinvented and “rediscovered” in the Middle Ages in Europe. It is still going on when those involved in healthcare don’t look at what is happening in spheres that are not scientific as per their understanding of the concept.

When?

So, when do you need to do the research? Early on is the answer. Or at least it is important that you think about the potential research your treatment or innovation can lead to so that you collect all the data you need, following GDPR, of course. A retrospective study, looking at data collected previously, may not be as prestigious as a RCT. However, as we have seen, if you know the limits of each research method, it can still be valuable.

Another option is if you are aware of the ongoing multi-centre trials, you may find something you can jump on. For example, if you have a point of care blood coagulation device, you may want to look at centres that are taking place in the CRASH-4 trial. On their website, they explain ‘the CRASH-4 trial aims to provide reliable evidence about the effects of early intramuscular TXA on intracranial haemorrhage, disability, death, and dementia in older adults with symptomatic mild head injury.’ Prehospital knowledge of blood coagulation may well be incorporated into later treatment algorithms for mild head injury. Talking to those involved in this type of research gives you a foot in the door to a specific clinician population who have a special interest in haemorrhage.

Take home.

Different research methods have their uses, perhaps more than we have previously thought when it comes to case studies.

Learning from the social sciences and using mixed methods can fill the gap in innovation research and implementation.

Remain critical of the research and yourself. You are already a researcher with bias, whether you realise it or not.

Innovative medical devices on their own are not enough. They need to be validated and integrated into the health system and the lives of patients.

Resources.

Five Misunderstandings About Case-Study Research. Flyvbjerg B

CASP Critical appraisal checklists.

Resus room podcasts discussing papers.

PROMPT criteria.

1.        Flyvbjerg, B. Five Misunderstandings About Case-Study Research: http://dx.doi.org/10.1177/1077800405284363 12, 219–245 (2016).

2.        Paparini, S. et al. Evaluating complex interventions in context: systematic, meta-narrative review of case study approaches. BMC Medical Research Methodology 2021 21:1 21, 1–22 (2021).

GDPR and health data – the questions you need to ask as a doctor.

As a doctor, I have always been very aware of the importance of patient confidentiality. Not only for ethical or legal reasons but also for purely practical purposes. If you don’t have all the information you can’t make the right decisions, and you will only get all the embarrassing information if patients are confident it won’t go any further.

However, from a legal perspective, it is not always that clear, especially when we are talking about health data which now comes from sources other than just the patient. Fitness trackers, for example, give useful information, but how should I store that data?

And if you are looking to buy into some new digital technology, what are the questions you need to ask?

If you are still using paper records or are outside of the EU, this too affects you as all data are covered by articles 2 and 3 of the GDPR.

Historically this has been recognised as a concern as early as 1970 with privacy being covered in the European Convention on Human Rights. Data protection was mentioned in 1981 in the Convention 108 for the Protection of Individuals with regard to Automatic Processing of Personal Data. Therefore the right to data protection is a fundamental right. Now, most people will have heard of the General Data Protection Regulation or GDPR which came into effect in May 2018, if only because of the pop-ups requesting permissions or in the case of certain non-EU websites, refusing access altogether.

For doctors, the essential concepts to understand about data processing or actions on information that can identify their patient are:

  1. Data controller: Person who decides what data is collected, how this data is collected and for which purpose. As a doctor, you or your institution can be a data controller.
  2. Data processor: Person or service who processes the data under the instructions of the controller and as a doctor using #digitaltech this can be the software you are buying storing the data and which needs to be formalised with a contract.
  3. Data subject: Patient or identifiable person.

Article 5 of the GDPR covers data processing, and as a doctor/data controller, you need to be aware that the data you collect should be:

  1. Lawful, fair and transparent.
  2. Limited to purpose – you need to be recording data with a specific, limited and explicit purpose.
  3. Minimised – irrelevant data should not be recorded.
  4. Accurate – doctors are used to keeping treatment changes, for example, and we are all aware of the legal consequences of not keeping legible notes.
  5. Storage limitation – this refers to not keeping the data for longer than required. Health is probably one of the few exceptions where you can argue that the data should be stored for the entire life of a person to give the best care.
  6. Integrity and confidentiality. This refers to the fact that the data must be protected appropriately through technical and organisational means. You need to consider not only loss and damage (accidental or other) but also that it is not accessed inappropriately by different members of staff. This is a core question when being presented with a new medical application or technology for your practice. Larger institutions such as hospitals will have an information security officer, but if you practise in a smaller setting, this responsibility will be yours.

Finally, to process any data, you need to be sure that there are legal grounds for processing the data you have collected. For doctors, the concepts are familiar:

  1. Consent has been given.
  2. It is necessary for a contract to be carried out and specifically, in the health care setting, this includes an agreement to medical treatment either implicitly or explicitly.
  3. You are complying with a legal obligation.
  4. You are protecting the vital interests of a patient.
  5. You are carrying out a task in the public interest or in your capacity as an official authority.
  6. There exists a legitimate interest for processing.

Sensitive data, as health data is, get more privacy protection, and Article 9 covers this specifically. Safeguards used include:

  1. Pseudonymisation: This is removing identifying fields such as name, date of birth and address but in health needs to go even further. A diagnosis of a specific disease and treating hospital plus gender may be enough to identify the patient. With big data and large amounts of patients, it becomes harder to identify individuals, but even there it is important to think about unusual characteristics which may make the patient stand out. Some doctors have fallen foul to this on twitter when making what they thought were generic comments about a type of patient they may have seen during a specific shift. However, at the same time you still have to have the correct data to treat your patient. This means that you need additional information in order to access all the information about your specific patient.
  2. Anonymisation: This means that you strip away all the identifying aspects from the data and can no longer identify the patient. This is a valid technique for research. You can no longer identify the person even if you have the additional information. As mentioned previously, it is very hard to anonymise medical data and there is a chilling report here for al those with any level of data protection responsibility about how supposedly anonymised health data sets were not so anonymous once compared to local newspaper reports. 43% of the individuals were identified.
  3. Encryption: This encoding of the data is very much more a technical aspect.  Most doctors would find it hard to know what questions to ask and then interpret the answers. However, thinking of specific clinical contexts may make the technical team think about uses and deviations which they had not come across.

In general, observing good medical practice will set you on the right road, but the questions come when you want to contract a new software.

  1. What / who is the data processor you use? Are they compliant with GDPR and what sort of guarantees do they offer?
  2. As this is sensitive data, how is it:
    1. Pseudonymised?
    2. Encrypted?
  3. How are you complying with data protection by design and default?

Although most clinicians without any programming or technical knowledge would find it hard to ask specific questions and then understand the answers. However, technicians don’t have the situation-specific understanding of how this data will be used and going through a typical consultation together step by step can help uncover moments when there may be data compliance issues. This is the data protection by default – only the sensitive data needed for the specific process can be processed. For example:

  • How do you lock the screen temporarily while examining a patient when family members may be present?
  • How do you deal with multiple doctors using the same computer?
  • How are blood results transferred between the laboratory and your EHR?
  • Are emails encrypted if you have to do a referral to a colleague?

The company selling you any software should be able to give you clear answers and explanations as to how they are helping you comply with your obligations as a data controller in the clinical setting. Your obligations when contracting a data processor are set out in Article 28, and even if you don’t know the article in detail (!), the people selling you the EHR should.