AR in medicine. The future?

What is augmented reality in medicine?

Augmented reality, or AR, is a relatively new technology in which a computer-generated image is superimposed on the user’s vision of the world.1  To create this augmented reality, hardware such as headsets, smart glasses or mobile devices are used. The difference with virtual reality is that the user keeps a link to the surrounding physical world.1

Augmented reality has many uses in medicine. These include medical training, especially anatomy but also simulation training. Surgeons can use AR to plan surgery, and all physicians can use AR to explain complex situations to patients and their relatives.2


In diagnostics, AR has been used to improve adenoma detection rate. A combination of computer vision algorithms and a large database of colonoscopy polyp images means the endoscopist gets real-time visual assistance. Images are overlaid on the primary monitor they are using or on an adjacent monitor.3


Therapeutics is another area where AR has been extensively used, especially in rehabilitation. The interactive aspect means that patients are encouraged to improve their motor actions.4 For people with severe mobility issues, including the elderly and paralyzed, AR becomes an integrated part of their daily life as part of a home appliances system. AR interacts with brain-computer interfaces to give back patients a degree of autonomy.5

When ultrasound was brought in, a new 2D perception of a 3D space was needed. Anyone who has ever used an ultrasound knows that this involves retraining your way of looking at spaces in what I felt was initially counterintuitive.  Ultrasound-guided biopsy is a minimally invasive procedure for tumour staging. Still, it requires long training not only on a manual technique level but also taking into account the change in perception of space. AR is used to plan the trajectory of the needle and then execute the process. A robot arm with pressure sensors is used, feeding back high-quality information to the operator. The person undertaking this ultrasound-guided biopsy is then able to overcome any needle deflection or target motion.6 


Anyone who has taken a basic or advanced life support course will remember meeting Resusci Annie, the rubber mannequin used to simulate emergency situations. Although a great resource for many years, there was never any doubt that you were dealing with a floppy doll. High fidelity simulation training uses complex mannequins who can breathe, have a variable heartbeat and affect ECG readings which take training to a level. The ultimate challenge is simulation training with a real person, but there you are limited to one hopefully stable pathology, and obviously, you can’t administer medications or electric shocks. When it comes to training in anatomy, there are financial, ethical and supervisory constraints on the use of cadavers.7

You also can’t see inside the body, and this is where AR takes medical training to a whole new level. One setting is airway training, where learning to intubate often means switching between the student and instructor who attempts to explain what they are seeing and how best to proceed with the tube. In surgery, AR laparoscopic training too has been shown to increase trainee skills, especially when combined with physical models.3 This freedom of sight is also a safety aspect.8 In addition, AR means the training can take place in a professional work environment, undertaking real tasks. Depending on the program used, this training can be independent without the need for an instructor to be constantly there.7 Emergency medicine training has already been done remotely using AR as distances can be a real issue in more remote clinical settings.9

There can be some disadvantages. Sometimes trainees find that AR can lead to dizziness or blurred vision, although less than with VR or virtual reality.7 Cost is another consideration, although this may be less important to students and institutions who see the skill gain as non-negotiable.

How soon will AR come into my practice, and how should I prepare?

Google Glass was the forerunner of easy access VR and which some considered being low-level AR. Some of you may have tried out these glasses in a non-clinical setting. Google Glass is a good entry-level AR due to the familiarity of the concept. Many of us already use normal eyeglasses. The first version is now obsolete, but the 2020 revised version has been launched with an increased facility for developers to build their own software.10 Now more than ever, as a practising clinician, if you think of a solution for an everyday frustration, you can approach developers to build it for you. The hands-free aspect in a sterile or semi sterile environment is an attractive proposition for situations where you need access to information but don’t have the staff, such as in primary care. Being able to easily scan patient records without the need to be looking at a computer all the time would in itself make a lot of patients and doctors happy.10  In the same way as AR has helped with polyp identification in real-time, external dermal or other lesions too will be superimposed with AR and the corresponding algorithms and knowledge databank.

However, machines, like humans, are not infallible and knowing where they may fall down leads to using them more safely. Although some authors claim that AR will be trained to see with fidelity and without bias, bias in algorithms is only now starting to rear its ugly head.3  There have been several high profile cases of algorithms misidentifying people of colour in facial recognition programs.11 The algorithm will only ever be as good as the input data, even if the data is extensive in quantity. Humans choose the data which will be used, and we all have our own unrecognized biases. Hidden or unidentified health inequalities are often a direct result of these biases, whether race, age or other.

Physicians may be concerned with privacy issues. In cultures where scribes writing down the notes are usual practice, the idea of someone doing the same thing remotely as you use google glass or another similar device may not be a problem.10 For other clinicians, this may take a bit more getting used to. The developers need to think like a doctor, like all doctors, to overcome resistance. Perhaps some clinicians prefer to have limited options, not all of them. At least at the beginning.

What do patients think about it?

It’s very hard to know what patients think of their doctors using AR. There is a lot of information available projecting on to patients what they should be thinking and how they should see improvements. Yet this may not be the reality. We need to ask them and listen. Specific AR therapies have good outcomes as defined by the study researchers, but you don’t know what you don’t know. Perhaps dizziness may be too much of an issue, or perhaps there are other side effects or worries which have yet to be voiced. As with telemedicine, these reticences can often be overcome once the real underlying worries are identified.12

So what now?

AR is one more technology that will come to the patient interaction. It’s only a matter of time. Like POCUS, point of care ultrasound, there will be fans and detractors. Individual knowledge and training are the keys, as is listening to patients. Even if you don’t like it, your patient may have heard about great outcomes for their specific condition. Or you may be encouraged by the increased safe prescribing options of AR but find that you lose patient engagement, and much as the course of antibiotics is not finished, the AR stays in the box after the first couple of days.

If you’ve had any feedback or have any thoughts on VR or AR from your patients or yourself, I’d love to hear from you. @alice_bbyram on Twitter or email me

1.        Tang, S. L., Kwoh, C. K., Teo, M. Y., Sing, N. W. & Ling, K. V. Augmented reality systems for medical applications: Improving surgical procedures by enhancing the surgeon’s “view” of the patient. IEEE Engineering in Medicine and Biology Magazine 17, 49–58 (1998).

2.        Eckert, M., Volmerg, J. S. & Friedrich, C. M. Augmented Reality in Medicine: Systematic and Bibliographic Review. JMIR mHealth and uHealth 7, (2019).

3.        Mahmud, N., Cohen, J., Tsourides, K. & Berzin, T. M. Computer vision and augmented reality in gastrointestinal endoscopy. Gastroenterology Report 3, 179–184 (2015).

4.        Yeo, S. M. et al. Effectiveness of interactive augmented reality-based telerehabilitation in patients with adhesive capsulitis: protocol for a multi-center randomized controlled trial. BMC Musculoskeletal Disorders 2021 22:1 22, 1–9 (2021).

5.        Park, S., Cha, H. S., Kwon, J., Kim, H. & Im, C. H. Development of an Online Home Appliance Control System Using Augmented Reality and an SSVEP-Based Brain-Computer Interface. 8th International Winter Conference on Brain-Computer Interface, BCI 2020 (2020) doi:10.1109/BCI48061.2020.9061633.

6.        Freschi, C. et al. Ultrasound guided robotic biopsy using augmented reality and human-robot cooperative control. Proceedings of the 31st Annual International Conference of the IEEE Engineering in Medicine and Biology Society: Engineering the Future of Biomedicine, EMBC 2009 5110–5113 (2009) doi:10.1109/IEMBS.2009.5332720.

7.        C, M., Z, Š., A, R. & A, S. The effectiveness of virtual and augmented reality in health sciences and medical anatomy. Anatomical sciences education 10, 549–559 (2017).

8.        D, P. & K, M. Current Perspectives on Augmented Reality in Medical Education: Applications, Affordances and Limitations. Advances in medical education and practice 12, 77–91 (2021).

9.        Munzer, B. W., Khan, M. M., Shipman, B. & Mahajan, P. Augmented Reality in Emergency Medicine: A Scoping Review. Journal of Medical Internet Research 21, (2019).

10.      TriHealth invests in Augmedix Inc.’s Google Glass health care venture – Cincinnati Business Courier.

11.      Raji, I. D. et al. Saving Face: Investigating the ethical concerns of facial recognition auditing. AIES 2020 – Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society 7, 145–151 (2020).

12.      Healthwatch England. Locked out: Digitally excluded people’s experiences of remote GP appointments. (2021).

13.      Liu, Y., Stiles, N. R. B. & Meister, M. Augmented reality powers a cognitive assistant for the blind. eLife 7, (2018).

14.      Kulkov, I., Berggren, B., Hellström, M. & Wikström, K. Navigating uncharted waters: Designing business models for virtual and augmented reality companies in the medical industry. Journal of Engineering and Technology Management 59, 101614 (2021).

Research methods in innovation.  

Types of research – a deep dive into case studies.


Quantitative research uses a statistical interpretation of, among others, observational data using predetermined methods. It is the type of research that is most published by medical journals but is by no means the only one.


Qualitative research is more used in the social sciences but is also relevant in medicine, especially to innovative projects. With qualitative methods, the questions are more open-ended, and data arrives in a less structured format. Interview data, documents, and audio-visual data are all used to bring out themes, patterns, and interpretations. As a clinician, you can immediately see how qualitative research is relevant to medical projects. Patients are more than a list of blood test parameters or diagnoses. This type of data represents not patients, but people with fluctuating disease, positions in society and their families, and a reality that we know is fundamental to understand for any therapeutic intervention to succeed. When innovative projects fail, it is rarely to do with the technology itself, but because these “non-clinical” factors are not taken into account.

One of the research methods used in qualitative research which you may have come across is case studies. In 2006 Flyvbjerg published an interesting paper about case studies and popular misconceptions or even negative attitudes towards them.1. I will go over these misunderstandings because you may find you also had these misconceptions without realising them.

  1. Theoretical knowledge is more valuable than practical knowledge. This refers to how people learn. Yet experts become so by working on many case studies. A fact recognised by Harvard since the 1980s when case study based learning was introduced. Kuhn argues that “a scientific discipline without a large number of thoroughly executed case studies is a discipline without systemic production of exemplars…and a discipline without exemplars is an ineffective one”. Thus, Kuhn concludes that social sciences need more good case studies. I would argue that this also extends to innovation in digital health and medicine generally.
  2. One can not generalise from a single case, the single case study cannot contribute to scientific development. To refute this argument, we need go no further than Galileo’s rejection of Aristotle’s law was based on a conceptual experiment and only later, if at all, on a practical one. Even if he did actually carry out the experiment, it is controversial that he used the leaning tower of Pisa to demonstrate that objects fall at the same speed, independently of their size, as Aristotle had previously argued. No RCTs were carried out to prove this. Case studies can be used to generalise the type of test that Popper called ‘falsification’, a clear-cut test he illustrated with the ‘All swans are white’ example. Popper said that if only one swan were to be black, the proposition would be false and needed further investigation. Flyvbjerg uses this to argue against the misunderstanding that case studies can not be used to generalise by identifying case studies as a method of identifying ‘black swans’ because of the in-depth nature of case studies.
  3. The case study is most useful for the generation of hypotheses – other methods are more suitable for hypotheses testing and theory building. This misunderstanding is based on the previous premise that you can’t generalise. So, if that is invalidated, this misunderstanding too is invalidated. Flyvberg maintains that although useful for generating hypotheses, case studies can also be used for other parts of the research process. It is also true that atypical or extreme cases often reveal more information.
  4. The case study contains a bias towards verification. Understood as a tendency to confirm the researcher’s preconceived notions, the study, therefore, becomes of doubtful scientific value. Bias is something inherent to all researchers, no matter what method they use. Even Charles Darwin developed his own method of recognising this ‘I had . . . during many years followed a golden rule, namely, that whenever a published fact, a new observation or thought came across me, which was opposed to my general results, to make a memorandum of it without fail and at once; for I had found by experience that such facts and thoughts were far more apt to escape from the memory than favorable ones. Owing to this habit, very few objections were raised against my views, which I had not at least noticed and attempted to answer.’ Case studies have their own but different rigour of methods which means researchers using the case study method have been led to change their initial hypotheses. Indeed, in other methods, the subjective bias may be across many cases rather than one without being challenged.
  5. Often difficult to summarise specific cases studies. This final misunderstanding which has been used as a criticism or limitation, is for the fans of case studies a sign that the study has uncovered a particularly rich problematic for which a summary is not desirable. Nietzsche explains, ‘Above all,’ he says about doing science, ‘one should not wish to divest existence of its rich ambiguity’ (emphasis in original).

The choice to use a case study is often because it allows multiple and emergent data collection methods.2

Mixed methods – the best of both worlds!

Mixed methods use both predetermine and emerging methods from a pragmatic point of view. Multiple forms and levels of data are used which include both statistical analysis and text analysis. Critical interpretation across databases is obtained, bringing together the different research traditions to benefit the patient. So, if you have an idea for a research project or your innovation throws up a question that doesn’t fit neatly into a box or method, perhaps it doesn’t have to. Mixed methods are, for me, the best way of looking into the reality of patients as people and, therefore, what makes them tick.

Lessons to be learnt from qualitative research methods and the social sciences.

  1. The importance of categories.

Although we may be aware of obvious cut off points such as age which has led to many trials excluding a large proportion of the population, there are other aspects to the categories you may choose when researching innovation. Applying a category gives power to the person deciding the categories, especially if it is imposed. Choosing to associate with a category may give back the power to the participant, but you have to be clear that this inherently gives legitimacy to certain viewpoints. And silences. Putting someone in a category is an action of power, but so is the ability to remove a person from a category. If you think about yourself, you might not want to be always classed as an iPhone user, or as an iPhone user to all the people you meet. It is a frivolous yet serious example if you start thinking about other categories you are placed in, such as hypertension, marital status, and weekly alcohol intake.

Age is a tightly coupled category, but some categories may be more transient, such as mental status and have real implications on access to certain services or stigma. When you want to capture or even market to innovation users, taking into account the categories you choose and the people you silence is vital.

2) Logics of inquiry

There are generally three logics of inquiry or ways of thinking about the research you wish to undertake or have been undertaken. Again, being aware of these will make you recognise your own biases and find the line of inquiry best suited to your innovation.

An inductive line of inquiry starts bottom-up with the people’s own subjective interpretations. These lead to questions and hypotheses to investigate, which in turn are confirmed or refuted. The end product of an inductive inquiry is a theory. You may well find yourself taking this approach based on your experience with an innovative product and experience.

A deductive approach is top-down, that is existing empirical work gives a set of statements. The researcher then applies the hypotheses and generates a new hypothesis or question.

Finally, the abductive line of inquiry is driven by a social phenomenon. It tries to explain why something is happening now and in what way. Here the researcher looks to find common processes.

How do you know which study is good?

Most medics are used to doing critical appraisals of scientific journal articles. Everyone has their own method, but for those who aren’t used to doing this, the Oxford-based CASP team have a checklist you can work through. If you are interested in hearing how clinicians debate these papers and the questions, try the Resus room papers podcasts—all links are in the podcast notes.

Suppose you are thinking of presenting research or presenting information. In that case, you can assess your own work before going in front of the lions with the PROMPT criteria in which presentation, relevance, objectivity, method, provenance and timeliness are all looked at. The PROMPT criteria are as relevant for researchers as for marketers and innovators coming up with a new idea. If you know the criteria your audience or buyers are using, you can prepare yourself better.

Finally, when you get involved in a discussion about research, recognise when what you see is “ad hominem”. A critique is directed at the person producing the research rather than the work itself, attacking the messenger, not the message. “Ad hominem” attacks are especially relevant as regards COVID research in areas such as vaccinations and face coverings where emotions run high.

What sort of research is needed in innovation?


Research is due diligence by another name, especially in the early stages. Further down the line, it can be seen as auditing or market research systematically. You are probably already doing it but perhaps not with the best tools.

Who by?

Yourself. As just mentioned, you are probably involved in some sort of research, whether you realise it or not. You can also outsource to people who have already published on the matter. They may be interested in collaborating, and at the very least, can direct you to others in the field. A quick search in PubMed can lead you to the people you need. Perhaps, you need to think outside the box and go to social scientists rather than traditional medical researchers. Social scientists may have a research approach more suited to the world of innovation in digital health and other spheres where mass RCTS or randomised controlled trials can’t be carried out. You can find these researchers in the sociology department of any university, and many will already be crossing over with psychology and other health sciences. There is a loss of knowledge when people research in geographically different spaces. You only need to look at the canons of Arabic medicine being reinvented and “rediscovered” in the Middle Ages in Europe. It is still going on when those involved in healthcare don’t look at what is happening in spheres that are not scientific as per their understanding of the concept.


So, when do you need to do the research? Early on is the answer. Or at least it is important that you think about the potential research your treatment or innovation can lead to so that you collect all the data you need, following GDPR, of course. A retrospective study, looking at data collected previously, may not be as prestigious as a RCT. However, as we have seen, if you know the limits of each research method, it can still be valuable.

Another option is if you are aware of the ongoing multi-centre trials, you may find something you can jump on. For example, if you have a point of care blood coagulation device, you may want to look at centres that are taking place in the CRASH-4 trial. On their website, they explain ‘the CRASH-4 trial aims to provide reliable evidence about the effects of early intramuscular TXA on intracranial haemorrhage, disability, death, and dementia in older adults with symptomatic mild head injury.’ Prehospital knowledge of blood coagulation may well be incorporated into later treatment algorithms for mild head injury. Talking to those involved in this type of research gives you a foot in the door to a specific clinician population who have a special interest in haemorrhage.

Take home.

Different research methods have their uses, perhaps more than we have previously thought when it comes to case studies.

Learning from the social sciences and using mixed methods can fill the gap in innovation research and implementation.

Remain critical of the research and yourself. You are already a researcher with bias, whether you realise it or not.

Innovative medical devices on their own are not enough. They need to be validated and integrated into the health system and the lives of patients.


Five Misunderstandings About Case-Study Research. Flyvbjerg B

CASP Critical appraisal checklists.

Resus room podcasts discussing papers.

PROMPT criteria.

1.        Flyvbjerg, B. Five Misunderstandings About Case-Study Research: 12, 219–245 (2016).

2.        Paparini, S. et al. Evaluating complex interventions in context: systematic, meta-narrative review of case study approaches. BMC Medical Research Methodology 2021 21:1 21, 1–22 (2021).

Why your #healthtech pizza can’t have too many toppings.

Have you ever been so exhausted with making decisions at work that you decide you just want pizza for dinner (any pizza, as long as someone else decides the toppings)? This decision fatigue (1) is a very real experience for all types of doctors and health professionals who spend their day taking important decisions with life or death consequences immediately or in the future. There has even been a scale developed to assess how health professionals are affected by this (2).

So when you present your amazing healthtech product with its many multiple options to clinicians, don’t feel offended that their eyes glaze over, or even droop. It’s not a case of reducing your offer of special functions available exclusive to your digital health product. Instead, tailor your product to the needs of the health professional in front of you.

What you really need to do is to know which functions will change their practise, decrease their levels of frustration with IT and set it up for them. Of course, they can do it themselves (this and a few more complicated procedures such as saving lives), but if you do it for them, you get a foot in the door. Leave it to them, and it will be pushed to the bottom of the non-urgent pile, and that is how digital health products end up not being implemented.

You can rail against health professionals pushing back against tech, but the reality is that if it doesn’t work for them, you are going to be the one left on the outside.

1. Linder JA, Doctor JN, Friedberg MW, et al. Time of Day and the Decision to Prescribe Antibiotics. JAMA Intern Med. 2014;174(12):2029–2031. doi:10.1001/jamainternmed.2014.5225 

2. Hickman RL, Pignatiello GA, Tahir S. Evaluation of the Decisional Fatigue Scale Among Surrogate Decision Makers of the Critically Ill. West J Nurs Res. 2018;


Why you need to clinically validate your #healthtech.

Quoted failure rates of #healthtech start-ups are almost as hysterical as the millions said start-ups are said to be receiving. Numbers vary vastly from 44% to 70%. The actual numbers don’t really matter (unless you are one of the investors or workers losing out), the real issue of how to avoid this happening in the first place in #digitalhealth. #Healthtech projects which have clinicians behind them do well both in the private and public sector; they have inbuilt clinical validation from the start. This is why you too should think about doing it. 

So that the #healthtech actually works.

It may seem an obvious point, but many digital health “solutions” fail because they are not in fact a solution. They are a product which is developed by non-healthcare professionals to answer a perceived need. Innovative technology is showcased brilliantly at industry events but then is either rejected or fails when it comes to the medical profession.

Bias in medicine is a dangerous thing, and as clinicians, we are continually being put in our places by patients who don’t conform to expectations. There has been much talk about Babylon’s diagnosing a woman as having anxiety instead of a heart attack, pointed out incidentally on #medtwitter. However, this is just one of many examples of bias which can mean that your non-clinically validate #healthtech not only doesn’t work but also becomes a liability. And as with Babylon, word spreads fast in the medical community. How many #healthtech developers are employing data scientists to look at potentially dangerous biases in their algorithms?

So that doctors support your #healthtech.

Lack of clinical take-up leads to a lot of “doctors will just have to get used to changing their practise whether they like it or not” comments, implying that they are stuck in their ways. This overlooks the fact that doctors, by definition, are lifelong learners, adapting their clinical practice on a daily basis. Every patient you see is a risk-balance assessment of what works best for that patient based on current evidence but also your own professional opinion. Healthcare professionals are your toughest critics because they are the ones who see the #clinicalreality and the aspects which you don’t. No man is an island and no patient is just one disease.

When you diagnose a patient, you do so not just by looking at a set of tests and variables such as heart rate, but by speaking and looking at the patient. The questions often seem random to a layperson, but sometimes the examination is even superfluous. I know I’m not the only person who has gone back into a cubicle to put a stethoscope on for the patient’s benefit as I’d already understood what was going on by the time we’d finished talking. Just how many #digitalhealth people realise that by the time you are ordering the tests, you are often just confirming the diagnosis. When you “treat” a patient, you do so not just following a protocol but based on many other factors.

However, there are many frustrations which we know technology could help with; having access to all the correct patient information, reducing the decision burden by incorporating protocols. So speak to your target clinicians. Now. Often. In their clinical setting. What they will tell you is that they will enthusiastically take on validated and evidence-based #healthtech which answers their needs. In fact, they will probably be able to tell you what you need to do to make your #digitalhealth technology work. Sometimes they have already done it themselves, and you can work with them.

So that patients go to their doctors asking for your #digitalhealth solutions.

And if you speak to the doctors, and nurses, and healthcare assistants, and receptionists, and porters, don’t stop there. Patients, especially chronic patients, have a very clear idea as to what works, what doesn’t work and which of their #digitalhealth needs aren’t being met. There is a whole #wearenotwaiting movement where type 1 diabetes patients have been going faster than the industry at developing openAPS or open artificial pancreas systems and glucose monitoring. After many years of being treated as dangerous mavericks, they are now being incorporated into paediatric diabetes care in major NHS hospitals. Even the fact that they are not FDA approved has not put off parents and doctors using them. That is what “disruptive” in #healthtech really means. Meanwhile, Medtronic and others who provide the “official” solutions, have recognised the fact that it makes more sense to employ directly the #wearenotwaiting developers rather than play catch-up.

Even patients who are not digitally savvy will be quick to tell you why they will or won’t use an app or technology. And often these are for very different reasons to the doctors. Maybe it is because they are more affected by the short-term side effects of a medication whose dose needs to be changed than targets- and they have to be able to access that information quickly. It may be that your amazing frailty support system doesn’t recognise the fact that being part of the #silvereconomy doesn’t mean being bedbound, and that they too want to go places in the world with no internet connection. Patients are whole persons whose disease lives with them once they leave the consulting room, and any treatment, digital or traditional, needs to take that into account.

So that you can expand into the community.

It is fair to say that in an era of influencers, traditional advertising is being rethought to reflect the age-old concept that you are more likely to follow the recommendation of someone you trust that the manufacturer. Doctors, suspicious as they are (!), prefer to hear about new medications and developments in medicine from other doctors. Pharmaceutical companies have long recognised this fact and this is another advantage of clinically validating your product. You speak the language of your target users, and once clinicians are prepared to listen, it can be a useful two-way conversation and is the way you get your #digitalhealth product to a clinical setting.

Patients too ask friends and family for advice. The reason that the instruction to only take medication which has been prescribed for you is precisely because people still take their family member’s medications for something which may or may not be a similar disease. Once you have patients with a vested interest, then others will follow. The way to do that is to listen, speak to and answer their needs.

It’s an exciting time to be in medicine, both as a professional and a patient or carer. It is in everyone’s interest in making sure that the progress in #healthtech works first time round….and keeps on working and being relevant.