Augmented Reality for Minimally Invasive Surgery Overview

Augmented Reality For Minimally Invasive Surgery Overview-PDF Download

  • Date:28 Jul 2020
  • Views:2
  • Downloads:0
  • Pages:29
  • Size:589.83 KB

Share Pdf : Augmented Reality For Minimally Invasive Surgery Overview

Download and Preview : Augmented Reality For Minimally Invasive Surgery Overview


Report CopyRight/DMCA Form For : Augmented Reality For Minimally Invasive Surgery Overview


Transcription:

74 Augmented Reality, research which requires the engagement of physicians engineers and ergonomists The. experience of the ARIS ER1 European consortium is presented providing some valuable. lessons learned about workflow centred design and the importance of field studies The. study of human factors sensorial and cognitive capabilities is also briefly addressed. The last section of the chapter describes four recent advances in the field 1 a new. videobased method for tracking surgical tools which eliminates the extra burden. introduced by existing solutions at a cost of some accuracy loss 2 an advanced. visualization strategy by means of novel collision detection feature for enhancing the safety. and accuracy of radiofrequency ablation of tumours 3 the development of the Endoclamp. positioning system for minimally invasive cardiac procedures which is designed to increase. the safety of the procedure by providing real time visualization and control of catheters. reducing also the need of radiation exposure and 4 the clinical adoption of the Resection. Map a navigation system for liver procedures which efficiently increases the orientation of. the surgeon, 2 AR in surgery technologies concepts and current systems. Augmented reality AR refers to a perception of a physical real world environment whose. elements are merged with or augmented by virtual computer generated stimuli visual or. haptic creating a mixed reality While Paul Milgram and his colleagues Milgram. Kishino 1994 characterize AR as being a fusion of real and virtual data within a real world. environment Azuma and his colleagues base their definition of AR on the attributes of the. AR application Azuma et al 2001 In addition to a mixture of real and virtual information. an AR application has to run in real time and its virtual objects have to be aligned. registered with real world structures Both of these requirements guarantee that the. dynamics of real world environments remain after virtual data has been added In order to. both register the data and fuse virtual and real imagery in real time special devices. implementing a variety techniques are used by today s AR systems. 2 1 Visual AR display technology, To combine visual information in real time one of the following three techniques is used in. AR display devices Optical See Through devices project the current rendering of the virtual. data onto a semi transparent mirror Fig 1a This special mirror allows the user to perceive. the real world through it while at the same time it passes on the virtual content to the eyes. of its user In contrast Video See Through devices capture the real world information with a. video camera Fig 1b Before the final result is presented to the user the captured video. will be blend with the rendering of the virtual content by the device Direct Augmentations. use projectors and the surfaces of the environment to present the virtual information. directly in the 3D real world environment Raskar et al 2001. Display devices can also be distinguished by where they are installed in 3D space Displays. worn on the head Head Mounted Display or HMD usually integrate video or optical see. through technology into a helmet Cakmakci Rolland 2006 Small carried displays or. Handheld Displays are equipped with a video camera and a 2D screen supporting video. 1 European Marie Curie Research Training Network for Augmented Reality in Minimally. Invasive Surgery www ariser info,www intechopen com. Augmented Reality for Minimally Invasive Surgery Overview and Some Recent Advances 75. see through systems Nevertheless both head mounted and handheld displays depend on. additional hardware which is usually either heavy or uncomfortable to wear or in case of. handheld displays suffer from a small physical screen size. MIS technologies include in many cases the use of miniaturised cameras and tools like in. laparoscopy or arthroscopy This makes potentially the Video See Through approach the most. suitable one for these applications Nevertheless percutaneous procedures like in. radiofrequency ablation of liver tumours can benefit from the other AR visual approaches. Fig 1 Techniques for combining visual information a Video See Through b Optical See. 2 2 Registration methods for the alignment of real and virtual realities. The second main requirement for an AR application is a three dimensional registration of its. virtual content relative to the real world objects Therefore AR systems estimate the position. of either the virtual objects relative to the video camera or of both the video camera and the. virtual objects relative to a common coordinate system e g to also allow registered. augmentations using optical see through devices Virtual models are generated by image. segmentation of medical image studies of the patient Massoptier Casciaro 2008. Up to now a number of different technologies have been developed to support the. registration of 3D objects ranging from analyses of the real world environment to. evaluations of intentionally introduced fiducials For example computer vision algorithms. are able to compute the position of a video camera directly from the images it delivers to the. system Davison et al 2007 To be able to compute a 3D position the algorithms have to. select features landmarks shades silhouettes directly out of the images and analyze. them Since feature detection and its subsequent processing is a computationally expensive. and often noisy operation some other technologies can help by adding artificial landmarks. into the 3D environment Teber et al 2009b Those aids called fiducials add a specific. stimulus to the environment which is easier and faster to detect and evaluate A number of. different types of stimuli have already been used as sources for fiducial tracking For. example retro reflective spheres allow for quick and precise visual identification see Fig. 2 Even audio sources Doussis 1993 or magnetic fields natural as well as syntactically. induced have been used The downside of adding fiducials to the environment is the. artificial modification of the environment as well as the preparation required along with the. need for specialized receivers However since the operation room allows preparing the. environment most medical AR application make use of the advantages of intentionally. introduced tracking targets,www intechopen com,76 Augmented Reality.
Registration is one of the current main challenges to solve in soft tissue surgery like. laparoscopy where organs and tissues are deformed cut dissected etc Automatic tracking. and compensation of these changes are required for a stable AR overlay The greater the. differences between the virtual reconstructed models and the real organs of the patient in. the OR the more difficult this challenge is,2 3 AR systems for MIS. AR and Virtual Reality VR applications are capable of not only supporting an intervention. itself but also its preparation and a number of follow up procedures However AR is a. rather complex aggregate technology and research on AR guided treatment is typically. targeted at a single phase or aspect in a surgical procedure Thus embedding AR into a. clinical workflow requires careful design of a software architecture that provides consistent. services for image processing visualization tracking etc throughout a variety of situations. and requirements, The challenge here is to provide useful high performance components that are also. sufficiently flexible to allow re use across different applications Componentization also. allows more careful testing of components in isolation unit tests and makes approval of. software components for clinical evaluation more straight forward. Fig 2 Tracking using retro reflective markers This data is used to register the virtual. counterparts of the liver with two vessel trees and a tumour This AR application supports. the insertion of a needle to ablate the tumour see section 5 2. An AR system for supporting medical applications has generally three main components. Consistent data models The requirements of surgical planning navigation and. simulation go beyond the simple display of volumetric data acquired in previous image. scans Anatomical and pathological structures must be explicitly modeled and. manipulated Physicians demand predictable and reproducible results so all these. representations must be kept in a consistent state throughout the medical workflow. while permitting arbitrary changes on the data, Real time data acquisition In contrast to modalities such as CT or MR which are. normally not acquired in real time AR applications require the management of. streaming input data such as tracking US or video data The handling of such data. requires real time algorithms and also careful synchronization in particular between. simultaneously acquired data from heterogeneous sources. www intechopen com, Augmented Reality for Minimally Invasive Surgery Overview and Some Recent Advances 77. Visualization Compared to conventional screen based navigation AR has elevated. requirements for visualization Techniques must be independent of the viewing mode. and display type and must be able to simultaneously display all kinds of data models. in real time,2 4 Application areas, One of the main strengths of AR in medical applications is its ability to overcome difficulties.
related to hand eye coordination Johansson et al 2001 For example AR displays are able. to present by means of registration of virtual objects within real world environments the. information exactly where the hands have to act Fig 3 shows examples of this concept. Another possibility is to bring the support into the classical 2D screen as done in the. support of needle ablation of tumours see section 5 2. This section reviews the current main areas of AR applications in surgery It presents the. applications characteristics and it discusses the data used in the main components of the AR. Fig 3 Overcoming the problem of hand eye coordination using Augmented Reality a. Typical examination using sonography b The AR display enables the user to see the data. at the same location where his her hands operate,Neurosurgery and orthopedics. AR and computed assisted surgery has already found a place in open and minimally. invasive procedures Teber et al 2009a Nowadays these systems are used in. neurosurgical craniomaxillofacial and orthopedic interventions The use of navigation. technologies reports substantial improvement regarding safety aesthetic and functional. aspects in a range of surgical procedures like dental implantology arthroscopy of the. temporomandibular joint osteotomies distraction osteogenesis image guided biopsies and. removals of foreign bodies Ewers et al 2005 Applications in this category are able to use a. virtual counterpart of hidden real structure which was generated before the intervention. was started Thus these applications are able to make use of high qualitative 3d models. Notice the usage a 3d model which is generated from pre operatively acquired volumetric. scans e g CT data is only possible because no deformation modifies the structure between. the scan of the patient using e g CT and the surgical procedure where the data is. visualized Consequently if rigid structures are in the focus of attention the acquired data. almost perfectly represents the intra operative scenario This is the reason for the extensive. www intechopen com,78 Augmented Reality, use of AR systems in those applications The surgical field is composed or framed by rigid. structures which implies an easier alignment between the virtual reconstruction of the. patient and the operating field the registration step. BrainLab Feldkirchen Germany and Medtronic Navigation Louisville USA demonstrate. that applications in this area are even able to move from a research center to an industrial. application Both companies develop commercialized surgical navigators for rigid. structures,Soft tissue surgery, In soft tissue surgery additional technical challenges exist Current research focuses on. intra operative deformations shifting and topological changes done to the organs. Baumhauer et al 2008 Due to those additional difficulties only some research prototypes. are able to offer a real time navigation environment in an immobilized anatomy displaying. tools location on preoperative or intra operative 3D images Beller et al 2007 Cash et al. 2007 To be able to present virtual counterparts of real world organs the virtual model has. to adapt to the real deformations Consequently besides the three main components of a. medical AR systems the data model used to generate the AR imagery has to be updated. regarding to the current deformation of the organs. Currently most soft tissue navigation systems are focused on liver surgery due to its. clinical relevance Indeed liver cancer is one of the most important causes of death El Serag. Rudolph 2007 As discussed before there is a need of enhancing the prognosis by a. better control of resection margins and risk areas important vessels during the. intervention The guidance of the needle in tumour ablation procedures which can be also. done under a laparoscopic approach is a more controlled problem that is quite addressed. using AR technology Maier Hein et al 2008 These navigation environments can increase. the accuracy of the needle placement St deli et al 2008 For a more detailed review on the. field of navigation in endoscopic soft tissue surgery the reader is referred to Baumhauer et. 5 Augmented Reality for Minimally Invasive Surgery Overview and Some Recent Advances Pablo Lamata 1 2 Wajid Ali 3 Alicia Cano 1 Jordi Cornella 3 Jerome Declerck 2 Ole J Elle 3 Adinda Freudenthal 4 Hugo Furtado 5 Denis Kalkofen 6 Edvard Naerum 3 Eigil Samset 3 Patricia S nchez Gonzalez 1 Francisco M S nchez Margallo 7 Dieter Schmalstieg 6 Mauro Sette 8

Related Books