Application of Stereo-Imaging Technology to Medical Field

Article information

Abstract

I. Introduction

Diverse kinds of three-dimensional (3D) stereo-imaging devices and related image processing techniques have recently been developed. These stereo-imaging devices can provide more realistic depth perception to the viewer than conventional 2D imaging devices and enable more accurate understanding and analysis of an object [1]. Therefore, these devices have been applied to a variety of fields such as the entertainment industry and the military and investigative fields. Particularly, in the medical field, various medical imaging devices have adopted the stereo-imaging technique, such as the stereo endoscopy and stereo microscopy, and have been developed and utilized clinically to improve surgical accuracy and patient safety. In this article, we describe the basic principles of stereo vision, a brief history of application of the stereoscopic techniques to the medical field, recent trends in stereoscopic medical devices, and related patent applications.

II. Principles of Stereoscopic Vision

2. Principles of the Stereoscopic System

The stereoscopic system consists of two separate visual inputs, which means that two separate lenses are utilized under the same photographic conditions with different viewing angles. There are various types of stereoscopic systems, and among those, the parallel-camera structure and the beam-splitter structure are the most commonly utilized. The parallel-camera structure arranges two cameras side-by-side, which is similar to the arrangement of the human eyes, as shown in Figure 1A. The advantages of this structure are that 1) it is convenient to adjust the setting and match the alignment and 2) there are no differences in light exposure and color between the two images. However, the ability to obtain close-up images is limited because the adjustment of the distance between two axes of the cameras is limited by the sizes of the lenses. Thus, the parallel-camera structure is more suitable for long-distance photographs. The beam-splitter structure arranges two cameras perpendicularly and utilizes a half-silvered mirror that transmits half of the light incident at a 45° angle and simultaneously reflects the remaining light, as shown in Figure 1B. The advantages of this structure are that 1) obtaining close-up images is possible because the distance between two axes of the cameras can be minimized and 2) various types of cameras and lenses can be equipped. However, the disadvantages of the beam-splitter structure are that 1) the quality of the half-silvered mirror can affect the amount of transmitted light and the perception of color, 2) proper alignment is relatively difficult and time consuming, and 3) the use of a lens filter is limited. Therefore, the beam-splitter structure is more suitable for close-up photographs.

Figure 1

Basic structure of the stereoscopic system. (A) Parallel-camera structure, (B) beam-splitter structure.

3. Human Factors in a Stereoscopic System

Human factors associated with 3D stereo-imaging systems have been reported in a number of previous studies [4]. Sources of visual discomfort while staring at the stereoscopic image include 1) excessive binocular disparities (vertical/horizontal/crossed/uncrossed disparity), 2) vergence-accommodation mismatch (conflict), 3) imperfect image separation (interocular crosstalk) [5], and 4) imperfect 3D rendering. Among these factors, in this article, we mainly focus on the vergence-accommodation mismatch and the resultant visual fatigue [6,7]. When a person stares at an object in the real world, the visual lines of the left and right eyes cross at the object and the focal distance and the vergence distances become identical. When staring at an object on a 3D display, however, the point at which the two visual lines cross is not the same as the focal points of the eyes. Consequently, the focal distance and the vergence distance are different from each other (Figure 2). This vergence-accommodation mismatch frequently reduces the viewer's ability to fuse the binocular stimulus and induces sensory conflict and accommodative asthenopia that result in visual fatigue [8,9]. Visual fatigue from viewing stereoscopic images includes a wide range of visual symptoms, including eyestrain, dried mucus or tears around the eyelids, the feeling of pressure in the eyes, pain around the eyes, discomfort when the eyes are open, burning eyes, difficulty in focusing or blurred vision, stiff shoulders, and headaches [10]. Causes of visual fatigue are thought to be as follows: 1) geometrical distortions between two images, 2) differences in electrical characteristics between two images, 3) conflicts between convergence eye movement and the accommodation function, and 4) excessive binocular parallax [11]. Causes 1) and 2) are related to the performance of the electric equipment utilized for stereoscopic images and therefore, can be prevented by improving the quality of the images using high-performance video equipment. However, causes 3) and 4) are related to a basic problem of binocular fusion in the brain, and the detailed physiological mechanism of this fusion is not well understood. The visual fatigue from viewing stereoscopic images is not so serious when the stereoscopic images are displayed within the depth of focus; however, the fatigue becomes serious outside the depth of focus.

Figure 2

Principle of vergence-accommodation mismatch in 3-dimensional (3D) display.

III. Application of Stereo-Imaging to the Medical Field: Historical Perspective

The stereograph was first introduced between the 1830s and 1840s and was based on the photography techniques suggested by Niepce, Daguerre, and Talbot. In 1850, Brewster invented a refracting stereoscope device called the lenticular stereoscope, which consisted of a closed box with one or two openings for the introduction of light into the box and two lenticular lenses, and enabled a viewer to see a 3D image on the floor of the closed box [12] (Figure 3). Soon after Röentgen first discovered the X-ray in 1895, Thomson [13] suggested acquiring and viewing stereoscopic X-ray images, and in 1898, Davidson [14,15] insisted on the advantages of stereoscopic photography and skiagraphy for recording various clinical and pathological appearances. In the early part of the 20th century, several early-stage stereoscopic devices provided stereo pairs of X-ray images to radiologists; however, it was difficult to align the films precisely. Therefore, the radiologists often experienced discomfort and eye strain when using the devices. Nevertheless, the demand for stereoscopic X-ray devices that can provide more realistic images of tissue morphology and anatomy gradually increased. Since the 1970s, several 3D imaging devices, such as X-ray computed tomography (CT) and magnetic resonance imaging (MRI), that can obtain several sliced images of specific body areas with uniform spacing and display the 3D structure of the body by using volume/surface rendering techniques have been developed and applied to various clinical diagnostic fields. Moreover, the application of 3D stereoscopic imaging has recently broadened to various other fields, such as teaching anatomy, digital mammography, diabetic retinopathy, and minimally invasive surgery [16].

Figure 3

Lenticular stereoscope suggested by Brewster [12].

IV. Recent Applications of Stereo-Imaging to the Medical field

1. Stereoscopic Endoscope

An endoscope is a medical instrument used to examine the interior of a hollow organ or a cavity of the body. Conventional endoscopes included just one camera module and therefore, could only provide 2D images of the operative site on a monitor. When performing laparoscopic surgery, surgeons depend on the endoscopic image to perform very sophisticated and time-consuming surgical processes, e.g., cutting, suturing, and clipping. However, there are possibilities of accidents during surgical procedures since the depth information is missing in the 2D endoscopic image. If the endoscopic image is provided to the surgeon in three dimensions, the accuracy of the operation can be improved. Recently, a stereoscopic endoscope has been developed and applied to minimally invasive surgery to improve the surgeon facility and the patient safety (Figure 4). For example, a da Vinci surgical robot system utilizes the stereoscopic endoscope to provide more realistic images of the operative site to the operator (Figure 5). In a previous study that objectively assessed the benefit of stereoscopic vision in robotic surgery for 11 surgeons, the authors evaluated the performance improvement of four surgical tasks-transfer, rope passing, suturing, and V-box-from 2D to 3D endoscopic vision, and the results of the experiments demonstrated that all of the four measurements-time taken for task completion, number of movements made, total distance traveled, and number of errors made-were improved with 3D endoscopic vision [17].

Figure 4

The 3-dimensional stereoscopic endoscope developed by the National Cancer Center, Korea.

Figure 5

Stereoscopic endoscope with two embedded camera modules (A) and a master console screen (B) of the da Vinci surgical robot (C) system in the National Cancer Center, Korea.

3. Current Status of Patents on Stereoscopic Technology

There are several categories for patents on stereoscopic technology. 1) The patents on 3D safety deal with the techniques that can overcome visual fatigue and visual discomfort while watching stereo images. 2) The patents on 3D displays deal with the techniques related to the 3D image construction and the objective evaluation of the quality of the 3D image. 3) The patents on 3D content applications deal with the techniques related to the construction, transmission, and representation of 3D graphic objects that can interact with the viewer more naturally. 4) The patents on 3D medical treatment deal with the techniques related to the construction, compression, recording, transferring, decompression, and displaying of high-quality 3D medical content and controlling the medical device based on the 3D images. Most of the related patent applications related to the use of 3D imaging technology to the medical field have been submitted by several major companies in Japan and the United States, such as Olympus (20%), Intuitive Surgical (19%), and Immersion (12%) [18] (Figure 6).

Figure 6

Worldwide patent application trend in 3D imaging technology for the medical field [18]. Reproduced by permission of the New Convergence Technology Standards Division, Korea Agency for Technology and Standards.

V. Conclusion

In this article, we first explained the basic principles of stereo vision and its related visual conflicts and then briefly reviewed the history of medical applications of stereo-imaging techniques, examples of recent stereoscopic medical devices, and patent application trends of 3D medical imaging devices. The significance of stereoscopic imaging in the medical field increases in accordance with the increasing number of laparoscopic surgeries that depend on an endoscope to obtain a limited vision of the operating site. In addition, stereo-imaging technology plays a key role in accurate and reliable operation of a surgical robot system and diagnosis of the detailed morphology of small biological specimens. In future, the stereo-imaging technology will be combined with the augmented reality to provide more detailed information about the patient to the surgeon during surgery to enhance operational accuracy and patient safety. In addition, microstructure of a cell or a biological specimen will be observed in real-time using the 3D stereo microscope. However, there are several optical problems to overcome to realize these stereoscopic technologies and also, the problems related to the human factor should be reduced. In conclusion, the application of 3D stereoscopic technology to the medical field will help improve the surgical accuracy, reduce operation time, and enhance patient safety. Therefore, it is important to face the challenges associated with this technology and take the initiative in developing more enhanced stereoscopic medical devices.

Article information Continued