Introduction

Radiologists are trained to identify three-dimensional structures by their two-dimensional projections on film. Surgeons (and other clinicians) are more comfortable viewing and manipulating real (three-dimensional) objects. Helical computed tomography (CT) and magnetic resonance (MR) imaging efficiently obtain clinically relevant volumetric data sets and are increasingly available. Reconstructing these radiological datasets as three-dimensional surface models improves the radiologist’s ability to communicate key anatomic relationships and enhances the clinician’s ability to appreciate these relationships. In fact, viewing and manipulating sequential radiological image data as three-dimensional models is already proving beneficial in many clinical circumstances[1] including diagnostic vascular imaging, surgical planning for reconstruction of bone defects, and radiotherapy.

In the last decade, the use of three-dimensional medical imaging has expanded rapidly. However, segmenting and rendering these polygonal models has required complex, dedicated equipment within departments of radiology, and special expertise on the part of application users. Consequently, designing patient-specific computer-generated radiological models has been limited to radiologists with a special interest, and remains beyond the scope of most physicians.

Automatically building triangle-based models from radiologic slice bitmaps is an extremely hardware-intensive process. Model creation requires a sophisticated program to load the slice data and to build and render the model in user-relevant time. Today's desktop computers can be programmed to perform these operations, but program execution can take an unreasonable amount of time. Once created, however, complex three-dimensional models can easily be displayed and navigated on today's desktop hardware. In fact, Virtual Reality Modeling Language (VRML) already exists as the standard for transmitting, displaying, and navigating three-dimensional models over the Internet[2]. VRML viewers are even included with the most recent versions of standard Web browsers.

Purpose

We created a simple method for Internet users to construct three-dimensional radiologic models using only a standard Web browser. We use distributed processing over the Internet, making model construction and manipulation potentially available to physicians worldwide without complex software installation or management at the user end. A single application controls both a remote processor for model creation and a local VRML viewer for model navigation. This application will allow physicians with minimal computer knowledge to access three-dimensional visualization techniques and share their results.

Methods

Automatically transforming slice data into a specified VRML model is the critical, hardware intensive step in producing medically relevant three-dimensional models. Our process uses the marching cubes algorithm provided in the Visualization Toolkit[3] (a public domain visualization package) to describe the data as isosurfaces at a specified threshold density level. To automatically get useful surfaces, we first window the data and use a basic median smoothing algorithm to remove aliasing and noise. A triangle decimation process (again, part of the Visualization Toolkit) then shrinks the model size (at a user-defined percentage level) by concatenating nearly coplanar triangles into single, larger triangles.

Our desire for platform independence directed us to the Java (Sun Microsystems, Inc.) programming language[4] as the basis for the interface. There are two main pieces of Java code; the user interface which is sent automatically as a Java applet upon accessing our web page; and the stand-alone server, which runs on a central machine that controls the hardware intensive rendering process. The user interface allows the parameters of the rendering process to be sent; threshold, filenames, decimation level, and cropping. The server runs continuously in the background on the remote machine, listens for requests from remote versions of the user interface, and services the requests.

From the user’s perspective, once the slice data has been transferred to the server, the process of model description and display occurs in three stages. First, the three-dimensional area of interest is specified. Second, the rendering parameters are declared. Finally, the rendering occurs on the server and a VRML file is sent back to the user for exploration. A detailed schematic of the overall process, as well as the data transferred during each step, is shown in figure 1 and is described subsequently.

When a user initially logs into this application, the web server sends the code for the user interface, as a Java applet (see figure 2). First, the user enters which dataset will be used to create the model, which slices within that dataset will be used, and what cropping rectangle to propagate through the selected slices. In this way, the user identifies a box-shaped area of interest. Only data points inside this box are considered during the VRML model creation.

Figure 1

Figure 2. User interface Java applet.

Figure 3. Single slice display with cropping rectangle defined.

Figure 4. Image of a VRML model being explored.

We have also implemented a graphic method for selecting the cropping rectangle. The exact two-dimensional slice image on which to draw the cropping rectangle is chosen by the user in the original user interface. The user also chooses the window width and center for this particular image (this is the standard method of determining how the density data of a medical CT slice is translated into pixel values). The server creates the slice image to the user’s specifications and sends the image file back to the display. The applet then displays this slice image, allowing the user to define a cropping rectangle with the mouse (see figure 3). The applet then automatically fills in the cropping parameters on the interface and propagates this box through all the selected slices. This graphic cropping process can be repeated.

In the second step, the other parameters of the rendering process are set by the user. The value for the threshold density level is critical in that it controls what density transition is converted to isosurfaces by the marching cubes algorithm. The user also sets a decimation level for the final model. At a smaller decimation percentage, the model is more accurate, but also is larger. A larger model takes longer to transfer, and is more unwieldy to explore. However, decimation is a computationally intensive process for the server side, thus more decimation results in a longer rendering period.

Once all the parameters are set, the user may initiate the rendering process. The applet will send the parameters to the server, and instruct the server to begin the rendering job. Once rendering is complete, a VRML model is automatically displayed for exploration (figure 4). The user may then change any of the parameters and start the rendering again, until the desired model is created.

Results

Our algorithms run on a Silicon Graphics Impact (Silicon Graphics, Inc.) utilizing the MIPS R10000 processor. We concentrated initially on using standard, archived computed tomography (CT) data exported from the General Electric Advantage Scanner (512x512 pixel slices, with 16 bits per pixel). Our image parsing software reads the header information and decompresses the image data. The only user inputs required are settings for threshold, decimation ratio, base filename of the slice data, and cropping region. Everything else (through VRML display) is done entirely without user intervention. Using an R5000 O2 (Silicon Graphics, Inc.) or a 200 Megahertz Pentium Processor (Intel), we are able to consistently analyze approximately 30 CT slices and build and display VRML models in under 5 minutes (figure 5). This assures us that a more powerful, public system would be capable of servicing simultaneous requests. Diagnostic quality models have been automatically generated remotely and viewed locally using only a standard installation of Netscape (Netscape Communications Corporation) equipped with the Cosmo Player (Silicon Graphics, Inc.). Refinement of rendering parameters and testing various types of smoothing algorithms have steadily improved results. We are currently exploring optimal parameter solutions for viewing portal vein anatomy and virtual colonoscopy.

Figure 5. A VRML model as it appeared on an R5000 02 less than 5 minutes after initiating segmentation.

Each model is written to the server as a standard VRML file before being sent to the requesting browser. Consequently, only one specific user (who generated the model) knows which VRML file name represents each model. One can then communicate the model file name to colleagues who have access to the server. The colleague can appreciate the model from wherever they are. If one also communicates the parameters used for model construction, the receiving colleague can even reconstruct related models, using his own parameters (including cropping), to generate the ideal image for a particular purpose.

Conclusion

The normally complex and multi-application process of windowing, thresholding, segmenting, decimating and navigating three-dimensional radiologic data models has been encapsulated into an entirely automated, remote process. This encapsulation is expected to greatly reduce the learning curve and iteration time for users, but is also a necessary breakthrough to allow for security and other considerations on the way to making this a public, Web-based application.

Our application in no way supplants the need for reading the two-dimensional slice images by an experienced radiologist and reporting his findings appropriately to colleagues. This is not teleradiology in its traditional sense with all the related legal issues (privacy, licensing, credentials) and questions of image quality[5]. Instead, we provide a method for a wide variety of clinicians, each with an established relationship to their own patients, to benefit from rapid, automated, three-dimensional visualization techniques for diagnostic and pretreatment review in any clinical setting without installation or maintenance of complex architectures.

We believe more powerful systems implementing our software would be of great service to the worldwide medical community. The ideal system would utilize a massive, high-power machine available for rendering jobs from regional clientele. It would provide nearly immediate three-dimensional models of patient data requiring nothing more than a standard desktop personal computer with a VRML-capable Web browser at the user end.

References

[1] M. Vannier and J. Marsh, Three-dimensional Imaging, Surgical Planning, and Image-guided Therapy, Radiologic Clinics of North America 34:3 (1996) 545-563.

[2] N. Hartman and J. Wernecke, The VRML 2.0 Handbook: Building Moving Worlds on the Web. Addison-Wesley Publishing, Reading, Massachusetts, 1996.

[3] W. Schroeder, K. Martin, and B. Lorensen, The Visualization Toolkit: an Object-oriented Approach to 3D Graphics. Prentice Hall PTR, Upper Saddle River, New Jersey, 1996.

[4] G. Cornell and C. Horstman, Core Java. Prentice Hall PTR, Upper Saddle River, New Jersey, 1997.