Jason C. Rubenstein BS, Jonathan C. Silverstein MD, Walter B. Panko PhD

While the use of three-dimensional models has been shown to be useful clinically, the specialized computational equipment and expertise necessary for their construction and use keeps these tools out of reach of most physicians. This paper explores the construction of a Web-based Java application that allows medical radiological models to be built on a remote server and navigated locally on the physician's desktop PC. This paper will also address issues that arose from a public, unrestricted testing of usability over the Internet, such as model size management, easy navigation, processor loading and security. Based on observations and data collected, we suggest what steps are necessary to make a telemedicine application useable in a true clinical setting.

INTRODUCTION

The potential clinical value of reconstructing planar patient data into easily-navigable three-dimensional models has been well-demonstrated [1]. Volume rendering of three-dimensional data is the most accurate, but it has the most expensive hardware requirements for optimal viewing and navigation [2]. An alternative visualization technique, surface modeling, allows for smaller models and can reduce the hardware needed for rapid viewing and navigation; surface models, however, require extreme hardware and software to construct the models from the raw data. The cost, training required, computational power, and specialized software necessary for building and viewing medical models keeps three-dimensional tools out of reach of most physicians. Earlier research theorized that through a Web-based Java application, surface models could be built on a central computer, using extreme hardware, and sent to a physician's desktop PC and navigated from within a Web browser [3]. This distributed computation allows a physician to take advantage of the smaller size of surface models while avoiding the local hardware and software investment and expertise necessary to build surface models. Here we discuss the issues arising from preliminary work to build such a system.

PURPOSE

This server-client application explores the feasibility of using a central server to build complex clinical models and then transmitting them to Internet users for PC navigation. We addressed several issues that arose as a result of moving a software model into a unrestricted, public telemedicine environment: creating an intuitive cross-platform user interface (complicated by limiting the architecture to Java); developing security and stability of the server-side application; more sophisticated logging procedures; and preventing collisions and multi-user tasking considerations to avoid excessive user wait time and possible server overload.

[Figure 1], Java client-side interface under Windows NT and Netscape

METHODS

The basic architecture and shareware tools at the root of the model building code are the same as in earlier versions of the application [3], but the surrounding interface and networking code was altered. The user interface (figure 1) was simplified and reworked to allow for easier user interaction with slower computers and connections. Our original user interface required the user to enter parameters for such values as threshold point (in Hounsfield units), decimation ratio, pixel window size, etc. Because the purpose of this application was to allow physicians with little experience in computer graphics to be able to generate clinically-relevant three-dimensional images, some of the parameters were removed. These parameters were augmented by a pull-down menu with several pre-selected configurations, such as, "bone, large model" or "air, small model". The predetermined parameter settings for each of the menu choices are based on known values of Hounsfield units typically used by radiologists. The single-slice image viewer was also improved. The new image viewer uses frame-buffering to allow for much faster redrawing of the image window. Cropping area (figure 2) was previously defined with two mouse clicks in the rectangle corners, but is now selected by "dragging" a box over the image, which is much more intuitive to users of Microsoft Windows or Apple Macintoshes.

We overhauled the server software to improve security and processor loading, and to allow statistic recording. The server code was overhauled, with special attention to parameter passing. All parameters the server received from the network were then passed through a tight set of criteria to prevent unexpected program behavior from nonstandard messages or parameter values. Also, on the user end, the interface pre-processed all of the parameters to not only to verify their appropriateness, but also to limit certain values to prevent excessively large jobs. Values that fell out of these ranges were automatically adjusted to the nearest acceptable value, along with a warning to the user to re-enter the value with a reminder of the allowed range. A queuing mechanism was added to allow a systematic way of limiting the number of concurrent processes running on the server computer at any given time. The server updated users continually on their current queue position as new jobs were processed, to allow for a better user estimate of queue wait time. The number of allowed concurrent processes was an easily-adjustable server parameter, allowing for regulation based on time of day, or specific server responsibilities, or fine-tuning of maximum tolerable load. Queue size and individual user’s wait time were recorded in the server log file. User info was also recorded in the log file, along with actual processor time (distinct from queue time) for the user’s job to render. Certain parameters of each user’s job were also recorded, along with the job’s execution time, to allow for better understanding of the relationship of the parameters (along with concurrent job loading) to render time.

A method for uploading CT images was developed that met several requirements. First, the public user must be limited within a particular directory to preserve machine security. Second, each user should be able to upload only, and not download any images on the server, to preserve patient information confidentiality. And finally, the interface should not require any special software, or any user-unique passwords (to minimize administration overhead). All of these parameters were met using IRIX ftp services with customized directory structures and security considerations.

Smaller model sizes allow reasonable transfer times and permit easy navigation with limited hardware. However, reducing a medical model simply by reducing the number of triangles created in the three-dimensional model limits the potential clinical application by reducing fidelity. Two other methods for reducing model size are included in our application. First, the user can select which CT slices should be considered during model building. By removing slices that do not help with the diagnostic process in mind, the user can create more focused models. Also, the user can select a "cropping region" (figure 2), or a region of interest on the planar CT images. All data outside the boundaries of this region are discarded from the model. This also reduces data size, and allows the user to create models with increased resolution within the area of interest, without necessarily creating large, unwieldy models.

Large model navigation is being addressed through the current development of a specialized Virtual Reality Modeling Language (VRML) navigator. The ideal navigator should include an external three-dimensional model for orientation and the internal three-dimensional model synchronized to give the user the best possible orientation in three directions as well as access to the relevant anatomy. It would also have the capacity to calculate center-of-passage paths for an anatomical lumen, given a user-defined point in the lumen. This would require a connectivity and filling algorithm, which has been defined by other groups [7]. This function would be most useable in virtual bronchoscopy and virtual colonoscopy. Both of these applications are difficult currently because without this center-of-passage feature, navigation through the lumen manually is extremely tedious within the navigation tools provided by standard web browsers.

These new user-interface and server-side features were tested first internally within our lab, then eventually externally via an unrestricted public test over the Internet.

RESULTS

Successful operation of our application during the public testing showed that the user interface and server/client relationships are viable. Some features of the user-interface were found to be more of a hindrance than helpful with slower computers. The "cropping box" graphical interface was changed from a two-click, define-the-corners method to a more intuitive rectangle-drag interface. However, this seemed to cause problems on slower computers where the dragged rectangle lagged considerably behind the mouse pointer. Also, the half megabyte single-slice image download time over a modem is more of a disturbance than over a direct connection. In the future, different compression schemes for both the image data and the VRML data should be explored to reduce download time for modem users. A compression option should be available to users on slow connections that can be disabled for those with fast connections where compression and decompression time would outweigh any transfer time savings.

The data collected with the detailed logging of the processor performance during public testing will allow for more accurate time estimates on model build times in the future. The results we gathered will allow future versions of the application to predict more accurate time estimates based on the user’s exact chosen parameters. Since a variety of data sets were used during the tests, the timing data recovered should represent the wide spectrum of possible medical data. We noticed a linear relationship between model build time and number of CT slices used in the data set (Figure 3). Had we found a exponential relationship between slice number and processing time, the performance of an application of this design with much larger diagnostic-quality models would have been too poor to use clinically. However, since it behaves with a linear relationship, this shows that once larger model sizes are more easily manipulated on desktop navigation systems, our centralized processor architecture will be well-suited for larger data sets. Decimation ratio proved to be unrelated to build time up until a ratio of around 90% (Figure 4). At this point, the build time increased exponentially in relation to decimation ratio, asymptotically rising to infinite build time as the decimation ratio approached one. This finding pointed to an ideal default decimation ratio of 0.9 which would allow users to reduce model size maximally without significant added processing time. The model sizes chosen for this public demonstration were intentionally limited, to insure delivery of models that could be displayed, navigated, and smoothly rotated on a typical desktop computer. While able to produce anatomically recognizable models with high animation speeds, the data resolution must be significantly higher to be of clinical use. Even if large models were easily navigable, a better VRML navigator needs to be designed that recognizes the lighting needs and spatial orientation clues that are necessary for many medical applications. While the default COSMO VRML browser is fine for most three-dimensional models (figure 5), traversing down, for example, a colon model is almost impossible because of lighting problems and the absence of any orientation landmarks. Without the cues provided by light shading, appreciation of the three-dimensional structure of the colon lumen is almost impossible [7]. Our ongoing construction of a customized VRML navigator with specialized lighting models and navigation controls specific for clinical circumstances may alleviate these problems.

[Figure 5], COSMO VRML Navigator under IRIX (Silicon Graphics)

CONCLUSIONS

REFERENCES

[1] M. Vannier and J. Marsh, Three-dimensional Imaging, Surgical Planning, and Image-guided Therapy, Radiologic Clinics of North America 34:3 (1996) 545-563.

[2] Fenlon HM; Ferrucci JT. 1997. Virtual colonoscopy: what will the issues be? American Journal of Roentgenology, 169:453-458.

[3] Silverstein J, Rubenstein J, Millman A, Panko W. Web-Based Segmentation and Display of 3-Dimensional Radiologic Image Data. In: Westwood JD, Hoffman HM, Stredney D, Weghorst SJ, eds, Proceedings of Medicine Meets Virtual Reality:6; 1998 Jan 28-31; San Diego. Amsterdam: IOS Press; 1998. p.53-9.

[4] N. Hartman and J. Wernecke, The VRML 2.0 Handbook: Building Moving Worlds on the Web. Addison-Wesley Publishing, Reading, Massachusetts, 1996.

[5] W. Schroeder, K. Martin, and B. Lorensen, The Visualization Toolkit: an Object-oriented Approach to 3D Graphics. Prentice Hall PTR, Upper Saddle River, New Jersey, 1996.

[6] G. Cornell and C. Horstman, Core Java. Prentice Hall PTR, Upper Saddle River, New Jersey, 1997.

[7] Abdalmajeid M. Alyassin, Ph.D. and William E. Lorensen. Virtual Endoscopy Software Application on a PC. In: : Westwood JD, Hoffman HM, Stredney D, Weghorst SJ, eds, Proceedings of Medicine Meets Virtual Reality:6; 1998 Jan 28-31; San Diego. Amsterdam: IOS Press; 1998. p.84-9.

[8] Pizer SM; ter Haar Romeny BM. 1991. Digital imaging basics. Journal of Digital Imaging, 4(1);1-20.