Hong Hua

Interests

Research

3D Visualization and Imaging Systems Laboratory (3DVIS Lab) specializes in a wide variety of optical technologies enabling advanced 3D displays, 3D visualization systems and collaborative immersive virtual and augmented environments, and novel imaging systems for medicine and surveillance applications. 3DVIS lab also pursues research projects to gain better understanding of human visual perception and visual artifacts via 3D displays, and to investigate design principles for effective human-computer interface in augmented environments. Examples of systems and technologies we have been researching on include, but not limited to, head-worn displays (aka head-mounted displays or HMD), multi-focal plane fixed-viewpoint volumetric displays, eye movement tracking, eyetracked head-worn displays, head-mounted projection displays, autostereoscopic displays, large-scale heterogeneous display environments for collaboration (SCAPE and Hybrid-SCAPE), multi-touch interface and gesture-based interface, endoscopy, microscopy, and panoramic video systems. 3DVISLab provide students training on a full range of multidisciplinary skills, from creation of novel system concepts and advanced optical system designs (such as freeform optical surfaces, diffractive optical element, aspherics), to system instrumentation, calibration and testing, programming, as well as user-in-the-loop subjective and objective assessment techniques.

Teaching

Optical engineering related coursesdisplay science and technology

Courses

Scholarly Contributions

Journals/Publications

• Huang, H., & Hua, H. (2019). Effects of ray position sampling on the visual responses of 3D light field displays. OPTICS EXPRESS, 27(7), 9343-9360.
• Huang, H., & Hua, H. (2019). Generalized methods and strategies for modeling and optimizing the optics of 3D head-mounted light field displays. OPTICS EXPRESS, 27(18), 25154-25171.
• Lee, S., Hua, H., Nguyen, M., & Hamilton, A. J. (2019). Comparison of six display modes for a multi-resolution foveated laparoscope. SURGICAL ENDOSCOPY AND OTHER INTERVENTIONAL TECHNIQUES, 33(1), 341-351.
• Lovett, M., Biffar, D., Hamilton, A., Katz, J., Lee, S., Hua, H., & Nguyen, M. (2019). EVALUATION OF LEARNING CURVE AND PERIPHERAL AWARENESS USING A NOVEL MULTIRESOLUTION FOVEATED LAPAROSCOPE. 2019 SPRING SIMULATION CONFERENCE (SPRINGSIM).
• Wang, X., & Hua, H. (2019). Digitally switchable multi-focal element for wearable displays. OPTICAL DESIGN CHALLENGE 2019, 11040.
• Wilson, A., & Hua, H. (2019). Design and demonstration of a van-focal optical see-through head-mounted display using freeform Alvarez lenses. OPTICS EXPRESS, 27(11), 15627-15637.
• Xu, M., & Hua, H. (2019). Co-axial depth map sensor with an extended depth range. OPTICAL DESIGN CHALLENGE 2019, 11040.
• Xu, M., & Hua, H. (2019). Methods of optimizing and evaluating geometrical lightguides with microstructure mirrors for augmented reality displays. OPTICS EXPRESS, 27(4), 5523-5543.
• Huang, H., & Hua, H. (2018). High-performance integral-imaging based light field augmented reality display. DIGITAL OPTICS FOR IMMERSIVE DISPLAYS, 10676.
• Huang, H., & Hua, H. (2018). High-performance integral-imaging-based light field augmented reality display using freeform optics. OPTICS EXPRESS, 26(13), 17578-17590.
• Lee, S., Hua, H., Nguyen, M., & Hamilton, A. J. (2018). Comparison of six display modes for a multi-resolution foveated laparoscope. Surgical endoscopy, 33(1), 341-351.
  More info

  To overcome the field of view and ergonomic limitations of standard laparoscopes, we are developing a multi-resolution foveated laparoscope that can simultaneously obtain both wide- and zoomed-in-view images through a single scope. To facilitate the effective access to the dual views of images with different resolution and field coverage acquired by our laparoscope, six different display modes have been developed. Each of the six display modes has inherent advantages and disadvantages. This study compares the six display modes through a human-subject experiment, which was conducted with an emulated laparoscope using a 4K camera.
• Wang, X., Qin, Y. i., Hua, H., Lee, Y., & Wu, S. (2018). Digitally switchable multi-focal lens using freeform optics. OPTICS EXPRESS, 26(8), 11007-11017.
• Wilson, A., & Hua, H. (2018). High-resolution optical see-through vari-focal-plane head-mounted display using freeform Alvarez lenses. DIGITAL OPTICS FOR IMMERSIVE DISPLAYS, 10676.
• Xu, M., & Hua, H. (2018). Ultra-thin Optical Combiner with Microstructure Mirrors in Augmented Reality. DIGITAL OPTICS FOR IMMERSIVE DISPLAYS, 10676.
• Hua, H. (2017). Enabling focus cues in head-mounted displays. Proceedings of the IEEE, 105(5), 20.
• Hua, H., & Huang, H. (2017). Systematic characterization and optimization of 3D light field displays. Optics Express, 25(16), 18.
• Hua, H., & Wilson, A. (2017). Design and prototype of an augmented reality display with per-pixel mutual occlusion capability. Optics Express, 25(24), 12.
• Hua, H., & Xu, M. (2017). High dynamic range head mounted display based on dual-layer spatial modulation. Optics Express, 25(19), 14.
• Huang, H., & Hua, H. (2017). An integral-imaging-based head-mounted light field display using a tunable lens and aperture array. Journal of the Society for Information Display.
• Huang, H., & Hua, H. (2017). Systematic characterization and optimization of 3D light field displays. Optics express, 25(16), 18508-18525.
  More info

  One of the key issues in conventional stereoscopic displays is the well-known vergence-accommodation conflict problem due to the lack of the ability to render correct focus cues for 3D scenes. Recently several light field display methods have been explored to reconstruct a true 3D scene by sampling either the projections of the 3D scene at different depths or the directions of the light rays apparently emitted by the 3D scene and viewed from different eye positions. These methods are potentially capable of rendering correct or nearly correct focus cues and addressing the vergence-accommodation conflict problem. In this paper, we describe a generalized framework to model the image formation process of the existing light-field display methods and present a systematic method to simulate and characterize the retinal image and the accommodation response rendered by a light field display. We further employ this framework to investigate the trade-offs and guidelines for an optimal 3D light field display design. Our method is based on quantitatively evaluating the modulation transfer functions of the perceived retinal image of a light field display by accounting for the ocular factors of the human visual system.
• Wilson, A., & Hua, H. (2017). Design and prototype of an augmented reality display with per-pixel mutual occlusion capability. Optics express, 25(24), 30539-30549.
  More info

  State-of-the-art optical see-through head-mounted displays for augmented reality (AR) applications lack mutual occlusion capability, which refers to the ability to render correct light blocking relationship when merging digital and physical objects, such that the virtual views appear to be ghost-like and lack realistic appearance. In this paper, using off-the-shelf optical components, we present the design and prototype of an AR display which is capable of rendering per-pixel mutual occlusion. Our prototype utilizes a miniature organic light emitting display coupled with a liquid crystal on silicon type spatial light modulator to achieve an occlusion capable AR display offering a 30° diagonal field of view and an angular resolution of 1.24 arcminutes, with an optical performance of > 0.4 contrast over the full field at the Nquist frequency of 24.2 cycles/degree. We experimentally demonstrate a monocular prototype achieving >100:1 dynamic range in well-lighted environments.
• Xu, M., & Hua, H. (2017). High dynamic range head mounted display based on dual-layer spatial modulation. Optics express, 25(19), 23320-23333.
  More info

  The human visual system can perceive a much wider range of contrast than the addressable dynamic range afforded by the state-of-art head mounted displays (HMD). Increasing the dynamic range of HMDs becomes critical especially for augmented reality applications where the dynamic range of outdoor scenes can be as large as 14 orders of magnitude. In this paper, we present the integrated work of the design, implementation, calibration, and image-rendering algorithm of a high dynamic range HMD system. By using a pair of LCoS microdisplays as the spatial light modulators, accompanied with the relay optics to optically overlay the modulation layers, we demonstrate the reconstruction of high dynamic range images with high accuracy.
• Hua, H., & Qin, Y. (2016). Continuously zoom imaging probe for the multi-resolution foveated laparoscope. Biomedical Express.
• Hua, H., & Qin, Y. (2016). Optical design and system engineering of a multi-resolution foveated laparoscope. Applied Optics.
• Hua, H., Wu, R., Benitez, P., Minano, J., & Liang, R. (2016). Design of compact and ultra efficient aspherical lenses for extended Lambertian sources in two-dimensional geometry. Optics Express.
• Lee, S., & Hua, H. (2016). Effects of Configuration of Optical Combiner on Near-Field Depth Perception in Optical See-Through Head-Mounted Displays. IEEE Transactions on Visualization and Computer Graphics, 22(4), 10. doi:10.1109/TVCG.2016.2518138
• Lu, S., & Hua, H. (2016). Structured Illumination Assisted Microdeflectometry with Optical Depth Scanning Capability. Optics Letter, 41(17), 4.
• Markman, A., Shen, X., Hua, H., & Javidi, B. (2016). Augmented reality three-dimensional object visualization and recognition with axially distributed sensing. OPTICS LETTERS, 41(2), 297-300.
• Qin, Y., & Hua, H. (2016). Continuously zoom imaging probe for the multi-resolution foveated laparoscope. Biomedical optics express, 7(4), 1175-82.
  More info

  In modern minimally invasive surgeries (MIS), standard laparoscopes suffer from the tradeoff between the spatial resolution and field of view (FOV). The inability of simultaneously acquiring high-resolution images for accurate operation and wide-angle overviews for situational awareness limits the efficiency and outcome of the MIS. A dual view multi-resolution foveated laparoscope (MRFL) which can simultaneously provide the surgeon with a high-resolution view as well as a wide-angle overview was proposed and demonstrated to have great potential for improving the MIS. Although experiment results demonstrated the high-magnification probe has an adequate magnification for viewing surgical details, the dual-view MRFL is limited to two fixed levels of magnifications. A fine adjustment of the magnification is highly desired for obtaining high resolution images with desired field coverage. In this paper, a high magnification probe with continuous zooming capability without any mechanical moving parts is demonstrated. By taking the advantages of two electrically tunable lenses, one for optical zoom and the other for image focus compensation, the optical magnification of the high-magnification probe varies from 2 × to 3 × compared with that of the wide-angle probe, while the focused object position stays the same as the wide-angle probe. The optical design and the tunable lens analysis are presented, followed by prototype demonstration.
• Qin, Y., & Hua, H. (2016). Optical design and system engineering of a multiresolution foveated laparoscope. Applied optics, 55(11), 3058-68.
  More info

  The trade-off between the spatial resolution and field of view is one major limitation of state-of-the-art laparoscopes. In order to address this limitation, we demonstrated a multiresolution foveated laparoscope (MRFL) which is capable of simultaneously capturing both a wide-angle overview for situational awareness and a high-resolution zoomed-in view for accurate surgical operation. In this paper, we focus on presenting the optical design and system engineering process for developing the MRFL prototype. More specifically, the first-order specifications and properties of the optical system are discussed, followed by a detailed discussion on the optical design strategy and procedures of each subsystem. The optical performance of the final system, including diffraction efficiency, tolerance analysis, stray light and ghost image, is fully analyzed. Finally, the prototype assembly process and the final prototype are demonstrated.
• Qin, Y., Hua, H., & Nguyen, M. (2014). Characterization and in-vivo evaluation of a multi-resolution foveated laparoscope for minimally invasive surgery. BIOMEDICAL OPTICS EXPRESS, 5(8), 2548-2562.
• Wang, Q., Cheng, W. C., Suresh, N., & Hua, H. (2016). Development of the local magnification method for quantitative evaluation of endoscope geometric distortion. Journal of biomedical optics, 21(5), 56003.
  More info

  With improved diagnostic capabilities and complex optical designs, endoscopic technologies are advancing. As one of the several important optical performance characteristics, geometric distortion can negatively affect size estimation and feature identification related diagnosis. Therefore, a quantitative and simple distortion evaluation method is imperative for both the endoscopic industry and the medical device regulatory agent. However, no such method is available yet. While the image correction techniques are rather mature, they heavily depend on computational power to process multidimensional image data based on complex mathematical model, i.e., difficult to understand. Some commonly used distortion evaluation methods, such as the picture height distortion (DPH) or radial distortion (DRAD), are either too simple to accurately describe the distortion or subject to the error of deriving a reference image. We developed the basic local magnification (ML) method to evaluate endoscope distortion. Based on the method, we also developed ways to calculate DPH and DRAD. The method overcomes the aforementioned limitations, has clear physical meaning in the whole field of view, and can facilitate lesion size estimation during diagnosis. Most importantly, the method can facilitate endoscopic technology to market and potentially be adopted in an international endoscope standard.
• Wu, R., & Hua, H. (2016). Direct design of aspherical lenses for extended non-Lambertian sources in three-dimensional rotational geometry. OPTICS EXPRESS, 24(2), 1017-1030.
• Wu, R., Qin, Y., & Hua, H. (2016). Improved illumination of laparoscopes using an aspherical lens array. Biomedical Optical Express, 7(6), 12.
• Cheng, D., Wang, Y., & Hua, H. (2010). Automatic image performance balancing in lens optimization. OPTICS EXPRESS, 18(11), 11574-11588.
• Hua, H. (2015). Design and tolerance of a freeform optical system for an optical see-through multi-focal plane display. Applied Optics.
• Hua, H. (2015). Effects of optical combiner and IPD change for convergence on near-field depth perception in an optical see-through HMD. IEEE Transactions on Visualization and Computer Graphics.
• Li, C., Chen, Q., Hua, H., Mao, C., & Shao, A. (2015). Digital three-dimensional reconstruction based on integral imaging. OPTICAL REVIEW, 22(3), 427-433.
• Lu, S., & Hua, H. (2015). Imaging properties of extended depth of field microscopy through single-shot focus scanning. OPTICS EXPRESS, 23(8), 10714-10731.
• Wu, R., Hua, H., Benitez, P., & Minano, J. C. (2015). Direct design of aspherical lenses for extended non-Lambertian sources in two-dimensional geometry. OPTICS LETTERS, 40(13), 3037-3040.
• Wu, R., Qin, Y., Hua, H., Meuret, Y., Benitez, P., & Minano, J. C. (2015). Prescribed intensity design for extended sources in three-dimensional rotational geometry. OPTICS LETTERS, 40(9), 2130-2133.
• Cheng, D., Wang, Y., Hua, H., & Sasian, J. (2014). Free-form optics enable lightweight head-mounted displays. LASER FOCUS WORLD, 48(4), 67-69.
• Gabbur, P., Hua, H., & Barnard, K. (2014). A fast connected components labeling algorithm and its application to real-time pupil detection. MACHINE VISION AND APPLICATIONS, 21(5), 779-787.
• Hua, H. (2014). A 3D integral imaging optical see-through head-mounted display. Optics Express.
• Hua, H. (2014). A robust camera-based method for optical distortion calibration of head-mounted displays. IEE/OSA Journal of Display Technology.
• Hua, H. (2014). Augmented reality 3D displays with micro integral imaging. IEEE/OSA Journal of Information Display.
• Hua, H. (2014). High-resolution optical see-through multi-focal-plane head-mounted display using freeform optics. IEEE/OSA Journal of Display Technology.
• Hua, H., & Javidi, B. (2014). A 3D integral imaging optical see-through head-mounted display. OPTICS EXPRESS, 22(11), 13484-13491.
• Qin, Y., Hua, H., & Mike Nguyen, . (2014). Multiresolution foveated laparoscope with high resolvability. OPTICS LETTERS, 38(13), 2191-2193.
• Zhang, R., & Hua, H. (2014). Characterizing polarization management in a p-HMPD system. APPLIED OPTICS, 47(4), 512-522.
• Cheng, D., Wang, Y., Hua, H., & Talha, M. M. (2013). Design of an optical see-through head-mounted display with a low f-number and large field of view using a freeform prism. APPLIED OPTICS, 48(14), 2655-2668.
• Gao, C., Lin, Y., & Hua, H. (2013). Optical see-through head-mounted display with occlusion capability. Proceedings of SPIE - The International Society for Optical Engineering, 8735.
  More info

  Abstract: Lack of mutual occlusion capability between computer-rendered and real objects is one of fundamental problems for most existing optical see-through head-mounted displays (OST-HMD). Without the proper occlusion management, the virtual view through an OST-HMD appears "ghost-like", floating in the real world. To address this challenge, we have developed an innovative optical scheme that uniquely combines the eyepiece and see-through relay optics to achieve an occlusion-capable OST-HMD system with a very compelling form factor and high optical performances. The proposed display system was based on emerging freeform optical design technologies and was designed for highly efficient liquid crystal on silicon (LCoS) type spatial light modulator (SLM) and bright Organic LED (OLED) microdisplay. The proposed display technology was capable of working in both indoor and outdoor environments. Our current design offered a 1280×1024 color resolution based on 0.8″ microdisplay and SLM. The MTF values for the majority of the fields at the cutoff frequency of 40lps/mm, which is determined by the pixel size of the microdisplay, are better than 15%. The design achieved a diagonal FOV of 40 degrees, 31.7 degrees horizontally and 25.6 degrees vertically, an exit pupil diameter of 8mm (non-vignetted), and an eye clearance of 18mm. The optics weights about 20 grams per eye. Our proposed occlusion capable OST-HMD system can easily find myriads of applications in various military and commercial sectors such as military training, gaming and entertainment. © 2013 SPIE.
• Hu, X., & Hua, H. (2013). Design and tolerance of a free-form optical system for an optical see-through multi-focal-plane display. APPLIED OPTICS, 54(33), 9990-9999.
• Hua, H., Xinda, H. u., & Gao, C. (2013). A high-resolution optical see-through headmounted display with eyetracking capability. Optics Express, 21(25), 30993-30998.
  More info

  Abstract: A head-mounted display system with fully-integrated eyetracking capability offers multi-fold benefits, not only to fundamental scientific research but also to emerging applications of such technology. A key limitation of the state-of-the-art eyetracked head-mounted display (ETHMD) technology is the lack of compactness and portability. In this paper, we present an innovative design of a high resolution optical see-through ET-HMD system based on freeform optical technology. A prototype system is demonstrated, which offers a goggle-like compact form factor, nonobstructive see-through field of view and true high-definition image resolution for the virtual display. The see-through view, via the combination of a freeform prism and corrector, achieved better than 0.5 arc minute of angular resolution for the central region of approximately 40- degrees to ensure minimal impacts on the see-through vision of an HMD user. © 2013 Optical Society of America.
• Lee, S., & Hua, H. (2013). A robust camera-based method for optical distortion calibration of head-mounted displays. Proceedings - IEEE Virtual Reality, 27-30.
  More info

  Abstract: One of the problems in using head-mounted displays (HMDs) is that the virtual images shown through the HMDs are usually distorted due to their optical distortions. In order to correctly compensate the optical distortions through a pre distortion technique, accurate values of the distortion parameters are required. Although several distortion calibration methods have been developed in prior work, these methods suffer a few critical limitations. In this paper, we proposed a method for robustly estimating the optical distortion parameters of both immersive and (optical) see-through HMDs, without the limitations of existing methods. The proposed method is based on photogrammetry and considers not only the radial distortion but also the tangential distortion. Its effectiveness was evaluated through an experiment conducted with two different types of HMDs, an optical see-through head-mounted projection display and a commercially available non-see-through HMD. According to the experimental results, the proposed method showed significantly lower reprojection error than a previous method proposed by Owen et al. In addition, when both the radial and tangential distortions were considered, the proposed method was significantly more effective than when only the radial distortion was considered. © 2013 IEEE.
• Qin, Y., Hua, H., & Nguyen, M. (2013). Development of a laparoscope with multi-resolution foveation capability for minimally invasive surgery. Progress in Biomedical Optics and Imaging - Proceedings of SPIE, 8573.
  More info

  Abstract: Laparoscope is the essential tool for minimally invasive surgery (MIS) within the abdominal cavity. However, the focal length of a conventional laparoscope is fixed. Therefore, it suffers from the tradeoff between field of view (FOV) and spatial resolution. In order to obtain large optical magnification to see more details, a conventional laparoscope is usually designed with a small working distance, typically less than 50mm. Such a small working distance limits the field of coverage, which causes the situational awareness challenge during the laparoscopic surgery. We developed a multi-resolution foveated laparoscope (MRFL) aiming to address this limitation. The MRFL was designed to support a large working distance range from 80mm to 180mm. It is able to simultaneously provide both wide-angle overview and high-resolution image of the surgical field in real time within a fully integrated system. The high-resolution imaging probe can automatically scan and engage to any subfield of the wide-angle view. During the surgery, MRFL does not need to move; therefore it can reduce the instruments conflicts. The FOV of the wide-angle imaging probe is 80° and that of the high-resolution imaging probe is 26.6°. The maximum resolution is about 45um in the object space at an 80mm working distance, which is about 5 times as good as a conventional laparoscope at a 50mm working distance. The prototype can realize an equivalent 10 million-pixel resolution by using only two HD cameras because of its foveation capability. It saves the bandwidth and improves the frame rate compared to the use of a super resolution camera. It has great potential to aid safety and accuracy of the laparoscopic surgery. © 2013 Copyright SPIE.
• Qin, Y., Hua, H., & Nguyen, M. (2013). Multiresolution foveated laparoscope with high resolvability. Optics Letters, 38(13), 2191-2193.
  More info

  PMID: 23811873;Abstract: A key limitation of the state-of-the-art laparoscopes for minimally invasive surgery is the tradeoff between the field of view and spatial resolution in a single-view camera system. As such, surgical procedures are usually performed at a zoomed-in view, which limits the surgeon's ability to see much outside the immediate focus of interest and causes a situational awareness challenge. We proposed a multiresolution foveated laparoscope (MRFL) aiming to address this limitation. The MRFL is able to simultaneously capture wide-angle overview and high-resolution images in real time; it can scan and engage the high-resolution images to any subregion of the entire surgical field in analogy to the fovea of human eye. The MRFL is able to render equivalently 10 million pixel resolution with a low data bandwidth requirement. The system has a large working distance (WD) from 80 to 180 mm. The spatial resolvability is about 45 μm in the object space at an 80 mm WD, while the resolvability of a conventional laparoscopeis about 250 μm at a typically 50 mm surgical distance. © 2013 Optical Society of America.
• Wang, J., Xiao, X., Hua, H., & Javidi, B. (2013). Augmented Reality 3D Displays With Micro Integral Imaging. JOURNAL OF DISPLAY TECHNOLOGY, 11(11), 889-893.
• Wang, Q., Cheng, D., Wang, Y., Hua, H., & Jin, G. (2013). Design, tolerance, and fabrication of an optical see-through head-mounted display with free-form surface elements. Applied Optics, 52(7), C88-C99.
  More info

  PMID: 23458822;Abstract: Free-form surfaces (FFSs) provide more freedom to design an optical system with fewer elements and hence to reduce the size and weight of the overall system than rotationally symmetric optical surfaces. In this paper, an optical see-through (OST), head-mounted display (HMD) consisting of a free-form, wedgeshaped prism and a free-form lens is designed and fabricated through the injection molding method. The free-form prism for the projection system is designed with a field-of-view (FOV) of 36°; the free-form lens is cemented to the prism for the see-through system to achieve a FOVof 50°. The free-form prism and lens are expanded at the edge area during the design stage in order to reduce the effects of surface deformation in the working area in molding fabrication process and to improve ergonomic fit with the head of a user. The tolerance analyzes considering the mold design for the free-form optical systems are carried out using the Monte Carlo method. The FFS optical elements are successfully fabricated and the system performance is carefully examined. The results indicate that the performance of the OST-HMD is sufficient for both entertainment and scientific applications. © 2013 Optical Society of America.
• Xinda, H. u., & Hua, H. (2013). An optical see-through multi-focal-plane stereoscopic display prototype enabling nearly-correct focus cues. Proceedings of SPIE - The International Society for Optical Engineering, 8648.
  More info

  Abstract: Conventional stereoscopic displays render 3D scenes from a single pair of 2D images at a fixed distance to the viewer. Therefore conventional stereoscopic displays force an unnatural decoupling of the accommodation and convergence cues, which may contribute to various visual artifacts and have significant effects on depth perception accuracy. In this paper, we present the design and implementation of an optical see-through depth-fused multi-focal-plane binocular stereoscopic display. The prototype is capable of rendering nearly-correct focus cues for a large volume of 3D objects extending into a depth range from 0 to 3 diopters and a diagonal field of view of 40 degrees at a flicker-free speed. With the optimized optical system including a freeform prism eyepiece and a see-through compensator, the prototype system demonstrates high image quality while having minimal degradation to the see-through view. © 2013 SPIE-IS&T.
• Gao, C., Lin, Y., & Hua, H. (2012). Occlusion capable optical see-through head-mounted display using freeform optics. ISMAR 2012 - 11th IEEE International Symposium on Mixed and Augmented Reality 2012, Science and Technology Papers, 281-282.
  More info

  Abstract: Most state-of-the-art optical see-through head-mounted display (OST-HMD) lacks mutual occlusion capability between computer-rendered and real objects so that the virtual view through an OST-HMD appears "ghost-like", floating in the real world. In this paper, we demonstrated a light-weight, compact OST-HMD with mutual occlusion capability by exploring a highly innovative optical approach based on emerging freeform optical design and fabrication technologies. Our approach enabled us to achieve an occlusion-capable OST-HMD system with a very compelling form factor and high optical performance. The proposed display technology is designed for highly efficient liquid crystal on silicon (LCoS) type spatial light modulator (SLM) and bright Organic LED (OLED) microdisplay, which is capable of working in both indoor and outdoor environments. Our current design offered a 1280x1024 color resolution with a field of view (FOV) of 40 degrees and lightweight optics about 30 grams per eye. © 2012 IEEE.
• Hua, H., & Gao, C. (2012). A compact eyetracked optical see-through head-mounted display. Proceedings of SPIE - The International Society for Optical Engineering, 8288.
  More info

  Abstract: An eye-tracked head-mounted display (ET-HMD) system is able to display virtual images as a classical HMD does, while additionally tracking the gaze direction of the user. There is ample evidence that a fully-integrated ETHMD system offers multi-fold benefits, not only to fundamental scientific research but also to emerging applications of such technology. For instance eyetracking capability in HMDs adds a very valuable tool and objective metric for scientists to quantitatively assess user interaction with 3D environments and investigate the effectiveness of various 3D visualization technologies for various specific tasks including training, education, and augmented cognition tasks. In this paper, we present an innovative optical approach to the design of an optical see-through ET-HMD system based on freeform optical technology and an innovative optical scheme that uniquely combines the display optics with the eye imaging optics. A preliminary design of the described ET-HMD system will be presented. © 2012 Copyright Society of Photo-Optical Instrumentation Engineers (SPIE).
• Hua, H., Brown, L. D., Zhang, R., & Furht, B. (2012). Head-Mounted Projection Display Technology and Applications. HANDBOOK OF AUGMENTED REALITY, 123-155.
• Qin, Y., Zheng, Z., & Hua, H. (2012). Multi-resolution foveated laparoscope. Frontiers in Optics, FIO 2012.
  More info

  Abstract: A multi-resolution foveated laparoscope is proposed, which could simultaneously provide a wide-angle "stadium view" and a high-resolution sub-field. The FOV-resolution tradeoff in traditional optical systems was addressed by this "foveal" imaging system. © OSA 2012.
• Xinda, H. u., & Hua, H. (2012). Design of an optical see-through multi-focal-plane stereoscopic 3D display using freeform prisms. Frontiers in Optics, FIO 2012.
  More info

  Abstract: We presented the design of an optical see-through multi-focal-plane 3D display using wedge-shaped freeform eyepiece and compensator, which is capable of rendering near-accurate focus cues for a 3-diopter depth range with high image quality. © OSA 2012.
• Cheng, D., Wang, Y., Hua, H., & Sasian, J. (2011). Design of a wide-angle, lightweight head-mounted display using free-form optics tiling. Optics Letters, 36(11), 2098-2100.
  More info

  PMID: 21633461;Abstract: We present a concept of a wide-angle, lightweight, optical see-through head-mounted display (HMD) using free-form optics tiling. Free-form optics tiling can potentially address several critical problems in existing tiled HMD designs that use rotationallysymmetric optics.The optical design of our tiled optical see-through HMDachieves a field of view (FOV) of 56° × 45° and an angular resolution of 3:2 arcmin with two display channels. We demonstrate a proof-of-concept prototype and present some of its manufacturing details. The FOV can be further enlarged by tiling more display channels together at their bottom and side surfaces. © 2011 Optical Society of America.
• Hua, H., Lee, S., & Hua, H. -. (2011). Effects of Viewing Conditions and Rotation Methods in a Collaborative Tabletop AR Environment. IEEE transactions on visualization and computer graphics.
  More info

  We investigate the effects of viewing conditions and rotation methods on different types of collaborative tasks in a two-user co-located tabletop augmented reality (AR) environment. The viewing condition means how the manipulation of a tabletop world by one user is shown in the other users' views and the rotation method means what type of input devices is used to rotate the tabletop world for alternative orientations. Our experiment considered two viewing conditions (consistent view and inconsistent view), two rotation methods (direct turn and indirect turn), and two task types (synchronous and referring-strong type, and asynchronous and orientation-strong type). A 3D display environment called "Stereoscopic Collaboration in Augmented and Projective Environments (SCAPE)" was utilized as a test environment. According to the results, the viewing conditions had significant effects on several objective and subjective measurements. On task completion time, their effect for the synchronous and referring-strong type of task was opposite to that for the asynchronous and orientation-strong type of task. On the other hand, the rotation methods had significant effects only on the accumulated turn angle (for both task types) and the number of negotiation phrases (only in the inconsistent viewing condition for the asynchronous and orientation-strong type of task).
• Hua, H., Liu, S., & Hua, H. -. (2011). Extended depth-of-field microscopic imaging with a variable focus microscope objective. Optics express, 19(1).
  More info

  Increasing the depth-of-field (DOF) while maintaining high resolution imaging has been a classical challenge for conventional microscopes. Extended DOF (EDOF) is especially essential for imaging thick specimens. We present a microscope capable of capturing EDOF images in a single shot. A volumetric optical sampling method is applied by rapidly scanning the focus of a vari-focal microscope objective throughout the extended depths of a thick specimen within the duration of a single detector exposure. An EDOF image is reconstructed by deconvolving the captured image with the response function of the system. Design of a vari-focal objective and algorithms for restoring EDOF images are presented. Proof-of-concept experimental results demonstrate significantly extended DOF compared to the conventional microscope counterparts.
• Hua, H., Pansing, C. W., & Rolland, J. P. (2011). Modeling of an eye-imaging system for optimizing illumination schemes in an eye-tracked head-mounted display. APPLIED OPTICS, 46(31), 7757-7770.
• Lee, S., & Hua, H. (2011). Effects of viewing conditions and rotation methods in a collaborative tabletop AR environment. IEEE Transactions on Visualization and Computer Graphics, 17(9), 1245-1258.
  More info

  Abstract: We investigate the effects of viewing conditions and rotation methods on different types of collaborative tasks in a two-user colocated tabletop augmented reality (AR) environment. The viewing condition means how the manipulation of a tabletop world by one user is shown in the other users' views and the rotation method means what type of input devices is used to rotate the tabletop world for alternative orientations. Our experiment considered two viewing conditions (consistent view and inconsistent view), two rotation methods (direct turn and indirect turn), and two task types (synchronous and referring-strong type, and asynchronous and orientation-strong type). A 3D display environment called Stereoscopic Collaboration in Augmented and Projective Environments (SCAPE) was utilized as a test environment. According to the results, the viewing conditions had significant effects on several objective and subjective measurements. On task completion time, their effect for the synchronous and referring-strong type of task was opposite to that for the asynchronous and orientation-strong type of task. On the other hand, the rotation methods had significant effects only on the accumulated turn angle (for both task types) and the number of negotiation phrases (only in the inconsistent viewing condition for the asynchronous and orientation-strong type of task). © 2011 IEEE.
• Liu, S., & Hua, H. (2011). Extended depth-of-field microscopic imaging with a variable focus microscope objective. OPTICS EXPRESS, 19(1), 353-362.
• Liu, S., & Hua, H. (2011). Extended depth-of-field microscopic imaging with a variable focus microscope objective. Optics Express, 19(1), 353-362.
  More info

  PMID: 21263574;Abstract: Increasing the depth-of-field (DOF) while maintaining high resolution imaging has been a classical challenge for conventional microscopes. Extended DOF (EDOF) is especially essential for imaging thick specimens. We present a microscope capable of capturing EDOF images in a single shot. A volumetric optical sampling method is applied by rapidly scanning the focus of a vari-focal microscope objective throughout the extended depths of a thick specimen within the duration of a single detector exposure. An EDOF image is reconstructed by deconvolving the captured image with the response function of the system. Design of a varifocal objective and algorithms for restoring EDOF images are presented. Proof-of-concept experimental results demonstrate significantly extended DOF compared to the conventional microscope counterparts. © 2010 Optical Society of America.
• Liu, S., & Hua, H. (2011). Time-multiplexed dual-focal plane head-mounted display with a liquid lens. OPTICS LETTERS, 34(11), 1642-1644.
• Xinda, H. u., & Hua, H. (2011). 48.1: Distinguished student paper: A depth-fused multi-focal-plane display prototype enabling focus cues in stereoscopic displays. Digest of Technical Papers - SID International Symposium, 42 1, 691-694.
  More info

  Abstract: Lack of focus cues in stereoscopic displays may have significant effects on depth perception accuracy and visual fatigue. In this paper, we present a depth-fused six-focal plane display prototype, capable of rendering nearly-accurate focus cues for a depth range of 3 diopters with high image quality at flicker-free speed. © 2011 SID.
• Xinda, H. u., & Hua, H. (2011). Distinguished student paper: A depth-fused multi-focal-plane display prototype enabling focus cues in stereoscopic displays. 49th Annual SID Symposium, Seminar, and Exhibition 2011, Display Week 2011, 2, 691-694.
  More info

  Abstract: Lack of focus cues in stereoscopic displays may have significant effects on depth perception accuracy and visual fatigue. In this paper, we present a depth-fused six-focal plane display prototype, capable of rendering nearly-accurate focus cues for a depth range of 3 diopters with high image quality at flicker-free speed.
• Brown, L. D., & Hua, H. (2010). An evaluation of physical affordances in augmented virtual environments: Dataset grounding and magic lens. Proceedings - IEEE Virtual Reality, 23-26.
  More info

  Abstract: In this paper, we quantify the effects of two physical affordances on user interaction in an augmented virtual environment: the grounding of datasets on a workbench and the augmentation of a Magic Lens interface tool. We investigated the effects of these affordances on subjects' performance and behavior in an information gathering task. Our results indicated that grounding had a significant main effect and that there was significant interaction between grounding and interface factors. Specifically, subjects tended to perform better with some level of affordance than none at all, although performance with grounded datasets decreased in the presence of augmented Magic Lens. Further, both affordance factors influenced behavior by reducing head mobility. ©2010 IEEE.
• Brown, L. D., & Hua, H. (2010). Magic lenses for augmented virtual environments. IEEE COMPUTER GRAPHICS AND APPLICATIONS, 26(4), 64-73.
• Cheng, D., Wang, Y., & Hua, H. (2010). Automatic image performance balancing in lens optimization. Optics Express, 18(11), 11574-11588.
  More info

  PMID: 20589018;Abstract: In the final stage of lens design, it is usually a critical step to balance the optical performance of a lens system across the sampled fields, which is achieved by adjusting the weights to these fields. Because the current optical design software packages use fixed weights in the optimization process, the task of weight adjustment is left to the optical designer, who has to change the weights manually after each optimization trail. However, this process may take a very long time to finish, especially when many fields of the lens system are sampled, and the results are subjectively affected by the designer's design experience. In this paper, we propose an automatic performance balancing method. An automatic outer loop is added in the optimization process. The weight for each sampled field and azimuth is calculated appropriately according to the actual performance of the current design and the system requirements, and it is applied to the corresponding field and azimuth automatically in the next optimization trial. The method is successfully implemented in CODE V, and design examples show that it is very effective. ©2010 Optical Society of America.
• Cheng, D., Wang, Y., & Hua, H. (2010). Free form optical system design with differential equations. Proceedings of SPIE - The International Society for Optical Engineering, 7849.
  More info

  Abstract: The general Wassermann-Wolf differential equations were derived to design an off-axis free form surface (FFS) prism head mounted display (HMD) system. A FFS prism HMD system with 20° degree, 8mm exit pupil and 15mm-effective focal length was designed and the image qualities were analyzed. © 2010 Copyright SPIE - The International Society for Optical Engineering.
• Cheng, D., Wang, Y., & Hua, H. (2010). Large field-of-view and high resolution free-form head-mounted display. Proceedings of SPIE - The International Society for Optical Engineering, 7652.
  More info

  Abstract: It is challenging to design a large field of view (FOV) and low F-number optical system, especially an eyepiece for head-mounted displays (HMD), which also demands compactness in size, lightness in weight and a large exit pupil distance. In this paper, we describe a novel design of a wide FOV free-form eyepiece by tiling multiple wedge-shaped free-form prisms. We compared our tiling method using the novel free-form prism eyepieces with a traditional tiling approach and the drawbacks of the traditional tiling method are analyzed. We further present a successive approximation method for designing a freeform surface and compare the accuracy of several surface conversion methods. An example design was also demonstrated. © 2010 Copyright SPIE - The International Society for Optical Engineering.
• Gao, C., Hua, H., & Ahuja, N. (2010). A hemispherical imaging camera. Computer Vision and Image Understanding, 114(2), 168-178.
  More info

  Abstract: Most of the existing panoramic cameras have cylindrical field of view (FOV) which has a 360° width in azimuth, but a limited height in elevation. However, a hemispherical (or near spherical) FOV is highly desirable in many applications such as robot navigation and surveillance. This paper presents a planar-mirror-based panoramic camera (HEM) which is capable of acquiring nearly hemispherical FOV panoramic images at high and substantially uniform resolution, from a single viewpoint, at video rates. The camera consists of multiple imaging sensors and a hexagonal prism made of planar mirror faces. The sensors are positioned in such a way that they image different parts of the scene from the same virtual viewpoint, either directly or after reflections off the prism. A panoramic image is constructed by concatenating the images taken by different sensors. The proposed system is designed such that the resolution across entire FOV has the same level of uniformity as delivered by a conventional, non-panoramic camera over a small FOV, and the sensor area utilization is maximized. We also present a novel camera placement technique that helps co-locate the optic points of all seven sensors at a single viewpoint, and a method to compensate for artifacts near mirror boundaries caused by finite size of lens aperture. An implementation of the proposed system as well as resulting hemispherical panoramic images are included. © 2009 Elsevier Inc. All rights reserved.
• Hua, H., & Gao, C. (2010). A systematic framework for on-line calibration of a head-mounted projection display for augmented-reality systems. JOURNAL OF THE SOCIETY FOR INFORMATION DISPLAY, 15(11), 905-913.
• Hua, H., & Liu, S. (2010). Depth-fused multi-focal plane displays enable accurate depth perception. Proceedings of SPIE - The International Society for Optical Engineering, 7849.
  More info

  Abstract: Many different approaches to three-dimensional (3-D) displays have been explored, most of which are considered to be stereoscopic-type. The stereoscopic-type displays create depth perception by presenting two perspective images, one for each eye, of a 3D scene from two slightly different viewing positions. They have been the dominant technology adopted for many applications, spanning the fields of flight simulation, scientific visualization, medicine, engineering design, education and training, and entertainment systems. Existing stereoscopic displays, however, lack the ability to produce accurate focus cues, which have been suggested to contribute to various visual artifacts such as visual fatigue. This paper will review some recent work on vari- and multi-focal plane display technologies that are capable of rendering nearly correct focus cues for 3D objects and these technologies have great promise of enabling more accurate depth perception for 3D tasks. © 2010 Copyright SPIE - The International Society for Optical Engineering.
• Hua, H., & Liu, S. (2010). Dual-sensor foveated imaging system. APPLIED OPTICS, 47(3), 317-327.
• Hua, H., Cheng, D., Wang, Y., & Liu, S. (2010). Near-eye displays: State-of-the-art and emerging technologies. Proceedings of SPIE - The International Society for Optical Engineering, 7690.
  More info

  Abstract: This paper will start with a brief review of the recent advancements in near-eye displays, then focus on the development and results of two emerging technologies aiming to address two critical issues related to near-eye displays: (a) a freeform optical technology promising near-eye displays with an ultimately compact form factor, close to a pair of eyeglasses rather than a traditional helmet style; and (b) a vari- and multi-focal technology promising more accurate rendering of depth cues than conventional stereoscopic displays. © 2010 SPIE.
• Hua, H., Gao, C., & Ahuja, N. (2010). Calibration of an HMPD-based augmented reality system. IEEE TRANSACTIONS ON SYSTEMS MAN AND CYBERNETICS PART A-SYSTEMS AND HUMANS, 37(3), 416-430.
• Hua, H., Liu, S., & Hua, H. -. (2010). A systematic method for designing depth-fused multi-focal plane three-dimensional displays. Optics express, 18(11).
  More info

  Lack of accurate focus cues in conventional stereoscopic displays has potentially significant effects on depth perception accuracy and visual fatigue. Recently several multi-focal plane display prototypes have been demonstrated with the promise of improving the accuracy of focus cue rendering in stereoscopic displays. In this paper, we present a systematic method to address two fundamental issues in designing a multi-focal plane display: (1) the appropriate dioptric spacing between adjacent focal planes; and (2) the depth-weighted fusing function to render a continuous three-dimensional (3-D) volume using a sparse number of focal planes placed in the space. By taking account of both ocular factors of the human visual system (HVS) and display factors of a multi-focal plane system, we determine that an appropriate spacing between two adjacent focal planes should be ~0.6 diopter (D) while a smaller spacing may be necessary for further improving retinal image quality. We further develop a set of nonlinear depth-weighted fusing function with the promise of balancing perceptual continuity of a 3-D scene and retinal image quality. Our method was based on quantitative evaluation of the modulation transfer functions (MTF) of depth-fused images formed on retina.
• Lee, S., & Hua, H. (2010). Effects of Viewing Conditions and Rotation Methods in a Collaborative Tabletop AR Environment. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, 17(9), 1245-1258.
• Lee, S., & Hua, H. (2010). Effects of viewing conditions and rotation methods in a collaborative tabletop AR environment. Proceedings - IEEE Virtual Reality, 163-170.
  More info

  Abstract: In this paper, we investigate the effects of viewing conditions and rotation methods on different types of collaborative tasks in a tabletop AR environment in which two users are co-located. The viewing condition means how the manipulation of a tabletop world by one user is shown in the other users' view and the rotation method means what type of input devices is used to rotate the tabletop world for alternative orientations. Our experiment considered two different viewing conditions - consistent view and inconsistent view and two different rotation methods - direct turn and indirect turn. In the experiment, a 3D display environment called "Stereoscopic Collaboration in Augmented and Projective Environments (SCAPE)" was utilized as a test environment, and two tasks were considered: Lego-like block building task and text label selection task. The former was designed for synchronous and referring-strong type, and the latter was designed for asynchronous and orientation-strong type. As dependent variables, various objective and subjective measurements including task completion time, quality of task result, turn angle, and questionnaire were measured. According to the results, the viewing conditions had significant effects on several objective and subjective measurements. On task completion time, their effect for the synchronous task was opposite to that for the asynchronous task. On the other hand, the rotation methods had significant effects only on turn angle. ©2010 IEEE.
• Liu, S., & Hua, H. (2010). A systematic method for designing depth-fused multi-focal plane three-dimensional displays. Optics Express, 18(11), 11562-11573.
  More info

  PMID: 20589017;Abstract: Lack of accurate focus cues in conventional stereoscopic displays has potentially significant effects on depth perception accuracy and visual fatigue. Recently several multi-focal plane display prototypes have been demonstrated with the promise of improving the accuracy of focus cue rendering in stereoscopic displays. In this paper, we present a systematic method to address two fundamental issues in designing a multi-focal plane display: (1) the appropriate dioptric spacing between adjacent focal planes; and (2) the depth-weighted fusing function to render a continuous threedimensional (3-D) volume using a sparse number of focal planes placed in the space. By taking account of both ocular factors of the human visual system (HVS) and display factors of a multi-focal plane system, we determine that an appropriate spacing between two adjacent focal planes should be -0.6 diopter (D) while a smaller spacing may be necessary for further improving retinal image quality. We further develop a set of nonlinear depth-weighted fusing function with the promise of balancing perceptual continuity of a 3-D scene and retinal image quality. Our method was based on quantitative evaluation of the modulation transfer functions (MTF) of depth-fused images formed on retina. © 2010 Optical Society of America.
• Liu, S., Hua, H., & Cheng, D. (2010). A novel prototype for an optical see-through head-mounted display with addressable focus cues. IEEE Transactions on Visualization and Computer Graphics, 16(3), 381-393.
  More info

  PMID: 20224134;Abstract: We present the design and implementation of an optical see-through head-mounted display (HMD) with addressable focus cues utilizing a liquid lens. We implemented a monocular bench prototype capable of addressing the focal distance of the display from infinity to as close as 8 diopters. Two operation modes of the system were demonstrated: a vari-focal plane mode in which the accommodation cue is addressable, and a time-multiplexed multi-focal plane mode in which both the accommodation and retinal blur cues can be rendered. We further performed experiments to assess the depth perception and eye accommodative response of the system operated in a vari-focal plane mode. Both subjective and objective measurements suggest that the perceived depths and accommodative responses of the user match with the rendered depths of the virtual display with addressable accommodation cues, approximating the real-world 3-D viewing condition. © 2010 IEEE.
• Wang, X., & Hua, H. (2010). Theoretical analysis for integral imaging performance based on microscanning of a microlens array. OPTICS LETTERS, 33(5), 449-451.
• Zhang, R., & Hua, H. (2010). Effects of a retroreflective screen on depth perception in a head-mounted projection display. 9th IEEE International Symposium on Mixed and Augmented Reality 2010: Science and Technology, ISMAR 2010 - Proceedings, 137-145.
  More info

  Abstract: The perceived depth accuracy is a fundamental performance metrics for an augmented reality display system. Many factors may affect the perceived depth of a head-mounted display (HMD), such as display resolution, interpupillary distance (IPD), conflicting depth cues, stereoacuity, and head tracking error. Besides these typical limiting factors to an HMD-type system, the perceived depth through a head-mounted projection display (HMPD) may be further affected by the usage of a retroreflective screen. In this paper, we will evaluate the perceived depth accuracy of an HMPD system using a perceptual depth matching method. The main factor to be investigated in our study is the position of a retroreflective screen relative to the projection image plane, with the projection image plane placed at different distances to the user. Overall, it is found that the ratio of the judged distance to real distance and the standard deviation of the judged distance increase linearly with the reference object distance. A transition effect from depth underestimation to overestimation has been observed at the reference object distance of around 1.4m. The position of a retroreflective screen only has a significant effect on the depth judgment error around this switching point. The paper also analyzes various effects brought by a retroreflective screen on the depth judgment. The depth cue and the image luminance reduction brought by a retroreflective screen could be the main factors that affect the depth judgment accuracy. ©2010 IEEE.
• Zhang, R., & Hua, H. (2010). Imaging quality of a retroreflective screen in head-mounted projection displays. JOURNAL OF THE OPTICAL SOCIETY OF AMERICA A-OPTICS IMAGE SCIENCE AND VISION, 26(5), 1240-1249.
• Cheng, D., Wang, Y., Hua, H., & Sasian, J. (2009). Design of a wide-angle, lightweight head-mounted display using free-form optics tiling. OPTICS LETTERS, 36(11), 2098-2100.
• Cheng, D., Wang, Y., Hua, H., & Talha, M. M. (2009). Design of an optical see-through head-mounted display with a low f-number and large field of view using a freeform prism. Applied Optics, 48(14), 2655-2668.
  More info

  PMID: 19424386;Abstract: It has been a challenge to design an optical see-through head-mounted display (OST-HMD) that has a wide field of view (FOV) and low f-number (f/#) while maintaining a compact, lightweight, and non- intrusive form factor. In this paper, we present an OST-HMD design using a wedge-shaped freeform prism cemented with a freeform lens. The prism, consisting of three freeform surfaces (FFSs), serves as the near-eye viewing optics that magnifies the image displayed through a microdisplay, and the free- form lens is an auxiliary element attached to the prism in order to maintain a nondistorted see-through view of a real-world scene. Both the freeform prism and the lens utilize plastic materials to achieve light weight. The overall dimension of the optical system per eye is no larger than 25 mm by 22 mm by 12 mm, and the weight is 8 g. Based on a 0.61 in. microdisplay, our system demonstrates a diagonal FOV of 53.5° and an f/# of 1.875, with an 8 mm exit pupil diameter and an 18.25 mm eye relief. © 2009 Optical Society of America.
• Hua, H., & Gao, C. (2009). Design of a bright polarized head-mounted projection display. APPLIED OPTICS, 46(14), 2600-2610.
• Hua, H., Ahuja, N., & Gao, C. (2009). Design analysis of a high-resolution panoramic camera using conventional imagers and a mirror pyramid. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 29(2), 356-361.
• Hua, H., Liu, S., & Hua, H. -. (2009). Time-multiplexed dual-focal plane head-mounted display with a liquid lens. Optics letters, 34(11).
  More info

  Focus cues play a significant role in three-dimensional (3-D) depth perception. Conventional stereoscopic displays, however, lack the ability to correctly render these cues, because they present a pair of stereoscopic images on a fixed image plane while forcing the eyes to converge at different distances to view objects at different depths. Using a fast liquid-lens device, we present the design and implementation of a time-multiplexed dual-focal plane display that is capable of rendering correct or near-correct focus cues as well as other depth cues, such as occlusion and shading for a 3-D scene. The focus range of the dual focal planes can vary from infinity to as close as 8 diopters. Two driving mechanisms are proposed to render near-correct focus cues and their effects on image sharpness, flicker, and brightness are compared.
• Hua, H., Zhang, R., & Hua, H. -. (2009). Imaging quality of a retroreflective screen in head-mounted projection displays. Journal of the Optical Society of America. A, Optics, image science, and vision, 26(5).
  More info

  A retroreflective screen composed of miniature corner cube reflectors (CCRs) or microbeads redirects an incident ray in the reverse direction. Recently the retroreflective screen was utilized as a key element in head-mounted projection displays (HMPDs). Most prior efforts in developing the HMPD technology have been focused on optimizing the optical design of the projection optics, neglecting the imaging artifacts caused by the screen. Few efforts have been attempted to analyze and evaluate the overall image quality of the HMPD system with the presence of a retroreflective screen. This paper first applies a ray-tracing method to examine the imaging properties of a single CCR. Through the combination of both the geometrical imaging effect and the diffraction effect, the imaging properties of a CCR-based retroreflective screen are analyzed and characterized. Based on these analytical results, the paper further evaluates how the imaging artifacts of a retroreflective screen degrade the spatial resolution of an HMPD system and limit the tolerance range of the distance from an HMPD user to the screen. Finally, a discussion is employed to illustrate potential techniques in minimizing the image quality degradation through the optimization of the corner cube size in a retroreflective screen.
• Liu, S., & Hua, H. (2009). Time-multiplexed dual-focal plane head-mounted display with a liquid lens. Optics Letters, 34(11), 1642-1644.
  More info

  PMID: 19488134;Abstract: Focus cues play a significant role in three-dimensional (3-D) depth perception. Conventional stereoscopic displays, however, lack the ability to correctly render these cues, because they present a pair of stereoscopic images on a fixed image plane while forcing the eyes to converge at different distances to view objects at different depths. Using a fast liquid-lens device, we present the design and implementation of a time-multiplexed dual-focal plane display that is capable of rendering correct or near-correct focus cues as well as other depth cues, such as occlusion and shading for a 3-D scene. The focus range of the dual focal planes can vary from infinity to as close as 8 diopters. Two driving mechanisms are proposed to render near-correct focus cues and their effects on image sharpness, flicker, and brightness are compared. © 2009 Optical Society of America.
• Zhang, R., & Hua, H. (2009). Imaging quality of a retroreflective screen in head-mounted projection displays. Journal of the Optical Society of America A: Optics and Image Science, and Vision, 26(5), 1240-1249.
  More info

  PMID: 19412243;Abstract: A retroreflective screen composed of miniature corner cube reflectors (CCRs) or microbeads redirects an incident ray in the reverse direction. Recently the retroreflective screen was utilized as a key element in headmounted projection displays (HMPDs). Most prior efforts in developing the HMPD technology have been focused on optimizing the optical design of the projection optics, neglecting the imaging artifacts caused by the screen. Few efforts have been attempted to analyze and evaluate the overall image quality of the HMPD system with the presence of a retroreflective screen. This paper first applies a ray-tracing method to examine the imaging properties of a single CCR. Through the combination of both the geometrical imaging effect and the diffraction effect, the imaging properties of a CCR-based retroreflective screen are analyzed and characterized. Based on these analytical results, the paper further evaluates how the imaging artifacts of a retroreflective screen degrade the spatial resolution of an HMPD system and limit the tolerance range of the distance from an HMPD user to the screen. Finally, a discussion is employed to illustrate potential techniques in minimizing the image quality degradation through the optimization of the corner cube size in a retroreflective screen. © 2009 Optical Society of America.
• Cheng, D., Wang, Y., Talha, M. M., Chang, J., & Hua, H. (2008). Effect of radius mismatch on performance of segmented telescopic systems. Proceedings of SPIE - The International Society for Optical Engineering, 6834.
  More info

  Abstract: There exist great interests in using segmented mirrors instead of monoliphic primary mirror to build large aperture telescopes. The use of segmented mirrors reduces the difficulties in fabrication, transportation and replication as well as decreases the impact of gravity. In the meanwhile, one of the grand challenges in this approach lies in matching the individual mirrors to yield the form of a continuous surface. The erroneous differences between the global radii of curvature (GROC) of the segmented mirrors have significant influence on image quality. In this paper, we present a model of segmented systems and an analysis technique which reveals the effect of these errors on the image quality. Our analytical results show that the GROC differences have great influence on image quality and the impact remains high despite of using active optics to compensate these errors. The local radii of curvature of individual mirror segment should be used to correct these errors. We propose a new approach to generate compensators that controls the local radius of curvature of each segment efficiently. Finally, through simulation results, we demonstrate that the new compensators can dramatically reduce the effect of radius mismatch.
• Hu, X., & Hua, H. (2008). Design and Assessment of a Depth-Fused Multi-Focal-Plane Display Prototype. JOURNAL OF DISPLAY TECHNOLOGY, 10(4), 308-316.
• Hua, H., & Liu, S. (2008). Dual-sensor foveated imaging system. Applied Optics, 47(3), 317-327.
  More info

  PMID: 18204718;Abstract: Conventional imaging techniques adopt a rectilinear sampling approach, where a finite number of pixels are spread evenly across an entire field of view (FOV). Consequently, their imaging capabilities are limited by an inherent trade-off between the FOV and the resolving power. In contrast, a foveation technique allocates the limited resources (e.g., a finite number of pixels or transmission bandwidth) as a function of foveal eccentricities, which can significantly simplify the optical and electronic designs and reduce the data throughput, while the observer's ability to see fine details is maintained over the whole FOV. We explore an approach to a foveated imaging system design. Our approach approximates the spatially variant properties (i.e., resolution, contrast, and color sensitivities) of the human visual system with multiple low-cost off-the-shelf imaging sensors and maximizes the information throughput and bandwidth savings of the foveated system. We further validate our approach with the design of a compact dual-sensor foveated imaging system. A proof-of-concept bench prototype and experimental results are demonstrated. © 2008 Optical Society of America.
• Hua, H., Hu, X., & Gao, C. (2008). A high-resolution optical see-through head-mounted display with eyetracking capability. OPTICS EXPRESS, 21(25), 30993-30998.
• Hua, H., Zhang, R., & Hua, H. -. (2008). Characterizing polarization management in a p-HMPD system. Applied optics, 47(4).
  More info

  It has been a common challenge to operate optical see-through head-mounted displays in well-lit environments due to the low image brightness and contrast compared with the direct view of a real-world scene. This problem is aggravated in the design of a see-through head-mounted projection display (HMPD) in which the projected light is split twice by a beam splitter and further attenuated greatly by a retroreflective screen. A polarizing head-mounted projection display (p-HMPD) design was recently proposed to enhance the overall flux transfer efficiency and thus increase the brightness and contrast of displayed images. Different from the conventional nonpolarizing HMPD designs, the light polarization states in the p-HMPD system are deliberately manipulated to maximize the flux transfer efficiency, which can potentially result in three times higher efficiency than that of a nonpolarizing HMPD. By measuring the Mueller matrices of the major elements in both a p-HMPD and a nonpolarizing HMPD, we characterize the polarization dependence of each element on incident angles and wavelengths, and also investigate the depolarization effect of the retroreflective screen. Based on these experimental results, we further examine the overall luminance efficiencies of the two types of systems and analyze how various aspects of display performances are affected by the angular and chromatic dependence of the polarization components.
• Hua, H., Zhang, R., & Hua, H. -. (2008). Design of a polarized head-mounted projection display using ferroelectric liquid-crystal-on-silicon microdisplays. Applied optics, 47(15).
  More info

  It has been a common problem in optical see-through head-mounted displays that the displayed image lacks brightness and contrast compared with the direct view of a real-world scene. This problem is aggravated in head-mounted projection displays in which multiple beam splitting and low retroreflectance of a typical retroreflective projection screen yield low luminous transfer efficiency. To address this problem, we recently proposed a polarized head-mounted projection display (p-HMPD) design where the polarization states of the light are deliberately manipulated to maximize the luminous transfer efficiency. We report the design of a compact p-HMPD prototype system using a pair of high-resolution ferroelectric liquid-crystal-on-silicon (FLCOS) microdisplays. In addition to higher resolution, the FLCOS displays have much higher optical efficiency than a transmissive-type liquid crystal display (LCD) and help to further improve the overall light efficiency and image quality. We detail the design of a compact illumination unit for the FLCOS microdisplay, also commonly referred to as the light engine, and a projection lens, both of which are key parts of the p-HMPD system. The performances of the light engine and projection lens are analyzed in detail. Finally, we present the design of a compact p-HMPD prototype using the custom-designed light engine and projection optics.
• Liu, S., & Hua, H. (2008). Spatialchromatic foveation for gaze contingent displays. Eye Tracking Research and Applications Symposium (ETRA), 139-142.
  More info

  Abstract: Spatially variant resolution method has been widely explored for Gaze Contingent Displays (GCDs). Recently several studies suggested that spatial chromatic foveation can further improve the sampling efficiency and save computational resources and communication bandwidths in GCDs. In this paper, we explore the spatial variance of the contrast sensitivity function (CSF) of the human visual system (HVS) to examine the potential of spatialchromatic foveation in GCDs. The proposed algorithm reveals that, not only the spatial resolution, but also the chrominance complexity can be monotonically degraded from the center of the field of view (FOV) to the periphery of a GCD. A perceptually-based spatialchromatic foveation metric is derived. Applying the proposed hue-resolution foveation metric, we demonstrate that over 65% of bandwidth can be saved. Copyright © 2008 by the Association for Computing Machinery, Inc.
• Liu, S., Cheng, D., & Hua, H. (2008). An optical see-through head mounted display with addressable focal planes. Proceedings - 7th IEEE International Symposium on Mixed and Augmented Reality 2008, ISMAR 2008, 33-42.
  More info

  Abstract: Most existing stereoscopic head mounted displays (HMDs), presenting a pair of stereoscopic images at a fixed focal distance, lack the ability to correctly render the naturally coupled accommodation and convergence cues. Psychophysical studies have shown that such displays may cause many adverse consequences such as visual fatigue, diplopic vision, degraded oculomotor response, and depth perception errors. In this paper, we present a see-through HMD with addressable focal planes utilizing a novel active optical element - a liquid lens. The element, with a varying optical power from -5 to 20 diopters, is able to address the focal distance of the HMD from infinity to the near point of the eye. A monocular prototype was built from off-the-shelf elements and experimental results are presented to validate the proposed designs. We also describe both subjective and objective measurements of the accommodation responses of the viewer to the focal distances presented by the prototype. © 2008 IEEE.
• Liu, S., Hua, H., & Gao, C. (2008). Illumination design of a multi-touch sensing projection screen for augmented virtual environments. Digest of Technical Papers - SID International Symposium, 39(1), 174-177.
  More info

  Abstract: The paper presents the illumination engineering for a low-cost multi-touch projection screen. Eight near infrared emitting diode (IRED) strips with optimized line densities serve as the illumination sources for the finger tracking. The design achieves highly uniformed illumination, with an illumination efficiency of 55% and contrast of 0.098. Implementation and experimental results are presented. © 2008 SID.
• Oh, J., & Hua, H. (2008). Usability of multi-scale interfaces for 3D workbench displays. Presence: Teleoperators and Virtual Environments, 17(5), 415-440.
  More info

  Abstract: We consider that multi-scale visualization interfaces support users to view different levels of scales simultaneously and to understand large-scale, complex 3D information in 3D display environments. This article presents a user evaluation on three multi-scale interfaces on a 3D workbench display: focus + context (f + c), fixed f + c, and overview + detail (o + d). The interfaces differ in terms of (1) window arrangement and (2) positioning of detailed information relative to the user. Our goal is to identify the effect of these interface differences in large scale information visualization on a 3D workbench. To address the usability of the interfaces for a wide range of applications, we designed two tasks that differ by the level of information integration and cognitive demand. The evaluation results suggest that focus-based interfaces (i.e., the f + c and fixed f + c interfaces) are useful for tasks that require tight coupling between information layers and the o + d interface is useful for tasks performed in a densely populated information space. In terms of interface design on a 3D workbench, it is important to provide an up-close view of the current region of interest for fast scene navigation and an easy way to change viewing direction to see the 3D information from more comfortable directions. The detailed design guidelines based on the evaluation analysis are presented in this article. © 2008 by the Massachusetts Institute of Technology.
• Wang, Q., Cheng, D., Wang, Y., Hua, H., & Jin, G. (2008). Design, tolerance, and fabrication of an optical see-through head-mounted display with free-form surface elements. APPLIED OPTICS, 52(7), C88-C99.
• Wang, X., & Hua, H. (2008). Theoretical analysis for integral imaging performance based on microscanning of a microlens array. Optics Letters, 33(5), 449-451.
  More info

  PMID: 18311288;Abstract: An integral imaging method based on microscanning of a microlens array is proposed. The effects of micro-scanning on the depth resolution and the product of depth of focus and lateral resolution squared (PDLRS) of an integral imaging system are quantitatively analyzed. Calculation results show that microscanning can improve not only the depth resolution, but the PDLRS can be increased. The amount of improvement of different microscanning modes to the performance of integral imaging is closely related to the fill factor of the detector and the diffraction factor of the microlens. © 2008 Optical Society of America.
• Wu, R., Hua, H., Benitez, P., Minano, J. C., & Liang, R. (2008). Design of compact and ultra efficient aspherical lenses for extended Lambertian sources in two-dimensional geometry. OPTICS EXPRESS, 24(5), 5078-5086.
• Zhang, R., & Hua, H. (2008). Characterizing polarization management in a p-HMPD system. Applied Optics, 47(4), 512-522.
  More info

  PMID: 18239710;Abstract: It has been a common challenge to operate optical see-through head-mounted displays in well-lit environments due to the low image brightness and contrast compared with the direct view of a real-world scene. This problem is aggravated in the design of a see-through head-mounted projection display (HMPD) in which the projected light is split twice by a beam splitter and further attenuated greatly by a retroreflective screen. A polarizing head-mounted projection display (p-HMPD) design was recently proposed to enhance the overall flux transfer efficiency and thus increase the brightness and contrast of displayed images. Different from the conventional nonpolarizing HMPD designs, the light polarization states in the p-HMPD system are deliberately manipulated to maximize the flux transfer efficiency, which can potentially result in three times higher efficiency than that of a nonpolarizing HMPD. By measuring the Mueller matrices of the major elements in both a p-HMPD and a nonpolarizing HMPD, we characterize the polarization dependence of each element on incident angles and wavelengths, and also investigate the depolarization effect of the retroreflective screen. Based on these experimental results, we further examine the overall luminance efficiencies of the two types of systems and analyze how various aspects of display performances are affected by the angular and chromatic dependence of the polarization components. © 2008 Optical Society of America.
• Zhang, R., & Hua, H. (2008). Design of a compact light engine for FLCOS microdisplays in a p-HMPD system. Digest of Technical Papers - SID International Symposium, 39(1), 85-88.
  More info

  Abstract: We have designed a compact light engine for a ferroelectric Liquid-crystal-on-Silicon (FLCOS) microdisplay used in a polarized head mounted projection display (p-HMPD). In the light engine, the polarization of the light is manipulated in order to achieve a compact structure and a mirror based tapered light pipe is designed to improve the light efficiency and image uniformity. The performance of the light engine is analyzed based on the simulation results using LightTools. © 2008 SID.
• Zhang, R., & Hua, H. (2008). Design of a polarized head-mounted projection display using ferroelectric liquid-crystal-on-silicon microdisplays. APPLIED OPTICS, 47(15), 2888-2896.
• Zhang, R., & Hua, H. (2008). Design of a polarized head-mounted projection display using ferroelectric liquid-crystal-on-silicon microdisplays. Applied Optics, 47(15), 2888-2896.
  More info

  PMID: 18493297;Abstract: It has been a common problem in optical see-through head-mounted displays that the displayed image lacks brightness and contrast compared with the direct view of a real-world scene. This problem is aggravated in head-mounted projection displays in which multiple beam splitting and low retroreflectance of a typical retroreflective projection screen yield low luminous transfer efficiency. To address this problem, we recently proposed a polarized head-mounted projection display (p-HMPD) design where the polarization states of the light are deliberately manipulated to maximize the luminous transfer efficiency. We report the design of a compact p-HMPD prototype system using a pair of high-resolution ferroelectric liquid-crystal-on-silicon (FLCOS) microdisplays. In addition to higher resolution, the FLCOS displays have much higher optical efficiency than a transmissive-type liquid crystal display (LCD) and help to further improve the overall light efficiency and image quality. We detail the design of a compact illumination unit for the FLCOS microdisplay, also commonly referred to as the light engine, and a projection lens, both of which are key parts of the p-HMPD system. The performances of the light engine and projection lens are analyzed in detail. Finally, we present the design of a compact p-HMPD prototype using the custom-designed light engine and projection optics. © 2008 Optical Society of America.
• Gao, C., Ahuja, N., & Hua, H. (2007). Active aperture control and sensor modulation for flexible imaging. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.
  More info

  Abstract: In the paper, we describe an optical system which is capable of providing external access to both the sensor and the lens aperture (i.e., projection center) of a conventional camera. The proposed optical system is attached in front of the camera, and is the equivalent of adding externally accessible intermediate image plane and projection center. The system offers controls of the response of each pixel which could be used to realize many added imaging functions, such as high dynamic range imaging, image modulation, and optical computation. The ability to access the optical center could enable a wide variety of applications by simply allowing manipulation of the geometric properties of the optical center. For instance, panoramic imaging can be implemented by rotating a planar mirror about the camera axis; and small base-line stereo can be implemented by shifting the camera center. We have implemented a bench setup to demonstrate some of these functions. The experimental results are included. © 2007 IEEE.
• Gao, C., Hua, H., & Ahuia, N. (2007). A hemispherical imaging camera. COMPUTER VISION AND IMAGE UNDERSTANDING, 114(2), 168-178.
• Hu, X., & Hua, H. (2007). High-resolution optical see-through multi-focal-plane head-mounted display using freeform optics. OPTICS EXPRESS, 22(11), 13896-13903.
• Hua, H., & Gao, C. (2007). A systematic framework for on-line calibration of a head-mounted projection display for augmented-reality systems. Journal of the Society for Information Display, 15(11), 905-913.
  More info

  Abstract: Augmented reality (AR) is a technology in which computer-generated virtual images are dynamically superimposed upon a real-world scene to enhance a user's perceptions of the physical environment. A successful AR system requires that the overlaid digital information be aligned with the user's real-world senses - a process known as registration. An accurate registration process requires the knowledge of both the intrinsic and extrinsic parameters of the viewing device and these parameters form the viewing and projection transformations for creating the simulations of virtual images. In our previous work, an easy off-line calibration method in which an image-based automatic matching method was used to establish the world-to-image correspondences was presented, and it is able to achieve subpixel accuracy. However, this off-line method yields accurate registration only when a user's eye placements relative to the display device coincides with locations established during the offline calibration process. A likely deviation of eye placements, for instance, due to helmet slippage or user-dependent factors such as interpupillary distance, will lead to misregistration. In this paper, a systematic on-line calibration framework to refine the off-line calibration results and to account for user-dependent factors is presented. Specifically, based on an equivalent viewing projection model, a six-parameter on-line calibration method to refine the user-dependent parameters in the viewing transformations is presented. Calibration procedures and results as well as evaluation experiments are described in detail. The evaluation experiments demonstrate the improvement of the registration accuracy. © Copyright 2007 Society for Information Display.
• Hua, H., & Gao, C. (2007). Design of a bright polarized head-mounted projection display. Applied Optics, 46(14), 2600-2610.
  More info

  PMID: 17446907;Abstract: In optical see-through head-mounted displays, it has been a common challenge that the displayed image lacks brightness and contrast compared with the direct view of a real-world scene. Consequently, such displays are usually used in dimmed lighting conditions, which limits the feasibility of applying such information displays outdoors or in scenarios where well-lit environments, such as in operation rooms, are required. The lack of image brightness is aggravated in the design of a see-through head-mounted projection display (HMPD). For instance, the overall flux transfer efficiency of existing HMPD designs is less than 10%. The design of a polarized head-mounted projection display (p-HMPD) is presented. The images of a p-HMPD system can potentially be three times brighter than those in existing HMPD designs. It is further demonstrated that the p-HMPD design is able to dramatically improve image brightness, contrast, and color vividness with experimental results. Finally, the design of a compact optical system and helmet prototype is described. © 2007 Optical Society of America.
• Hua, H., Ahuja, N., & Gao, C. (2007). Design analysis of a high-resolution panoramic camera using conventional imagers and a mirror pyramid. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(2), 356-361.
  More info

  PMID: 17170487;Abstract: Wide field of view (FOV) and high-resolution image acquisition is highly desirable in many vision-based applications. Several systems have reported the use of reflections off mirror pyramids to capture high-resolution, single-viewpoint, and wide-FOV images. Using a dual mirror pyramid (DMP) panoramic camera as an example, in this paper, we examine how the pyramid geometry, and the selection and placement of imager clusters can be optimized to maximize the overall panoramic FOV, sensor utilization efficiency, and image uniformity. The analysis can be generalized and applied to other pyramid-based designs. © 2007 IEEE.
• Hua, H., Gao, C., & Ahuja, N. (2007). Calibration of an HMPD-based augmented reality system. IEEE Transactions on Systems, Man, and Cybernetics Part A:Systems and Humans, 37(3), 416-430.
  More info

  Abstract: In augmented reality (AR) applications, accurately registering a virtual object with its real counterpart is a challenging problem. The size, depth, and geometry, as well as the physical attributes of the virtual object, have to be rendered precisely relative to a physical reference. This paper presents a systematic calibration process to address the registration challenge in a custom-designed AR system, which is based upon recent head-mounted projective display (HMPD) technology. Following a concise review of the HMPD concept and our system configuration, we first present a computational model of the HMPD viewing system and requirements for the system calibration. Then, we describe, in detail, the calibration procedures to obtain estimates of unknown transformations, summarize the application of the estimates in a customized graphics rendering toolkit, and discuss the evaluation experiments and observations. Finally, the implementation of a testbed to demonstrate a successful registration is briefly described, and experimental results are presented. © 2007 IEEE.
• Hua, H., Pansing, C. W., & Rolland, J. P. (2007). Modeling of an eye-imaging system for optimizing illumination schemes in an eye-tracked head-mounted display. Applied Optics, 46(31), 7757-7770.
  More info

  PMID: 17973021;Abstract: While the augmentation of head-mounted displays (HMDs) with eye-tracking (ET) capabilities adds challenges to designing compact and portable displays, a systematic approach to integration offers opportunities to improve overall performance and robustness. To investigate the design and optimization of illumination schemes in an ET-HMD system, we present a simulated eye illumination and imaging system, which allows us to explore the critical parameters that affect the quality of the eye illumination. We present details on the modeling process and simulation results. © 2007 Optical Society of America.
• Lee, S., & Hua, H. (2007). A Robust Camera-Based Method for Optical Distortion Calibration of Head-Mounted Displays. JOURNAL OF DISPLAY TECHNOLOGY, 11(10), 845-853.
• Liu, S., Hua, H., & Cheng, D. (2007). A Novel Prototype for an Optical See-Through Head-Mounted Display with Addressable Focus Cues. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, 16(3), 381-393.
• Oh, J., & Hua, H. (2007). Usability of multi-scale interfaces for 3D workbench displays. PRESENCE-TELEOPERATORS AND VIRTUAL ENVIRONMENTS, 17(5), 415-440.
• Oh, J., & Hua, H. (2007). User evaluations on form factors of tangible magic lenses. Proceedings - ISMAR 2006: Fifth IEEE and ACM International Symposium on Mixed and Augmented Reality, 23-32.
  More info

  Abstract: Magic Lens is a small inset window embedded in a large context display, which provides an alternative view to the region of interest selected from the context view. This metaphor is used for 3D visualization in our Augmented Virtual Environment infrastructure, SCAPE (Stereoscopic Collaboration in Augmented and Projective Environments), which is composed of an immersive room display for a high level of detail, life-size virtual world and a workbench display for simplified god-like view to the world. A tangible Magic Lens is used on the workbench display to allow direct and intuitive selection of continuous levels of detail, bridging the gap between the two extreme levels of detail in SCAPE. This paper presents our first step to the user evaluations of tangible Magic Lens. We conducted two sets of user evaluations, one mainly testing the lens aspect ratio, and another for the lens size. For both of the tests, two types of tasks are conducted: information gathering and relating the detailed information with the context. We found that the aspect ratio of a lens plays more important role in user preference for smaller lenses than for larger ones. Mean while, the size of a lens is the most important factor that affects the user performance in the two types of tasks. ©2006 IEEE.
• Zhang, R., & Hua, H. (2007). Design of a polarized head-mounted projection display using FLCOS micro displays. Proceedings of SPIE - The International Society for Optical Engineering, 6489.
  More info

  Abstract: Head-mounted projection display (HMPD) technology, as an alternative to conventional head mounted displays (HMD), offers the potential of designing wide field-of-view (FOV) optical see-through HMDs. Due to multiple passes through a beamsplitter, however, existing HMPD designs suffer from low luminance efficiency and thus the displayed image is lack of brightness and contrast. The design of a polarized head-mounted projection display (p-HMPD) was recently proposed. The major departure of a p-HMPD design from other existing HMPD systems is the usage of polarization management to minimize light loss through beamsplitting. A p-HMPD consists of a pair of projections lenses, microdisplays, polarization control elements, and retroreflective sheeting material as a projection screen. In this paper, we explore the usage of a ferroelectric liquid-crystal-on-silicon (FLCOS) microdisplay as the image source and present a compact design of an illumination unit with double telecentric optics to achieve highly bright and uniform illumination on the FLCOS microdisplay. The key contribution of this design lies in the compactness which is a critical factor in HMD systems. The first-order optics and the transformation of polarization in the design will be described in detail. The simulation of the illumination unit will be shown and its luminous efficiency and uniformity will be discussed. Based on this illumination engine, we will further describe the design of a compact projection lens and discuss the overall performance of the optics.
• Brown, L. D., & Hua, H. (2006). Magic lenses for augmented virtual environments. IEEE Computer Graphics and Applications, 26(4), 64-73.
  More info

  PMID: 16863099;Abstract: The authors developed a Magic Lens framework for Scape, an augmented virtual environment (AVE). They generalize the functional characteristics of Magic Lenses in terms of 3D visualization in AVEs and present two tangible Magic Lens-enabled devices with complementary interface capabilities. The authors also demonstrate their Magic Lens devices through testbed applications relevant to urban planning and medical training. © 2006 IEEE.
• Curatu, C., Hua, H., & Rolland, J. (2006). Dual purpose lens for an eye-tracked projection head-mounted display. Proceedings of SPIE - The International Society for Optical Engineering, 6342 I.
  More info

  Abstract: The novel concept of the ET-HMPD, which consists of a Head-Mounted Projection Display (HMPD) with an integrated Eye-Tracking (ET) capability, was recently presented as well as the design of some of its components [Curatu, Hua and Rolland, Proceedings of the SPIE 5875, 2005]. In this paper, we present the overall system design and performance, assuming an ideal cold cube and semi-transparent hot plate. © 2006 SPIE-OSA.
• Hua, H. (2006). Merging the worlds of atoms and bits: Augmented virtual environments. Optics and Photonics News, 17(10), 26-33.
  More info

  Abstract: The issues related to advancements in creating interactive augmented virtual environments are discussed. Augmented reality seeks to replace users' perceptions of their real environment with computer-generated simulations and integrate digital information into the fabric of the physical world and enhance sensory perceptions. The research into immersive virtual environments has focused on the development of visualization tools and interaction techniques for multiple-user interaction. In a 3D visualization system, the support of multiple-user interactions and the maintenance of smooth transition and communication impose great challenges in both display and user interaction techniques. It is recommended that efforts should be made to develop the less-intrusive display technologies and novel visualization and interaction methods to support applications that involve heterogeneous data sets.
• Hua, H., Krishnaswamy, P., & Rolland, J. P. (2006). Video-based eyetracking methods and algorithms in head-mounted displays. OPTICS EXPRESS, 14(10), 4328-4350.
• Hua, H., Krishnaswamy, P., & Rolland, J. P. (2006). Video-based eyetracking methods and algorithms in head-mounted displays. Optics Express, 14(10), 4328-4350.
  More info

  PMID: 19516585;Abstract: Head pose is utilized to approximate a user's line-of-sight for real-time image rendering and interaction in most of the 3D visualization applications using head-mounted displays (HMD). The eye often reaches an object of interest before the completion of most head movements. It is highly desirable to integrate eye-tracking capability into HMDs in various applications. While the added complexity of an eyetracked-HMD (ET-HMD) imposes challenges on designing a compact, portable, and robust system, the integration offers opportunities to improve eye tracking accuracy and robustness. In this paper, based on the modeling of an eye imaging and tracking system, we examine the challenges and identify parametric requirements for video-based pupil-glint tracking methods in an ET-HMD design, and predict how these parameters may affect the tracking accuracy, resolution, and robustness. We further present novel methods and associated algorithms that effectively improve eye-tracking accuracy and extend the tracking range. © 2006 Optical Society of America.
• Liu, S., & Hua, H. (2006). A systematic method for designing depth-fused multi-focal plane three-dimensional displays. OPTICS EXPRESS, 18(11), 11562-11573.
• Liu, S., Pansing, C., & Hua, H. (2006). Design of a foveated imaging system using a two-axis MEMS mirror. Proceedings of SPIE - The International Society for Optical Engineering, 6342 II.
  More info

  Abstract: We present a novel foveated imaging system which consists of three major components: a foveated imager, a peripheral imager, and a 2D scanning sub-system. The foveated imager captures a narrow field of view (FOV) with high angular resolution to mimic the foveated region of interest (FRoI), and the peripheral imager captures a wide FOV with low angular resolution to provide the peripheral region for context. The scanning system, implemented with a single-element two-axis MEMS mirror, is capable of sweeping the FRoI across the entire FOV at a maximum speed of 100Hz. A bench prototype and experimental results are presented as verifications of our optical design concept and feasibility. © 2006 SPIE-OSA.
• Curatu, C., Hua, H., & Rolland, J. (2005). Projection-based head-mounted display with eye-tracking capabilities. Proceedings of SPIE - The International Society for Optical Engineering, 5875, 1-9.
  More info

  Abstract: We propose a novel conceptual design for a Head-Mounted Projection Display (HMPD) with Eye-Tracking (ET) capabilities. We present a fully integrated system that is robust, easy to calibrate, inexpensive, and lightweight. The HMD-ET integration is performed from a low-level optical configuration in order to achieve a compact, comfortable, easy-to-use system. The idea behind the full integration consists of sharing the optical path between the HMD and the Eye-Tracker. Along with lens design and optimization, system level issues such as eye illumination options, hardware alternatives are discussed.
• Hua, H., & Gao, C. (2005). A polarized head-mounted projective display. Proceedings - Fourth IEEE and ACM International Symposium on Symposium on Mixed and Augmented Reality, ISMAR 2005, 2005, 32-35.
  More info

  Abstract: The lack of image brightness is a common problem in optical see-through head-mounted displays (OST-HMD) where a beamsplitter is required to combine views from HMD image source and the direct-view of a real world scene. This problem is further aggregated in a head-mounted projective display (HMPD) due to the fact that light passes through the beamsplitter multiple times. In this paper, we present a novel design of an ultra-bright polarized head-mounted projective display (p-HMPD). The image brightness observed by a viewer theoretically is four-times brighter than existing designs. We further demonstrate a design with currently available technology that leads to a display in which the observed image is significantly brighter than existing designs. Finally, experimental results from a bench setup are presented. © 2005 IEEE.
• Pansing, C. W., Hua, H., & Rolland, J. P. (2005). Optimization of illumination schemes in a head-mounted display integrated with eye tracking capabilities. Proceedings of SPIE - The International Society for Optical Engineering, 5875, 1-13.
  More info

  Abstract: Head-mounted display (HMD) technologies find a variety of applications in the field of 3D virtual and augmented environments, 3D scientific visualization, as well as wearable displays. While most of the current HMDs use head pose to approximate line of sight, we propose to investigate approaches and designs for integrating eye tracking capability into HMDs from a low-level system design perspective and to explore schemes for optimizing system performance. In this paper, we particularly propose to optimize the illumination scheme, which is a critical component in designing an eye tracking-HMD (ET-HMD) integrated system. An optimal design can improve not only eye tracking accuracy, but also robustness. Using LightTools®, we present the simulation of a complete eye illumination and imaging system using an eye model along with multiple near infrared LED (IRLED) illuminators and imaging optics, showing the irradiance variation of the different eye structures. The simulation of dark pupil effects along with multiple lst-order Purkinje images will be presented. A parametric analysis is performed to investigate the relationships between the IRLED configurations and the irradiance distribution at the eye, and a set of optimal configuration parameters is recommended. The analysis will be further refined by actual eye image acquisition and processing.
• Hua, H., Brown, L. D., & Gao, C. (2004). Scape: Supporting Stereoscopic Collaboration in Augmented and Projective Environments. IEEE Computer Graphics and Applications, 24(1), 66-75.
  More info

  PMID: 15384670;Abstract: A collaborative infrastructure for stereoscopic collaboration in augmented and projective environments (Scape) was developed. The system enables users to view the task from their individual perspectives, have equal and natural access to the task, perceive the presence of local and distant members, communicate by verbal and nonverbal means and dynamically switch focus between the shared workspace and interpersonal communication space. A custom-designed cross-platform application programming interface (API) allows high and medium-level controls over the Scape workspace. It was stated that implementing an auto-collaborator class enables a developer to build a augmented virtual environments with minimal coding.
• Hua, H., Brown, L. D., & Gao, C. (2004). System and interface framework for SCAPE as a collaborative infrastructure. Presence: Teleoperators and Virtual Environments, 13(2), 234-250.
  More info

  Abstract: We have developed a multi-user collaborative infrastructure, SCAPE (an acnonym for Stereoscopic Collaboration in Augmented and Projective Environments), which is based on recent advancement in head-mounted projective display (HMPD) technology. SCAPE combines the functionalities of an interactive workbench and a room-sized immersive display to concurrently create both exocentric and egocentric perspectives. SCAPE intuitively provides a shared space in which multiple users can simultaneously interact with a 3D synthetic environment from their individual viewpoints, and each user has concurrent access to the environment from multiple perspectives at multiple scales. SCAPE also creates a platform to merge the traditionally separate paradigms of virtual and augmented realities. In this paper, we discuss the design principles we have followed to conceptualize the SCAPE system and briefly summarize SCAPE's hardware implementation. Furthermore, we discuss in detail the high-level design and implementation of the SCAPE architecture, and present a set of unique widget interfaces currently available in our implementation that enable and facilitate interaction and cooperation. Finally, we demonstrate SCAPEs unique visualization and interface capabilities via a testbed application-Aztec Explorer.
• Tan, K., Hua, H., & Ahuja, N. (2004). Multiview panoramic cameras using mirror pyramids. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(7), 941-946.
  More info

  PMID: 18579952;Abstract: A mirror pyramid consists of a set of planar mirror faces arranged around an axis of symmetry and inclined to form a pyramid. By strategically positioning a number of conventional cameras around a mirror pyramid, the viewpoints of the cameras' mirror images can be located at a single point within the pyramid and their optical axes pointed in different directions to effectively form a virtual camera with a panoramic field of view. Mirror pyramid-based panoramic cameras have a number of attractive properties, including single-viewpoint imaging, high resolution, and video rate capture. It is also possible to place multiple viewpoints within a single mirror pyramid, yielding compact designs for simultaneous multiview panoramic video rate imaging. Nalwa [4] first described some of the basic ideas behind mirror pyramid cameras. In this paper, we analyze the general class of multiview panoramic cameras, provide a method for designing these cameras, and present experimental results using a prototype we have developed to validate single-pyramid multiview designs. We first give a description of mirror pyramid cameras, including the imaging geometry, and investigate the relationship between the placement of viewpoints within the pyramid and the cameras' field of view (FOV), using simulations to illustrate the concepts. A method for maximizing sensor utilization in a mirror pyramid-based multiview panoramic camera is also presented. Images acquired using the experimental prototype for two viewpoints are shown.
• Brown, L. D., Hua, H., & Gao, C. (2003). A widget framework for augmented interaction in SCAPE. UIST: Proceedings of the Annual ACM Symposium on User Interface Softaware and Technology, 1-10.
  More info

  Abstract: We have previously developed a collaborative infrastructure called SCAPE - an acronym for Stereoscopic Collaboration in Augmented and Projective Environments that integrates the traditionally separate paradigms of virtual and augmented reality. In this paper, we extend SCAPE by formalizing its underlying mathematical framework and detailing three augmented Widgets constructed via this framework: CoCylinder, Magnifier, and CoCube. These devices promote intuitive ways of selecting, examining, and sharing synthetic objects, and retrieving associated documentary text. Finally we present a testbed application to showcase SCAPE's capabilities for interaction in large, augmented virtual environments. © 2003 ACM.
• Gao, C., Hua, H., & Ahuja, N. (2003). Easy calibration of a head-mounted projective display for augmented reality systems. Proceedings - Virtual Reality Annual International Symposium, 53-60.
  More info

  Abstract: Augmented reality (AR) superimposes computer-generated virtual images on the real world to allow users exploring both virtual and real worlds simultaneously. For a successful augmented reality application, an accurate registration of a virtual object with its physical counterpart has to be achieved, which requires precise knowledge of the projection information of the viewing device. This paper proposes a fast and easy off-line calibration strategy based on well-established camera calibration methods. Our method does not need exhausting effort on the collection of world-to-image correspondence data. All the correspondence data are sampled with an image based method and they are able to achieve sub-pixel accuracy. The method is applicable for all AR systems based on optical see-through head-mounted display (HMD), though we took a head-mounted projective display (HMPD) as the example. In this paper, we first review the calibration requirements for an augmented reality system and the existing calibration methods. Then a new view projection model for optical see through HMD is addressed in detail, and proposed calibration method and experimental result are presented. Finally, the evaluation experiments and error analysis are also included. The evaluation results show that our calibration method is fairly accurate and consistent.
• Hua, H., Brown, L. D., Gao, C., & Ahuja, N. (2003). A new collaborative infrastructure: SCAPE. Proceedings - Virtual Reality Annual International Symposium, 171-179.
  More info

  Abstract: This paper presents a multi-user collaborative infrastructure, SCAPE (an acronym for Stereoscopic Collaboration in Augmented and Projective Environments), which is based on recent advancement in head-mounted projective display (HMPD) technology. The SCAPE mainly consists of a 3′×5′ interactive workbench and a 12′×12′×9′ room-sized walk-through display environment, multiple head-tracked HMPDs, multi-modality interface devices, and a generic application-programming interface (API) designed to coordinate the components. The infrastructure provides a shared space in which multiple users can simultaneously interact with a 3D synthetic environment from their individual viewpoints. We detail the SCAPE implementation and include an application example that demonstrates major interface and cooperation features.
• Hua, H., Yonggang, H. a., & Rolland, J. P. (2003). Design of an ultralight and compact projection lens. Applied Optics, 42(1), 97-107.
  More info

  PMID: 12518828;Abstract: Driven by the need for lightweight head-mounted displays, we present the design of an ultralight and compact projection lens for a head-mounted projective display (HMPD). An HMPD consists of a pair of miniature projection lenses, beam splitters, and miniature displays mounted on the helmet and retroreflective sheeting materials placed strategically in the environment. The HMPD has been proposed recently as an alternative modality for three-dimensional visualization. After demonstrating the concept, building a first-generation custom-designed prototype, and investigating perception issues and application potentials, we designed an ultralight and compact projective lens with a diffractive optical element (DOE), plastic components, and aspheric surfaces for the next-generation prototype. The key contribution here lies in the conception, optimization, and assessment of the projection optics. Thus a brief review of the HMPD technology and related research is followed by a detail discussion of the conception and optimization of the ultralight and high-performance projection optics. The design of the DOE will be particularly described in detail. Finally, the diffraction efficiency of the DOE will be evaluated, and the overall performance of the optics will be assessed in both object space for the optical designer and visual space for possible end-users of the technology. © 2003 Optical Society of America.
• Hua, H., Gao, C., & Rolland, J. P. (2002). Study of the imaging properties of retro-reflective materials used in head-mounted projective displays (HMPDs). Proceedings of SPIE - The International Society for Optical Engineering, 4711, 194-201.
  More info

  Abstract: The concept of head-mounted projective display (HMPD) has been recently proposed as an alternative to conventional eyepiece-type head-mounted displays (HMDs). An HMPD consists of a pair of miniature projection lenses and flat panel displays mounted on the head and retro-reflective sheeting material placed strategically in the environment. Recent efforts have been made to demonstrate the feasibility of the imaging concept and prototypes have been built. Our research indicates that the quality and properties of the retro-reflective material play critical roles in the overall imaging quality of HMPDs. The retro-reflective sheeting material is commonly used in traffic control and photonic lighting systems, rather than optimized for imaging purpose as in the HMPDs. The size and shape of the microstructures cause artifacts on images. In this paper, we will mainly focus on the evaluation of the various existing retro-reflective materials, and the examination of the impact of the material characteristics on imaging properties. The basic structures of the existing materials are briefly reviewed, the characteristic parameters used to quantify the material properties are defined, and a few samples are evaluated. The characteristics of interest include: the size and shape of the microstructure, the distribution pattern and density of the microstructure, retro-reflectivity, the profile of the reflected light, diffraction artifacts and ghost imaging. Finally, a comprehensive analysis are presented to examine how the material characteristics play their roles in an imaging system, such as the HMPD, and predict the imaging artifacts caused by these characteristics.
• Hua, H., Gao, C., Brown, L., Biocca, F., & Rolland, J. P. (2002). Design of an ultra-light head-mounted projective display (HMPD) and its applications in augmented collaborative environments. Proceedings of SPIE - The International Society for Optical Engineering, 4660, 492-497.
  More info

  Abstract: Head-mounted displays (HMDs) are widely used for 3D visualization tasks such as surgical planning, scientific visualization, or engineering design. Even though the HMD technologies have undergone great development, tradeoffs in capability and limitation exist. The concept of head-mounted projective displays (HMPDs) is an emerging Technology on the boundary of conventional HMDs and projective displays such as the CAVE technology. It has been recently demonstrated to yield 3D visualization capability with potentially a large FOV, lightweight optics, low distortion, as well as correct occlusion of virtual objects by real objects. As such, the HMPD has been proposed as an alternative to stereoscopic displays for 3D visualization applications. In this paper, a brief review the HMPD technology is followed by the presentation of a recent design and implementation of a compact HMPD prototype based on an ultra-light design of projective optics using diffractive optical element (DOE) and plastic components. Finally, we will include applications of the HMPD technology being developed across three universities for augmented visualization tasks and distributed collaboration in augmented environments.
• Hua, H., Gao, C., Rolland, J. P., Brown, L. D., & Ahuja, N. (2002). A testbed for precise registration, natural occlusion and interaction in an augmented environment using a head-mounted projective display (HMPD). Proceedings - Virtual Reality Annual International Symposium, 81-89.
  More info

  Abstract: A head-mounted projective display (HMPD) consists of a pair of miniature projection lenses, beam splitters, and displays mounted on the helmet and retro-reflective sheeting materials placed strategically in the environment. It has been recently proposed as an alternative to existing 3D visualization devices. In this paper, we first briefly review the HMPD technology, including its featured capabilities and the recent development in both display implementations and applications. Then the implementation of a testbed, namely 'GO' game with a remote opponent in a 3D augmented environment," is described. The testbed not only demonstrates the capabilities of virtual-real augmentation and registration, natural occlusion of virtual objects by real, interaction with augmented environments, as well as networking collaboration, but also embodies part of our long-term objective to develop a collaborative framework in 3D augmented environments. Through the testbed, major calibration issues, such as accommodation/convergence considerations and determination of viewing transformations, are studied and discussed in detail. Both calibration methods and results are included, which are applicable to other applications. Finally, experimental results of the testbed implementation are presented.
• Hua, H., Wang, Y., & Yan, D. (2002). A low-cost dynamic range-finding device based on amplitude-modulated continuous ultrasonic wave. IEEE Transactions on Instrumentation and Measurement, 51(2), 362-367.
  More info

  Abstract: A practical low-cost acoustic range-finding method is presented, which provides a high-speed and high-accuracy solution for distance measurement. It is based on the linear phase delay of the received signal, with respect to the transmitted signal caused by the distance between the transmitter and the receiver. A continuous ultrasonic signal amplitude modulated by a low-frequency acoustic signal is used. The accuracy can be up to 2 mm within 1.5 m and the dynamic data update rate can be up to 150 Hz. The design principles, hardware implementation, experimental results, and performance analysis are included.
• Rolland, J., Davis, L., Ha, Y., Hamza-Lup, F., Vento, B. D., Gao, C., Hua, H., & Biocca, F. (2002). Head-mounted projective displays for creating distributed collaborative environments. Proceedings of SPIE - The International Society for Optical Engineering, 4711, 395-399.
  More info

  Abstract: In this paper, we shall present an overview of research in augmented reality technology and applications conducted in collaboration with the 3DVIS Lab and the MIND Lab. We present research in the technology of head-mounted projective displays and tracking probes. We then review mathematical methods developed for augmented reality. Finally we discuss applications in medical augmented reality and point to current developments in distributed 3d collaborative environments.
• Yonggang, H. a., Hua, H., Martins, R., & Rolland, J. (2002). Design of a wearable wide-angle projection color display. Proceedings of SPIE - The International Society for Optical Engineering, 4832, 67-73.
  More info

  Abstract: In this paper, we investigate the design and fabrication of ultra-light weight projection lenses for color wearable displays. Driven by field of view requirements from 40 degree to 90 degrees, we employed the combination of plastic, glass, and diffractive optics to yield
• Aggarwal, M., Hua, H., & Ahuja, N. (2001). On cosine-fourth and vignetting effects in real lenses. Proceedings of the IEEE International Conference on Computer Vision, 1, 472-479.
  More info

  Abstract: This paper has been prompted by observations of disparities between the observed fall-off in irradiance for off-axis points and that accounted for by the cosine-fourth and vignetting effects. A closer examination of the image formation process for real lenses revealed that even in the absence of vignetting a point light source does not uniformly illuminate the aperture, an effect known as pupil aberration. For example, we found the variation for a 16mm lens to be as large as 31% for a field angle of 10°. In this paper, we critically evaluate the roles of cosine-fourth and vignetting effects and demonstrate the significance of the pupil aberration on the fall-off in irradiance away from image center. The pupil aberration effect strongly depends on the aperture size and shape and this dependence has been demonstrated through two sets of experiments with three real lenses. The effect of pupil aberration is thus a third important cause of fall in irradiance away from the image center in addition to the familiar cosine-fourth and vignetting effects, that must be taken into account in applications that rely heavily on photometric variation such as shape from shading and mosaicing.
• Hua, H. (2001). Integration of eye tracking capability into optical see-through head-mounted displays. Proceedings of SPIE - The International Society for Optical Engineering, 4297, 496-503.
  More info

  Abstract: An overview is given on the brief survey of the eye tracking technologies that are suitable for head-mounted display (HMD) integration. An approach to integrating eye tracking technology into optical see-through type of HMDs is presented. Early engineering implementation and experimental results are also highlighted.
• Hua, H., & Ahuja, N. (2001). A high-resolution panoramic camera. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1, I960-I967.
  More info

  Abstract: Wide field of view (FOV) and high resolution are two desirable properties in many vision-based applications such as tele-conferencing, surveillance, and robot navigation. In some applications such as 3D reconstruction and rendering, it is also desired that all viewing directions share a single viewpoint, the entire FOV be imaged simultaneously, in real-time, and the depth of field be large. In this paper, we review such a panoramic camera proposed by Nalwa in 1996 that uses reflections off planar mirrors to achieve the first four of the aforementioned capabilities. He uses a single mirror pyramid (SMP) and a number of cameras that point to the individual pyramid faces. Together the cameras yield a visual field having a width of 360 degrees and a height same as that of the individual cameras. We propose a double mirror-pyramid (DMP) design that still achieves a 360-degree FOV horizontally but doubles the vertical FOV. It retains the other three capabilities namely high resolution, a single apparent viewpoint across the entire FOV, and real-time panoramic capture. We specify the visual field mapping from the scene to the sensor realized by the proposed camera. Finally, an implementation of the proposed DMP design is described and examples of preliminary panoramic images obtained are included.
• Hua, H., Gao, C., Biocca, F., & Rolland, J. P. (2001). An ultra-light and compact design and implementation of head-mounted projective displays. Proceedings - Virtual Reality Annual International Symposium, 175-182.
  More info

  Abstract: Head-mounted projective displays (HMPD) have been recently proposed as an alternative to conventional eyepiece-type head-mounted displays. HMPDs consist of a pair of miniature projection lenses and displays mounted on the helmet and retro-reflective sheeting materials placed strategically in the environment. Its novel concept and properties suggest solutions to part of the problems of state-of-art visualization devices and make it extremely suitable for multiple-user collaborative applications and wearable systems. In this paper, a brief review of conventional visualization techniques is followed by an extensive discussion of HMPD technology, which includes a summary of its features and a comparison with conventional head-mounted displays (HMDs), projection-based displays, and HMPDs. An ultra-light and compact design (i.e. 8g) of a projection lens system using diffractive optical element (DOE) as well as plastic components for a HMPD is presented. Through the usage of fast prototyping technology, a compact stereoscopic head-mounted prototype with weight less than 700 grams was implemented, and optomechanical adjustments and ergonomic considerations are discussed. Finally, the motivated application in multiuser tele-collaboration is described.
• Rolland, J. P., Ha, Y., Davis, L., Hua, H., Gao, C., & Biocca, F. (2001). A new paradigm for head-mounted display technology: Application to medical visualization and remote collaborative environments. Proceedings of SPIE - The International Society for Optical Engineering, 4442, 1-5.
  More info

  Abstract: Today advanced 3D virtual environments are mostly based on either a technology known as the cave or head-mounted displays. A new type of head-mounted display, which consists of a pair of miniature projection lenses and displays mounted on the helmet and retro-reflective sheeting materials placed strategically in the environment, has been proposed as an alternative to eyepiece optics types of displays. The novel concept and properties of the head-mounted projective display (HMPD) suggests solutions to part of the problems of state-of-art visualization devices and make it extremely suitable for multiple-user collaborative environments. In this paper, we first review the concept of the HMPD and present the latest prototype developed. We then discuss its application to medical visualization and remote collaborative environments.
• Chang, H., Wang, Y., Yan, D., Zhou, Y., Hua, H., & Tong, X. u. (2000). Tracking scheme based on color filtering for virtual reality. Guangxue Xuebao/Acta Optica Sinica, 20(9), 1224-1228.
  More info

  Abstract: An image processing method based on color filtering was proposed to realize hand tracking in a virtual reality system. Based on computer vision techniques, the method simulates the exclusive property of color vision of human eyes. Using the color of hand as a reference, the image received by CCD is processed to eliminate the images of the background and other objects. After smoothing and contour extraction, only the profile of the hand is left in the image. The method can eliminate the disturbance of the background and other objects. It can improve the reliability and increase the speed of hand recognition.
• Hua, H., Girardot, A., Gao, C., & Rolland, J. P. (2000). Engineering of head-mounted projective displays. Applied Optics, 39(22), 3814-3824.
  More info

  PMID: 18349957;Abstract: Head-mounted projective displays (HMPD's) are a novel type of head-mounted display. A HMPD consists of a miniature projection lens mounted upon the user's head and retroreflective sheeting material placed strategically in the environment. First, the imaging concept of a HMPD is reviewed and its potential advantages and disadvantages are discussed. The design and a bench prototype implementation are then presented. Finally, the effects of retroreflective materials on the imaging properties and the optical properties of HMPD's are comprehensively investigated. © 2000 Optical Society of America.
• Chang, H., Wang, Y., Hua, H., Tong, X. u., Zhou, Y., & Cheng, X. (1999). Hand tracking method based on computer vision. Beijing Ligong Daxue Xuebao/Transaction of Beijing Institute of Technology, 19(6), 739-743.
  More info

  Abstract: Aim To propose a method of realizing hand tracking in virtual reality systems. Methods Based on computer vision techniques, reflective markers were placed on the moving hand to simplify the recognition calculation and improve the reliability of the device. Images captured by a CCD camera were analyzed by the R, G, B value of the pixels in real-time. The positions of markers were calculated. The relationship of 3D position and 2D image of hand was analyzed and the positions of the palm and the fingers were automatically calculated. Results and Conclusion The method overcomes the drawbacks of traditional data-gloves which are cumbersome and fragile. Ordinary visible light source is used instead of infrared light sources in other studies to reduce the cost of the device.
• Hua, H., Wang, Y., & Tong, X. u. (1998). Dynamic range finding device using amplitude modulated continuous ultrasonic wave. Journal of Beijing Institute of Technology (English Edition), 7(1), 55-61.
  More info

  Abstract: A high speed and high resolution dynamic range finding device for the measurement of large distances was developed. It was comprised of an ultrasonic transmitter and a receiver. The amplitude of a continuous ultrasonic wave was modulated by a low frequency acoustic signal. The range finding was achieved by detecting the phase difference between the transmitted and received ultrasonic signals. The design principle, hardware implementation, experimental results and performance analysis of the device are included. Experiments show that the accuracy of the device can be up to 1 mm within 1.5 m while its dynamic data update rate can be up to 40 kHz.
• Zhang, S., Wang, Y., & Hua, H. (1998). Calculation of diffraction efficiency for a planar grating using rigorous coupled-wave theory. Guangdianzi Jiguang/Journal of Optoelectronics Laser, 9(1), 2-6, 12.
  More info

  Abstract: The basic concept and formulae of the rigorous coupled-wave theory were give. The diffraction of planar dielectric phase style harmonic grating was discussed when electromagnetic wave of planar polarization was incident upon. The method of using the theory to compute the diffraction efficiency of a planar grating was described, and the procedure of programming was discussed also. A numerical example shows that the computed results of energy distribution of all diffraction orders satisfy the law of energy conservation.
• Ding, H. e., Wang, Y., Yuan, X., & Hua, H. (1997). Optical viewing systems for virtual reality. Guangdian Gongcheng/Opto-Electronic Engineering, 24(5), 41-45.
  More info

  Abstract: An optical viewing system used in helmet mounted display (HMD) was described. Its design outlines were as follows: 1. lifelike stereoscopic vision; 2. visual field, it is a key factor of immersion; 3. resolution, the maximum resolution of human-eye is 1 mrad; 4. color; 5. eliminating distortion; 6. adjusting focus and convergence. Besides, the system is suitable for the great majority of people, and the light weight. The illusion created by the HMD display is amazing, if the present liquid crystal display and computer are immature.
• Ding, H. e., Tong, X. u., Wang, Y., Hua, H., & Ying, H. u. (1996). Head-mounted display system for virtual reality. Proceedings of SPIE - The International Society for Optical Engineering, 2892, 126-128.
  More info

  Abstract: A head-mounted-display system for virtual reality is developed, which is mainly comprised of a pair of viewing lenses together with LCDs to provide the stereoscopic image, and a tracking device to detect the motion of the head. Each viewing lens contains 4 optical elements, and can give a 120° field of view for each eye when used with a 2.2' LCD. The tracking device consists of a 3-axis fluxgate magnetometer and a pendulum, which determines the orientation angles of the helmet. Another version of the tracking device capable of measuring 6 degrees of freedom movement of the helmet is currently under development.
• Wang, Y., Yuan, X., Ding, H. e., Hua, H., & Lin, L. i. (1996). Design of a viewing lens for virtual reality. Proceedings of SPIE - The International Society for Optical Engineering, 2778(PART 1), 71-72.
  More info

  Abstract: The design principles of a viewing lens for v irtual reality are discussed, and a design example given. Used with a 2.2" colour LCD. The system gives a 12°c field of view for each eye. Monochromatic and chromatic aberrations are corrected for all the points in the field, while the large barrel distortion of the lens is compensated by a reverse compression ot the image on the LCD, The system can establish a good stereoscopic view with a strong sense of immersion. Copyright 1996,The Society of Photo-Optical Instrumentation Engineers.

Proceedings Publications

• Hua, H., & Huang, H. (2018, May). Design of a high-performance optical see-through light field head-mounted display. In SID Display Week 2018.
• Hua, H., & Wilson, A. (2018, April). High-resolution optical see-through vari-focal-plane head-mounted display using freeform Alvarez lenses. In SPIE Conference on Digital Optics for Immersive Displays.
• Hua, H., & Xu, M. (2018, April). Ultra-thin optical combiner with microstructure mirrors in augmented reality. In SPIE Conference on Digital Optics for Immersive Displays.
• Hua, H., & Xu, M. (2018, June). Systematic analysis method for multilayer light field display. In OSA 2018 Imaging and Applied Optics Congress (Topical Meeting on 3D Image Acquisition and Display: Technology, Perception, and Applications).
• Hua, H. (2016, March). Effects of configuration of optical combiner on near-field depth perception in optical see-through head-mounted displays. In 2016 IEEE Virtual Reality.
• Huang, H., & Hua, H. (2016, July). Design of an optical see-through multi-focal plane stereoscopic 3D display with eye-tracking ability. In OSA Topical Meeting on 3D Image Acquisition and Display.
• Lu, S., & Hua, H. (2016, July). Multifunctional three-dimensional microscopic system. In OSA Topical Meeting on 3D Image Acquisition and Display.
• Hua, H. (2014, August). Past and future of wearable augmented reality displays and their applications. In SPIE Annual Conference.
• Hua, H. (2014, June). Head-worn displays for augmented reality applications. In SID (Society of Information Display) Display Week Seminar.
• Hua, H. (2014, June). Optical design of an eyetracked head-mounted display using freeform waveguide. In 2014 OSA Classical Optics Congress-International Optical Design Conference.
• Hua, H. (2014, May). Eyetracked optical see-through head-mounted display as an AAC device. In Proc. of SPIE Defense, Security & Sensing—Three-Dimensional Imaging, Visualization, and Display.
• Hua, H., Hu, X., Gao, C., & Qin, X. (2014, 2014). Eyetracked optical see-through head-mounted display as an AAC device. In THREE-DIMENSIONAL IMAGING, VISUALIZATION, AND DISPLAY 2014, 9117.
• Lee, S., Hua, H., & Lok, B. (2014, 2010). Effects of Viewing Conditions and Rotation Methods in a Collaborative Tabletop AR Environment. In IEEE VIRTUAL REALITY 2010, PROCEEDINGS, 163-170.
• Liu, S., Pansing, C., Hua, H., Gregory, G., Howard, J., & Koshel, R. (2014, 2006). Design of a foveated imaging system using a two-axis MEMS mirror - art. no. 63422W. In International Optical Design Conference 2006, Pts 1 and 2, 6342, W3422-W3422.
• Hu, X., Hua, H., Woods, A., Holliman, N., & Favalora, G. (2013, February). An Optical See-through Multi-Focal-Plane Stereoscopic Display Prototype Enabling Nearly-Correct Focus Cues. In STEREOSCOPIC DISPLAYS AND APPLICATIONS XXIV, 8648.
• Oh, J., Hua, H., & , . (2013, 2006). User evaluations on form factors of tangible magic lenses. In 2006 IEEE/ACM INTERNATIONAL SYMPOSIUM ON MIXED AND AUGMENTED REALITY, 116-125.
• Qin, Y., Hua, H., & Nguyen, M. (2013, February). Development of a Laparoscope with Multi-Resolution Foveation Capability for Minimally Invasive Surgery. In DESIGN AND QUALITY FOR BIOMEDICAL TECHNOLOGIES VI, 8573.
• Zhang, R., Hua, H., Wu, M., & Lin, H. (2013, 2007). Design of a polarized head-mounted projection display using FLCOS microdisplays. In PROJECTION DISPLAYS XII, 6489.
• Hua, H., Barrett, H., Greivenkamp, J., & Dereniak, E. (2011, 2014). Past and future of wearable augmented reality displays and their applications. In FIFTY YEARS OF OPTICAL SCIENCES AT THE UNIVERSITY OF ARIZONA, 9186.
• Liu, S., Hua, H., Gao, C., & , . (2010, 2008). Illumination design of a multi-touch sensing projection screen for augmented virtual environments. In 2008 SID INTERNATIONAL SYMPOSIUM, DIGEST OF TECHNICAL PAPERS, VOL XXXIX, BOOKS I-III, 39, 174-177.
• Hua, H., Cheng, D., Wang, Y., Liu, S., Javidi, B., Son, J., Thomas, J., & Desjardins, D. (2008, 2010). Near-eye displays: state-of-the-art and emerging technologies. In THREE-DIMENSIONAL IMAGING, VISUALIZATION, AND DISPLAY 2010 AND DISPLAY TECHNOLOGIES AND APPLICATIONS FOR DEFENSE, SECURITY, AND AVIONICS IV, 7690.
• Hua, H., Gao, C., Woods, A., Holliman, N., & Favalora, G. (2008, 2012). A compact eyetracked optical see-through head-mounted display. In STEREOSCOPIC DISPLAYS AND APPLICATIONS XXIII, 8288.
• Hua, H., Liu, S., Wang, Y., Bentley, J., Du, C., Tatsuno, K., & Urbach, H. (2008, 2010). Depth-fused multi-focal plane displays enable accurate depth perception. In OPTICAL DESIGN AND TESTING IV, 7849.
• Liu, S., & Hua, H. (2008, March). Spatialchromatic Foveation for Gaze Contingent Displays. In PROCEEDINGS OF THE EYE TRACKING RESEARCH AND APPLICATIONS SYMPOSIUM (ETRA 2008), 139-142.
• Brown, L. D., Hua, H., & Lok, B. (2006, 2010). An Evaluation of Physical Affordances in Augmented Virtual Environments: Dataset Grounding and Magic Lens. In IEEE VIRTUAL REALITY 2010, PROCEEDINGS, 23-26.
• Lee, S., Hua, H., Coquillart, S., LaViola, J., & Schmalstieg, D. (2006, 2013). A Robust Camera-based Method for Optical Distortion Calibration of Head-Mounted Displays. In 2013 IEEE VIRTUAL REALITY CONFERENCE (VR), 27-30.

Presentations

• Hua, H. (2019, April). Evaluation of learning curve and peripheral awareness using a novel multiresolution foveated laparoscope. 2019 Spring Simulation Conference.
• Hua, H. (2019, April). Fundamentals of head-mounted displays for virtual and augmented reality. Applied Materials. San Jose, CA: Applied Materials.
• Hua, H. (2019, February). Optical challenges for scaling next-generation VR/AR displays. SPIE Photonic West. San Francisco.
• Hua, H. (2019, January). Modeling retinal image formation for light field displays. IS&T Electronic Imaging 2019. San Francisco.
• Hua, H. (2019, January). Optical challenges for next-generation VR/AR displays. Arizona Photonics Day. Tucson.
• Hua, H. (2019, January). Plenary: The quest for vision comfort: head-mounted light field displays for virtual and augmented reality. IS&T Electronic Imaging 2019. San Francisco: Electronic Imaging.
• Hua, H. (2019, June). Challenges of scaling in head-mounted displays. OSA Imaging and Applied Optics Congress. Munich, Germany: OSA.
• Hua, H. (2019, March). the Future of head-mounted displays. IEEE Virtual Reality 2019. Osaka, Japan: IEEE.
• Hua, H. (2019, November). Wearable displays as assistive technology. Tech Launch Arizona. College of Optical Sciences: Tech Launch Arizona.
• Hua, H. (2019, October). The quest for visual comfort in virtual and augmented reality displays. School of Information Colloquium, University of Arizona. School of Information Colloquium, University of Arizona: School of Information Colloquium, University of Arizona.
• Hua, H. (2019, September). Mutual occlusion in augmented reality displays. OSA Frontiers in Optics. Washington DC: OSA.
• Hua, H. (2018, April). Optical challenges for next-generation AR headsets. SPIE Conference on Digital Optics for Immersive Displays. Strasbourg.
• Hua, H. (2018, February). Head-mounted light field displays for virtual and augmented reality: challenges and opportunities. College of Sciences Distinguished Lecture Series, Rochester Institute of Technology. Rochester: Rochester Institute of Technology.
• Hua, H. (2018, January). Angular and Spatial Sampling Requirements in 3D Light Field Displays. IS&T Electronic Imaging 2018. San Francisco.
• Hua, H. (2018, January). Optimizing head-mounted light field displays for quality and comfort. SPIE Photonic West: VR,AR, MR One-day Industry Conference. San Francisco.
• Hua, H. (2018, November). Optical challenges for next-generation VR/AR displays. OPPO Technology Expo. Shengzhen, China: OPPO Inc..
• Hua, H. (2018, September). Head-mounted light field displays for virtual and augmented reality: challenges and opportunities. Colloquium at Food and Drug Administration. Washington DC.
• Hua, H. (2018, September). Optimizing head-mounted light field displays for quality and comfort. OSA Frontiers in Optics. Washington DC.
• Hua, H., & Huang, H. (2018, April). High-performance integral-imaging based light field augmented reality display. SPIE Conference on Digital Optics for Immersive Displays. Strasbourg.
• Hua, H., & Wilson, A. (2017, September). Towards occlusion-capable optical see-through head-mounted displays. OSA Frontiers in Optics/Laser Science Conference.
• Hua, H. (2017, April). Enabling focus cues in head-mounted displays. SPIE Commercial + Scientific Sensing and Imaging.
• Hua, H. (2017, August). Enabling Focus Cues in Head-mounted Displays for Virtual and Augmented Reality. Samsung Seminar. Seoul, South Korea.
• Hua, H. (2017, August). Head-mounted light field displays for virtual and augmented reality. 17th Internal Meeting on Information Display (IMID 2017). Busan, South Korea.
• Hua, H. (2017, February). Recent advances in head-mounted light field displays for virtual and augmented reality. SPIE Photonics West-Emerging Liquid Crystal Technologies XII.
• Hua, H. (2017, June). Enabling focus cues in head-mounted displays. OSA 2017 Imaging and Applied Optics Congress (Topical Meeting on Imaging Systems and Applications).
• Hua, H. (2017, May). Fundamentals of head-mounted displays for virtual and augmented reality. Applied Materials Seminar.
• Hua, H. (2017, May). Recent advances in head-mounted light field displays. Display Week 2017.
• Hua, H. (2017, November). Rendering focus cues in head-mopunted displays for virtual and augmented reality displays. College of Optical Sciences Colloquium.
• Hua, H., & Huang, H. (2017, June). Modeling of eye’s response in viewing 3D light field display. OSA 2017 Imaging and Applied Optics Congress (Topical Meeting on 3D Image Acquisition and Display: Technology, Perception, and Applications).
• Hua, H., & Huang, H. (2017, May). An integral-imaging-based head-mounted light-field display using a tunable lens and aperture array. Display Week 2017.
• Hua, H., & Xu, M. (2017, May). Dual-layer high dynamic range head mounted display. Display Week 2017.
• Hua, H., Kwan, E., & Qin, Y. (2017, June). Development of a light field laparoscope for depth reconstruction. OSA 2017 Imaging and Applied Optics Congress (Topical Meeting on 3D Image Acquisition and Display: Technology, Perception, and Applications).
• Hua, H. (2016, August). Head-worn displays for augmented reality applications. Applied Materials AR/VR Workshop. Santa Clara.
• Hua, H. (2016, July). Head-worn displays for augmented reality applications. OSA Topical Meeting on 3D Image Acquisition and Display.
• Hua, H. (2016, June). Near-to-eye displays for augmented reality. Corning Glass Summit.
• Hua, H. (2016, May). Near-to-eye lightfield displays for augmented reality. Huawei Science and Technology Workshop.
• Hua, H. (2016, September). Engineering challenges in optical see-through augmented reality displays. Corning AR/VR workshop. San Jose: Corning Inc..
• Hua, H. (2015, April). An integral imaging augmented reality display. SPIE-Three Dimensional Imaging, Visualization, and Display 2015.
• Hua, H. (2015, April). Review and current status of head-mounted display technologies,. SPIE-Defense Security and Sensing.
• Hua, H. (2015, February). Near-to-eye light field displays for augmented reality. Workshop on Light Field Imaging. Stanford University: Stanford University.
• Hua, H. (2015, June). Head-worn displays for augmented reality applications. SID Display Week Seminar.
• Hua, H. (2014, April). Advances in augmented reality displays. College of Optical Sciences Colloquium.
• Hua, H. (2014, Feb). DMD technology enables an optical see-through head-mounted 3D display with true focus cues. SPIE MOEMS-MEMS (Emerging Digital Micromirror Device Based Systems and Applications VI.
• Hua, H. (2014, June). Augmented reality display technology. Qualcomm.
• Hua, H. (2014, June). Wearable displays using freeform optics. 2014 OSA Classical Optics Congress-Computational Optical Sensing and Imaging.
• Hua, H. (2014, May). Effects of optical combiner on depth perception in optical see-through head-mounted displays. 2014 SPIE Defense, Security & Sensing—Head and helmet-mounted displays XIX: design and applications.

Poster Presentations

• Hua, H., & Wang, X. (2017, May). A digitally switchable multi-focal lens using freeform optics for wearable displays. Display Week 2017.
• Hua, H., & Wilson, A. (2017, May). Demonstration of an occlusion-capable optical see-through head-mounted display. Display Week 2017.

Reviews

• Hua, H. (2016. Advances in head-mounted light-field displays for virtual and augmented reality(p. 7).

Other Teaching Materials

• Hua, H. (2016. Fundamentals of head-mounted displays for virtual and augmented reality,. Display Week, Society of Information Displays.

Others

• Javidi, B., Martinez-Corral, M., Ferraro, P., Hua, H., Huang, Y. P., Matoba, O., Son, J. Y., & Stern, A. (2010, OCT). Untitled. JOURNAL OF DISPLAY TECHNOLOGY.