3D Augmented Reality (AR) Platform

See our videos on www.youtube.com/haptica100 showing Augmented Reality in ProMIS Modules. See also our video showcasing our platform AR technology at http://www.youtube.com/watch?v=xXAX0Tf6jR0.

The ProMIS Simulator by Haptica is one of the only commercial, off-the-shelf 3D augmented reality products in the world.

Haptica’s core technology is based on Augmented Reality (AR). AR involves overlaying a graphical image on a real object to somehow enhance the experience of the user or the information they are getting from the video. AR is gaining popularity – especially with iPhone Apps and has been listed as a Top10 2010 Technology by journals ranging from the Economist to Wired to Time magazine.

Haptica’s strength is in having a commercial track record in AR: it underpins our commercial, off-the-shelf surgical simulator products for surgical training (see www.haptica.com). And the technology – running on a standard pc/laptop and webcam - has been developed as a platform to enable fast, efficient development. We are also different in that we track in 3D, i.e. we overlay 3D images and graphics on an object viewed in real-time video.

This video (http://www.youtube.com/watch?v=xXAX0Tf6jR0) aims to showcase our capability.  Donncha is 'trying-on' a virtual t-shirt and can customise the colour and logo of the t-shirt.

Augmented Reality

In ProMIS™, users interact with virtual and physical models in the same unit. Physical models give real haptics. Graphics and animation are super-imposed on physical models to provide instruction and realism.

For example, in the basic 'Object Positioning' task, graphical guide-marks show the user what to do - and indicate successful achievement of a target. In the LapColectomy Modules, 3D animation provides instruction and mimics events such as bleeding.

Virtual models are built from real video and textured to deliver realism. As the user interacts with those models, cameras track the instruments in 3D space.

Whether the Task is virtual or real, the cameras record and interpret the user's instrument handling to provide video and virtual playback as well as numerical data.

Instrument Tracking

ProMIS is a unique 'hybrid' simulator: its patented technology is a unique fusion of camera-based - vision - tracking systems, advanced video analysis, virtual reality and artificial intelligence. This 'Augmented Reality' technology enables Virtual Reality and Physical Reality to be used together in the same system.With Haptica's patented technology, instrument usage is precisely tracked and user feedback is provided as numeric data and graphical and video playback. ProMIS can also track objects and can automatically calculate, for example, the area of tissue dissected. (See Metrics below).

See our videos on www.youtube.com/haptica100 showing Instrument Tracking in ProMIS.

Read more on Haptica's core platform Augmented Reality technology here >>>

Metrics

ProMIS analyses performance by tracking instruments in 3-dimensional space. This patented technology is unique to ProMIS and enables real-time measurement of instruments in use.

All Tasks use scientifically validated metrics, including time taken, path length, and economy of movement.

Once a task is completed, the user is immediately presented with an analysis of their performance. Detailed metrics are given in summary, and for right and left hand as well as for each step of the Task. The user can also see a graphical playback that shows the exact track of their instruments during the Task. In physical Tasks, they may also view a video of their activity.

The images opposite show how ProMIS detects errors: in the first image, ProMIS tracks not only the instruments, but also the 'pegs' and identifies whether they were dropped. The second image shows a Dissection task: using vision technology, ProMIS automatically measures the area dissected. Click here to read SAGES paper on this >>>

Virtual Reality

ProMIS uniquely provides a mix of real and virtual reality. As well as using the combined technologies (Augmented Reality), ProMIS uses pure virtual reality in some tasks. For example, an Instrument Handling task enables users to use real instruments to interact with static and dynamic virtual targets.