Today's surgical procedures are characterized by a high degree of complexity. This is the result of continuously increasing demands that need to be fulfilled by the performing surgeon. Thus, increasingly finer anatomical structures are subject of surgical therapies performed. At the same time, the aim is to keep the postoperative trauma of patients at a minimum. For this reason, minimally invasive surgery has gained great importance. Interventions of this kind are distinguished from conventional surgery by small incisions as well as the use of natural means of access. As a result, the recovery period is shortened and less painful for the patient. On the other hand, the surgeon's view of the operative field is limited, resulting in prolonged operative time and delayed access times in case of complications. This is due to the fact that the visual control of the operation by the surgeon is performed on the basis of an endoscopic image.

Furthermore, the surgeon is provided with data of different medical imaging modalities. Computed tomography, that primarily delivers a visualization of bony structures, and magnetic resonance imaging, which is used for the diagnosis of soft tissue, are the most commonly used ones. Based on these datasets, the preoperative planning of the intervention is performed. It comprises of the localization of anatomical risk structures such as nerves or blood vessels, and the planning of access paths. The achieved medical planning data must be transferred into the operating theatre and has to be displayed to the surgeon during surgery.

Additionally, navigation systems enable the determination of the position and orientation (pose) of medical instruments relative to the patient in the operating room. By merging this additional information with the pre-acquired planning data, navigation instructions can be generated and presented to the surgeon. These include visualizations of cut- and drill-marks, anatomical structures as well as alignment representations.

The biggest challenge in this context lies not in the acquisition of the data themselves, but rather in the choice of an appropriate representation. For this purpose, the use of monitor systems has been established. Problematic is the simultaneous display of a variety of visual information. Therefore, the surgeon must train to deal with such systems first. On the other hand, the patient gets out of focus through the concentration of the surgeon on a monitor. The exchange of glances between the patient and the monitor leads to rapid exhaustion and loss of accuracy. A further complication is that a hand-eye coordination is not intuitively possible. Along with the demanding handling of the micro-tools used for minimally invasive surgery, significant stress of the surgeon during the intervention results.

For the purpose of optimizing the presentation of information, augmented reality approaches are the subject of current research. In the field of medical technology, the goal is the visualization of data directly in the surgeon's field of view. This approach is based on a simultaneous representation of reality, in this case the patient, and additional visual information. Using a mini-projector and an optical navigation system, a handheld projection device was built up, enabling the projection of anatomical structures as well as navigation instructions in real time directly onto the surface of the patient.

Today's surgical procedures are characterized by a high degree of complexity. This is the result of continuously increasing demands that need to be fulfilled by the performing surgeon. Thus, increasingly finer anatomical structures are subject of surgical therapies performed. At the same time, the aim is to keep the postoperative trauma of patients at a minimum. For this reason, minimally invasive surgery has gained great importance. Interventions of this kind are distinguished from conventional surgery by small incisions as well as the use of natural means of access. As a result, the recovery period is shortened and less painful for the patient. On the other hand, the surgeon's view of the operative field is limited, resulting in prolonged operative time and delayed access times in case of complications. This is due to the fact that the visual control of the operation by the surgeon is performed on the basis of an endoscopic image.

Furthermore, the surgeon is provided with data of different medical imaging modalities. Computed tomography, that primarily delivers a visualization of bony structures, and magnetic resonance imaging, which is used for the diagnosis of soft tissue, are the most commonly used ones. Based on these datasets, the preoperative planning of the intervention is performed. It comprises of the localization of anatomical risk structures such as nerves or blood vessels, and the planning of access paths. The achieved medical planning data must be transferred into the operating theatre and has to be displayed to the surgeon during surgery.

Additionally, navigation systems enable the determination of the position and orientation (pose) of medical instruments relative to the patient in the operating room. By merging this additional information with the pre-acquired planning data, navigation instructions can be generated and presented to the surgeon. These include visualizations of cut- and drill-marks, anatomical structures as well as alignment representations.

The biggest challenge in this context lies not in the acquisition of the data themselves, but rather in the choice of an appropriate representation. For this purpose, the use of monitor systems has been established. Problematic is the simultaneous display of a variety of visual information. Therefore, the surgeon must train to deal with such systems first. On the other hand, the patient gets out of focus through the concentration of the surgeon on a monitor. The exchange of glances between the patient and the monitor leads to rapid exhaustion and loss of accuracy. A further complication is that a hand-eye coordination is not intuitively possible. Along with the demanding handling of the micro-tools used for minimally invasive surgery, significant stress of the surgeon during the intervention results.

For the purpose of optimizing the presentation of information, augmented reality approaches are the subject of current research. In the field of medical technology, the goal is the visualization of data directly in the surgeon's field of view. This approach is based on a simultaneous representation of reality, in this case the patient, and additional visual information. Using a mini-projector and an optical navigation system, a handheld projection device was built up, enabling the projection of anatomical structures as well as navigation instructions in real time directly onto the surface of the patient.