HOLOMEDUnity AR App

This Augmented Reality (AR) app was produced as a supporting application for my 3rd year Design and Manufacture group project. As a group of 5 we were tasked with manufacturing a mechanical arm attachment to enable the precise positioning of a specially designed steerable needle for "minimally invasive treatment in neurosurgery", courtesy of the EDEN2020 project.

As an additional part of the project an AR app was to be created which assisted the neurosurgical procedure by allowing the surgeon to:

1. Visualise the internal structures of the head
2. Crop out the most viable area of entry with a tracked stylus which would then be processed by a path planning algorithm (Courtesy of the EDEN2020 team) providing the 10 most optimal paths to the tumor from the entry region
3. Choose the best path from amongst the 10 optimal paths
4. Finally align the mechanical arm attachment (trochar holder) with the aid of a guiding system shown via the app

The app was created using Unity/C# and used the Vuforia AR engine to enable the realisation of the above features. My role was to create the UI features guiding the surgeon through all 4 stages of the procedure.

Important Note: Due to the Covid lockdown the project could not use the envisioned Infrared (IR) tracking system provided by the EDEN 2020 team which dealt a huge blow to the project. However, in order to present the potential promise of the application within the limitations of our homes, we decided to replace the IR tracking system with that of a card tracking system as shown in the demo below.

What did I learn?

• How to use C# within the development of a Unity app
• How to use the Vuforia AR Engine to display 3d models onto the camera view
• The use of objects, classes (private and public), arrays, lists, public and private variables
• Creating scenes and UI features within Unity apps
• Introduction to cloud version control via the Unity Hub collaborative features
• 3D modelling nuances such as vectors and quartenions
• How to deploy a live Unity app
• The importance of researching into whether the app will actually run on the intended device, especially given the requirements of AR features