Patria is trying out AR technology in the design and maintenance of equipment for the Aviation business unit and in the training of aircraft mechanics. AR would make it possible to minimise errors and enhance efficiency in design and training. Pilots would be able to influence design work early on in the process and the number of work phases could be reduced.
Text: Päivi Brink Photos: Lauri Oittinen, Patria
Augmented reality (AR) technology is at the cutting edge of aircraft design, maintenance and user training. Patria is testing and investigating how AR could be utilised and how mature the technology is for the requirements of the aviation industry.
“AR isn’t always necessary. For instance, if the maintenance instructions are very simple, this technology won’t help you to perform the tasks. However, if the instructions are highly detailed, then augmented reality comes in handy. Aircraft mechanics could then examine the object under maintenance while the instructions are simultaneously displayed on a terminal. For instance, if the maintenance involves one hundred items to be checked, AR can help you keep track of which ones you’ve already completed. This would enhance work efficiency and reduce errors,” says Lauri Oittinen, System Specialist.
Patria is currently testing out a variety of terminals for AR technology.
“Terminal technology has become advanced enough to enable the use of AR in production only in recent years. The simplest terminal device is a smartphone. The problem with a phone is that it’s awkward to use – you have to manually point it at the object. We’ve tested how well the app ensures that the AR content remains correctly positioned if you occasionally lay the phone down on the worktop. When you do this, the content is sometimes displayed incorrectly. That is not acceptable in the aviation industry.”
The most convenient AR terminals are AR glasses. Patria is trying out Microsoft’s HoloLens2, the state-of-the-art in AR technology.
“We’ve tested HoloLens2 in maintenance, for instance. It displays virtual instructions in the user’s field of vision, leaving their hands free. This comes in very useful in the small cockpit of the Hawk, for example. We traditionally rely on large instruction folders – the heavy folder gets in your way when you leaf through the instructions. AR-assisted maintenance requires digital instructions – the benefit is that they’re always updated to the latest version. The user can easily browse backwards and forwards by giving voice commands to the glasses,” says Oittinen.
AR-assisted training
AR technology could be used in the training of aircraft mechanics, designers and pilots. The systems are complex and learning them from block diagrams, for example, is challenging.
“We’ve tested the possibilities of using HoloLens2 in teaching how systems such as the Hornet landing gear work. On the glass screen, we can display the names of the parts of the landing gear and demonstrate how the fine mechanics function. The landing gear folds into a smaller space when retracted inside the plane. With the HoloLens2, we can virtually position ourselves under the plane to see how the hydraulics function when the landing gear is retracted,” says Oittinen.
HoloLens2 can also highlight points of interest.
“For instance, in maintenance training, the glasses can highlight parts that break frequently. When you’re a novice trainee, it’s particularly difficult to figure out where all the parts are. With the aid of AR, both trainees and experienced mechanics would be able to select the right items faster and the number of errors would decline.”
Using Varjo XR-1 to support design
On the design side, Patria has tried out Varjo XR-1, a Finnish mixed reality headset. It enables the user to examine 3D models while looking at a real object, such as an aircraft cockpit. Varjo XR-1 combines AR and VR technology – the user can switch over from augmented to virtual reality. For this reason, Varjo calls the XR-1 a “mixed reality” device. In the future, it will be possible to use the XR-1 to visualise the structure and features of an aircraft during different phases of design.
“The Aviation and Systems business units are working together to test the use of Varjo XR-1 in cockpit concept design. We visualise new cockpits using both the features of XR-1 and 3D models. As design progresses, we can show all the parties involved where the device components are located under the control panel. Pilots will be able to comment on the digital cockpit – which is presented to them in actual size right in front of their eyes in different lighting conditions – at an early stage. They will be able to easily notice if some aspect of the design hinders user comfort, visibility or functionality. In future, these comments will enable us to position cockpit technology with an eye on improving ergonomic efficiency.”
This will reduce future design phases, boost process efficiency and improve the end result.
“However, it’ll still take some time before we can harness this system due to its complexity. AR is still so new that we haven’t had much experience of its reliability. The use of AR will grow substantially in the future, but time will tell in which environments it’ll be most useful,” says Oittinen.
