Ispezione di lamiere metalliche in Realtà Aumentata su HoloLens 2: architettura client-server con segmentazione tramite Intelligenza Artificiale

In Industry 4.0, quality control is still one of the few steps left almost entirely to the human operator: inspecting a metal sheet means constantly looking away from the physical piece to check a technical sheet or a nearby monitor, a continuous context switching that slows down inspection and increases the risk of error, especially during longer shifts. Augmented Reality on headsets like the HoloLens 2 or Magic Leap could solve this by projecting operational information directly onto the physical object, but the headset alone lacks both the computing power and the cooling capacity needed to run an AI model able to recognize a metal sheet's shape in real time, without triggering thermal throttling and a drop in frame rate. This thesis proposes a system that solves the problem by splitting the work between the headset and an external server (computational offloading). The client, built in Unity for the HoloLens 2, captures an image, communicates with the server asynchronously through Coroutines coordinated by a finite state machine, and rebuilds the sheet's holographic mesh at runtime with a Triangle Fan algorithm; it also handles the placement of defect markers (Pins) and a floating panel interface (TagAlong) designed not to tire the operator during long inspections. The server, written in Python, has the photo segmented by FastSAM accelerated with NVIDIA TensorRT, extracts its contour with OpenCV, validates its geometry to flag anomalous shapes, and computes the piece's real-world scale. To recognize and anchor the hologram to the sheet, the system offers two alternative, mutually exclusive scanning modes, chosen by the operator before starting: with the QR code, it uniquely identifies the piece through an alphanumeric string (e.g. LAM1, LAM2), uses the marker's known size to compute the scale, and uses that same identifier server-side to pull an already-processed inspection from cache; with Vuforia's Image Targeting, instead, the headset optically recognizes the piece through its feature points, with no QR code in frame at all - the system understands that there is a sheet to work on, but it cannot tell two visually identical pieces apart, since unique identification remains an exclusive feature of the QR path. Using Vuforia also required solving a deeper conflict: the library tends to take over control of the user's position in space as well, competing with OpenXR's native SLAM to establish the same absolute coordinates and causing the hologram to drift and jitter. The problem was resolved by limiting Vuforia to local optical marker recognition only, leaving environmental stability entirely to OpenXR; this conflict never arises on the QR path, which is natively integrated into the same stack. Once the mask has been generated, its position and proportions can finally be fine-tuned through a dedicated manual calibration panel, to correct any small inaccuracies before confirming the inspection. The system was validated by actually installing it on the headset and testing it on real metal sheets in the lab, not just in the editor or in simulation. Testing confirmed inference times compatible with real-time use and put optical tracking to the test on reflective metal surfaces, highlighting the practical limits of Computer Vision on textureless materials. Usability tests on defect placement and management, contextual menus and history browsing further confirmed the application's robustness and overall usability. Overall, the work shows that splitting the workload between headset and server makes it possible to bring AI-assisted Augmented Reality into a real industrial inspection setting, laying solid groundwork for future deployment in an Industry 4.0 context.