Master Thesis: Multimodal Sensor Fusion and Robust Odometry for XR Perception

Join our Team

About this opportunity:

We are looking for a motivated Master's student to develop a robust perception and motion-estimation system for an XR platform operating in visually degraded environments.

The thesis will investigate how complementary sensing modalities—such as RGB or stereo cameras, infrared or thermal cameras, LiDAR or depth sensors, and an IMU—can be combined to improve pose estimation, depth perception, and 3D reconstruction when individual sensors may fail.

The work will contribute to the NEXSoS XR project, with a focus on challenging conditions such as darkness, smoke, low texture, motion blur, and partial sensor degradation. You will build on existing algorithms and open-source frameworks where appropriate, while developing the integration, calibration, synchronisation, and evaluation methodology needed for a practical multimodal system.

What you will do:

- Review approaches for visual-inertial odometry, LiDAR odometry, SLAM, and multimodal sensor fusion, and define a suitable architecture for the NEXSoS XR platform.
- Integrate available sensing hardware and develop calibration, synchronisation, preprocessing, and coordinate-transformation components.
- Implement or adapt a sensor-fusion and odometry pipeline using frameworks such as ORB-SLAM3, RTAB-Map, or other relevant methods.
- Evaluate the system under different environmental conditions.
- Compare individual sensors with multimodal configurations in terms of accuracy, robustness, latency, and computational requirements.
- Analyse and visualise trajectories, point clouds, depth maps, and 3D reconstructions.
- Document system limitations and provide recommendations for future development.

The skills you bring:

- You are enrolled in or recently admitted to a Master's programme in Robotics, Computer Science, Electrical Engineering, or a related field.
- You have basic knowledge of computer vision, robotics, estimation, 3D perception, or signal processing.
- You have programming experience in Python and/or C++.
- You understand coordinate transformations, rigid-body motion, and sensor data processing at a basic level.
- You are interested in working with real sensors, experimental hardware, and research software.
- You can analyse experimental results and communicate technical findings clearly.

The following are considered a plus:

- Experience with visual, inertial, or LiDAR odometry, or SLAM.
- Familiarity with ROS/ROS 2, ORB-SLAM3, RTAB-Map, OpenCV, Open3D, PCL, or similar tools.
- Knowledge of Kalman filtering, factor graphs, nonlinear optimisation, or bundle adjustment.
- Experience with camera–IMU or LiDAR–camera calibration.
- Previous experience with thermal, infrared, depth, or LiDAR sensors.