RealSense camera
Used an Intel RealSense D435i to collect aligned colour and depth frames.
Space robotics · Hardware-in-the-loop
Robotic testbeds for spacecraft proximity operations
My current ASTRO Lab gantry work and earlier spacecraft-vision research both use robotics to emulate tracking, approach, and capture during close-proximity space operations.
Robotic gantryYOLOv5Intel RealSenseCNN pose estimation
Current work · ASTRO Lab
A robotic gantry for close-proximity operations.
As part of my MSc work in ASTRO Lab, I am developing a robotic gantry testbed that can emulate close-proximity spacecraft operations on the ground. The goal is to move formation-control and relative-navigation work from simulation toward hardware-in-the-loop validation.
This continues the spacecraft-robotics work I began in 2022–2023 at York University in collaboration with MDA.
Earlier work · 2022–2023
Satellite tracking and grasping.
This project used machine learning and a robotic arm to capture a dummy satellite in a simulated space environment. It included arm path planning, gripper force sensing, position tracking, and orientation estimation.
My main role was the vision code: tracking the satellite and determining the position, velocity, and orientation of features that the robotic system could grasp.
My role
Vision code for position and orientation tracking.
YOLOv5 tracking
Implemented target detection and extracted three-dimensional position from the aligned depth data.
Quaternion CNN
Developed a custom convolutional neural network to estimate the satellite’s orientation as a quaternion.
Tracking pipeline
Position and orientation processing.
Capture RGB-D frames
Collect colour and depth information from the D435i.
Detect the target
Use YOLOv5 to locate the satellite in each frame.
Calculate position
Read the aligned depth frame to recover the target’s three-dimensional position.
Estimate orientation
Pass the cropped target image to the quaternion CNN.
Full project record
Tracking, grasping, and path planning.
The project combined path planning, robotic-gripper force sensing, position tracking, and orientation estimation to capture a dummy satellite in a simulated space environment. My primary role was the vision code used to determine the position, velocity, and orientation of graspable satellite features.
Path planning
End-effector waypoint interpolation.
The robotic-arm path is defined by a set of points, the instant at which the end effector must reach each point, and its velocity. Interpolating position and velocity between those points provides precise movement and alignment during grasping.
Force sensing
Robotiq gripper and force-sensitive resistors.
The force-sensing system uses a Robotiq gripper and force-sensitive resistors. Its resistance output corresponds to applied force and can be used to train the AI grasping node with enhanced tactile-sensing ability.
Position tracking code
YOLOv5 with aligned RGB and depth frames.
The Intel RealSense D435i supplies aligned RGB and depth frames. YOLOv5 detects the target in each colour frame, and positional data are recovered from the aligned depth information.
The tracker reads the camera’s intrinsic parameters, detects the target in each colour frame, projects the detection centre into the aligned depth frame to recover a three-dimensional position, and logs the resulting trajectory. Verification replays a recorded run against the commanded trajectory to measure tracking error.
Orientation estimation
Quaternion prediction from synthetic imagery.
The network predicts satellite orientation as a quaternion. Training and verification use imagery and labels produced with NVIDIA Deep Data Synthesizer.
Training data comes from NVIDIA Deep Data Synthesizer renders with known ground-truth attitude. Detections are cropped and resized to a fixed input, the network regresses a unit quaternion, and evaluation converts predicted and true quaternions to Euler angles to produce per-axis error statistics against the synthetic ground truth. The live pipeline runs position and orientation estimation together on the RealSense stream.
Project videos
Full demonstrations.
Repository
Public project code
The tracking scripts, verification tools, and supporting files are available in the SatTracking repository.
Open SatTracking on GitHub