{"id":135676,"date":"2026-11-11T06:22:18","date_gmt":"2026-11-11T06:22:18","guid":{"rendered":"https:\/\/www.seeedstudio.com\/blog\/?p=135676"},"modified":"2026-11-11T06:28:33","modified_gmt":"2026-11-11T06:28:33","slug":"mobile-robot-full-stack-development-in-practice-m4-jetson-ai-vision","status":"publish","type":"post","link":"https:\/\/www.seeedstudio.com\/blog\/2026\/11\/11\/mobile-robot-full-stack-development-in-practice-m4-jetson-ai-vision\/","title":{"rendered":"Mobile Robot Full-stack Development In Practice \u2163: How Does a Robot Understand What It Sees? YOLO, ByteTrack, SegFormer &#038; FoundationPose"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\"><em>From object detection and multi-object tracking to semantic segmentation and 6D pose estimation on NVIDIA Jetson with ROS 2.<\/em><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In Part III, we looked at how a mobile robot estimates its motion by bringing LiDAR, IMU, wheel feedback and GNSS into a consistent sensor-fusion workflow. But knowing how the robot is moving is only part of the picture. It also needs to interpret what its cameras observe: identify objects, keep their identities across frames, distinguish relevant regions, and estimate the pose of objects it needs to interact with.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A detection box, a track ID, a pixel-level class map and a 6D pose are different kinds of information. Each answers a different question, relies on different inputs and needs its own validation. This Module 04 guide follows the AI-perception experiments on<a href=\"https:\/\/www.seeedstudio.com\/reComputer-Robotics-J5011-with-GMSL-extension-board-p-6681.html\"> <strong>reComputer Robotics J501<\/strong><\/a>, focusing on how model outputs are inspected and passed into ROS 2 interfaces that other robot components can use.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>What you will learn:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Deploy YOLO11 with TensorRT and verify detection boxes, coordinates and ROS 2 messages.<\/li>\n\n\n\n<li>Use ByteTrack to maintain object identities across frames and investigate ID changes.<\/li>\n\n\n\n<li>Use SegFormer to interpret pixel-level classes and derive a task-specific ground-candidate mask.<\/li>\n\n\n\n<li>Estimate a known object&#8217;s 6D pose from recorded RGB-D data with FoundationPose, then understand the transform into the robot frame.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Before you start:<\/strong> basic familiarity with ROS 2 topics, terminal commands and the earlier Jetson development environment will help. You do not need to run all four experiments at once; each section can be explored and validated independently.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><a href=\"https:\/\/github.com\/Seeed-Projects\/mobile-robot-full-stack-course\"><img fetchpriority=\"high\" decoding=\"async\" width=\"1030\" height=\"290\" src=\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/course-overview-1-1030x290.png\" alt=\"Official Mobile Robot Full-Stack Development course roadmap, from perception and mapping to manipulation and deployment\" class=\"wp-image-135688\" srcset=\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/course-overview-1-1030x290.png 1030w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/course-overview-1-300x84.png 300w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/course-overview-1-765x215.png 765w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/course-overview-1-32x9.png 32w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/course-overview-1-1536x432.png 1536w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/course-overview-1-2048x576.png 2048w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/course-overview-1-1021x287.png 1021w\" sizes=\"(max-width: 1030px) 100vw, 1030px\" \/><\/a><figcaption class=\"wp-element-caption\">Mobile Robot Full-Stack Development Course Roadmap<\/figcaption><\/figure>\n<\/div>\n\n\n<p class=\"wp-block-paragraph\">If this is your first time joining the series, start with the <a href=\"https:\/\/www.seeedstudio.com\/blog\/2026\/09\/18\/mobile-robot-full-stack-development-course-introduction\/\">course introduction<\/a> for the complete learning roadmap. You can also revisit <a href=\"https:\/\/www.seeedstudio.com\/blog\/2026\/09\/30\/jetson-powered-mobile-robot-full-stack-development-in-practice-module-1-jetson-ros2\/\">Part I: the Jetson and ROS 2 development foundation<\/a>, <a href=\"https:\/\/www.seeedstudio.com\/blog\/2026\/10\/16\/mobile-robot-full-stack-development-in-practice-module-2-gmsl-calibration-bev\/\">Part II: camera integration and calibration<\/a>, and <a href=\"https:\/\/www.seeedstudio.com\/blog\/2026\/10\/23\/mobile-robot-full-stack-development-m3-robot-sensor-fusion-with-ros-2\/\">Part III: LiDAR, IMU and sensor fusion<\/a>. The <a href=\"https:\/\/github.com\/Seeed-Projects\/mobile-robot-full-stack-course\">open-source course repository<\/a> contains the full documentation, code and updates.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Start with the outputs.<\/strong> The examples below show what each experiment produces. Keep the input fixed and learn to explain each result before adding another component.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><img decoding=\"async\" width=\"1030\" height=\"820\" src=\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/11-results-overview-1-edited-1030x820.png\" alt=\"Four original M04 examples showing YOLO detection boxes, ByteTrack IDs, SegFormer ground selection and a known object pose on recorded RGB-D\" class=\"wp-image-135691\" srcset=\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/11-results-overview-1-edited-1030x820.png 1030w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/11-results-overview-1-edited-300x239.png 300w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/11-results-overview-1-edited-768x611.png 768w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/11-results-overview-1-edited-1536x1222.png 1536w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/11-results-overview-1-edited-2048x1630.png 2048w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/11-results-overview-1-edited-32x25.png 32w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/11-results-overview-1-edited-1024x815.png 1024w\" sizes=\"(max-width: 1030px) 100vw, 1030px\" \/><figcaption class=\"wp-element-caption\">Mobile Robot Full-Stack Development Course M04 Preview<\/figcaption><\/figure>\n<\/div>\n\n\n<p class=\"wp-block-paragraph\"><strong>Jump to:<\/strong> <a href=\"#deploy-yolo-with-tensorrt-and-inspect-its-ros-2-output\">YOLO deployment<\/a> \u00b7 <a href=\"#track-the-same-object-across-frames-with-bytetrack\">Object tracking<\/a> \u00b7 <a href=\"#use-segformer-to-separate-pixel-classes-from-ground-candidates\">Segmentation<\/a> \u00b7 <a href=\"#estimate-a-known-objects-6d-pose-with-foundationpose\">6D pose<\/a> \u00b7 <a href=\"#choose-the-next-experiment-and-find-the-course-code\">Course code<\/a><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Match the Perception Task to the Information You Need<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Imagine a robot entering a work area. It may need to identify nearby objects, keep observing one as it moves, distinguish ground from surrounding regions, or determine the position and orientation of a known part. These requirements lead to different tasks:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Object detection \u2014 YOLO:<\/strong> predict classes, 2D boxes and confidence scores from an RGB image. The course deploys the model through PyTorch, ONNX and TensorRT, then publishes detections as ROS 2 messages.<\/li>\n\n\n\n<li><strong>Multi-object tracking \u2014 ByteTrack:<\/strong> associate detections across frames using motion prediction and matching, assigning persistent track IDs.<\/li>\n\n\n\n<li><strong>Semantic segmentation \u2014 SegFormer-B0:<\/strong> predict a class for every pixel, then select configured classes to form a ground-candidate mask.<\/li>\n\n\n\n<li><strong>6D pose estimation \u2014 FoundationPose:<\/strong> combine RGB-D, camera intrinsics, a target-instance mask and a CAD mesh to estimate a known object&#8217;s 3D translation and rotation in the camera frame.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Three Distinct Perception Data Paths<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Detection and tracking<\/strong> form one connected path: RGB images feed YOLO, its detections go to <code>\/perception\/detections<\/code>, and ByteTrack produces identified detections on <code>\/perception\/tracks<\/code>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Segmentation<\/strong> runs independently on RGB images. SegFormer produces pixel classes on <code>\/perception\/semantic_mask<\/code>; a class-selection rule generates <code>\/perception\/drivable_mask<\/code>. Semantic labels do not identify individual object instances, and selected ground pixels do not establish that a robot can safely drive there.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Object pose<\/strong> requires RGB-D and <code>CameraInfo<\/code>, a mask for the particular target instance, and its mesh. FoundationPose generates pose hypotheses, refines them and scores candidates. A detection box or a semantic class mask is not a substitute for that instance mask. The resulting camera-frame pose needs a coordinate transform before use in the robot&#8217;s base frame.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><img decoding=\"async\" width=\"697\" height=\"1030\" src=\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/12-perception-routes-edited-697x1030.png\" alt=\"Three separate perception data paths: YOLO to Byte Track, SegFormer to selected ground classes, and RGB-D with an instance mask and mesh to FoundationPose and a calibrated robot-frame transform\" class=\"wp-image-135693\" srcset=\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/12-perception-routes-edited-697x1030.png 697w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/12-perception-routes-edited-203x300.png 203w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/12-perception-routes-edited-768x1135.png 768w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/12-perception-routes-edited-1040x1536.png 1040w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/12-perception-routes-edited-1386x2048.png 1386w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/12-perception-routes-edited-22x32.png 22w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/12-perception-routes-edited-1024x1513.png 1024w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/12-perception-routes-edited.png 1520w\" sizes=\"(max-width: 697px) 100vw, 697px\" \/><figcaption class=\"wp-element-caption\">Detection and tracking share one path. Segmentation and known-object pose use separate inputs and outputs; they are not successive models in one pipeline<\/figcaption><\/figure>\n<\/div>\n\n\n<p class=\"wp-block-paragraph\">The course offers two implementations of the pose task: <strong>NVIDIA Isaac ROS FoundationPose in M4.4<\/strong> and <strong>native NVlabs FoundationPose in M4.5<\/strong>. They are different engineering routes, rather than consecutive models in one pipeline.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Prepare the Jetson Platform and Inputs<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The course reference platform is <strong>Seeed reComputer Robotics J501 with NVIDIA Jetson AGX Orin 32GB<\/strong>. Match its documented environment before changing versions:<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><thead><tr><th>Component<\/th><th>Course configuration<\/th><\/tr><\/thead><tbody><tr><td>Hardware<\/td><td>reComputer Robotics J501<\/td><\/tr><tr><td>JetPack \/ L4T<\/td><td>6.2.1 \/ R36.4.4<\/td><\/tr><tr><td>Ubuntu<\/td><td>22.04<\/td><\/tr><tr><td>ROS 2<\/td><td>Humble<\/td><\/tr><tr><td>CUDA<\/td><td>12.6<\/td><\/tr><tr><td>TensorRT<\/td><td>10.3.x<\/td><\/tr><tr><td>Isaac ROS<\/td><td>3.2<\/td><\/tr><tr><td>Python<\/td><td>3.10<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">The front GMSL camera supplies images through <code>\/dev\/video0<\/code>. Pose exercises also use recorded RGB-D data; connecting a live RGB-D camera introduces additional alignment and synchronization work. The course previews below use different inputs: indoor detection, recorded road video for tracking and segmentation, and the Mustard RGB-D sequence for pose.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Install dependencies, build the workspace and prepare the model assets before launching an exercise. The repository does not contain every weight, engine or mesh. The maintained setup instructions and lesson links are collected in the course navigation at the end.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">After that preparation, set the code-directory path for your checkout. New inspection terminals also need the ROS and workspace environments:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>export M4_CODE_ROOT=\"$HOME\/mobile-robot-full-stack-course\/docs\/M04-AI-Vision-And-Edge-Acceleration\/code\"\ncd \"$M4_CODE_ROOT\"\nsource \/opt\/ros\/humble\/setup.bash\nsource \"$M4_CODE_ROOT\/ros2_ws\/install\/setup.bash\"\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">For parameter comparisons, keep the input, resolution, model and configuration fixed. Change one setting at a time so that a new scene or camera geometry does not obscure the effect.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Deploy YOLO with TensorRT and Inspect Its ROS 2 Output<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Understand the Candidates Before Filtering Them<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The course&#8217;s YOLO11n COCO export takes a 640 \u00d7 640 image and returns <code>[1, 84, 8400]<\/code>: one batch, four box values plus 80 class scores, and 8,400 candidate locations from <code>80\u00d780 + 40\u00d740 + 20\u00d720<\/code>. Other models and export modes may use different output contracts.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Post-processing reads the class scores and box coordinates, filters low-confidence candidates, applies class-wise non-maximum suppression (NMS), and restores boxes to the original image. Confidence controls which predictions survive; the NMS IoU threshold controls duplicate suppression. A confidence of 0.8 is not an 80% measured accuracy rate.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"889\" height=\"1030\" src=\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/06-yolo-postprocess-edited-889x1030.png\" alt=\"YOLO post-processing from candidate decoding through confidence filtering, class-wise NMS and coordinate recovery to ROS 2 messages\" class=\"wp-image-135695\" srcset=\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/06-yolo-postprocess-edited-889x1030.png 889w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/06-yolo-postprocess-edited-259x300.png 259w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/06-yolo-postprocess-edited-768x890.png 768w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/06-yolo-postprocess-edited-1325x1536.png 1325w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/06-yolo-postprocess-edited-28x32.png 28w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/06-yolo-postprocess-edited-1024x1187.png 1024w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/06-yolo-postprocess-edited.png 1520w\" sizes=\"(max-width: 889px) 100vw, 889px\" \/><\/figure>\n<\/div>\n\n\n<p class=\"wp-block-paragraph\">With <code>yolo11n.pt<\/code> and the required dependencies prepared, export and build the engine:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>yolo export model=yolo11n.pt format=onnx simplify=True dynamic=False\npython3 -c \"import onnx; onnx.checker.check_model(onnx.load('yolo11n.onnx'))\"\n\/usr\/src\/tensorrt\/bin\/trtexec \\\n  --onnx=yolo11n.onnx \\\n  --saveEngine=yolo11n_fp16.engine \\\n  --fp16 --memPoolSize=workspace:4096\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">These steps export a static graph, check its structure and build an FP16 engine for the course&#8217;s TensorRT 10.3 environment. They do not retrain the detector. Place the engine and labels according to the node configuration, and recheck compatibility when changing hardware, runtime or export settings. Detecting objects outside the pretrained classes requires a separate data and training workflow.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Correct Class, Displaced Box? Check Letterbox Geometry<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Letterbox preprocessing preserves aspect ratio and pads the image to the model input size. To map a predicted point back, subtract the padding first, then divide by the scale saved during preprocessing.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"790\" height=\"1030\" src=\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/07-letterbox-edited-790x1030.png\" alt=\"Letterbox geometry showing a 1920 by 1080 image scaled to 640 by 360 with 140-pixel top and bottom padding and equations for reversing the transformation\" class=\"wp-image-135697\" srcset=\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/07-letterbox-edited-790x1030.png 790w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/07-letterbox-edited-230x300.png 230w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/07-letterbox-edited-768x1001.png 768w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/07-letterbox-edited-1179x1536.png 1179w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/07-letterbox-edited-25x32.png 25w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/07-letterbox-edited-1024x1335.png 1024w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/07-letterbox-edited.png 1520w\" sizes=\"(max-width: 790px) 100vw, 790px\" \/><\/figure>\n<\/div>\n\n\n<p class=\"wp-block-paragraph\">For example, 1920 \u00d7 1080 becomes 640 \u00d7 360, with 140 pixels above and below. The original center <code>(960, 540)<\/code> maps to <code>(320, 320)<\/code> in the padded input. Omitting the padding subtraction shifts the recovered box. Use the actual scale and padding saved during preprocessing, including rounding; also check BGR\/RGB, normalization and HWC\/NCHW layout.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Check the Message as Well as the Overlay<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">From the prepared code directory, start the detector:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>.\/scripts\/m4\/run_m4_1_demo.sh\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">In another sourced terminal, inspect the interface and QoS, publishing rate, then the message itself:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ros2 topic info -v \/perception\/detections\nros2 topic hz \/perception\/detections\nros2 topic echo \/perception\/detections --once<\/code><\/pre>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1030\" height=\"584\" src=\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/detection-1030x584.png\" alt=\"Course YOLO detection preview with class labels and bounding boxes over and indoor camera image\" class=\"wp-image-135698\" srcset=\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/detection-1030x584.png 1030w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/detection-32x18.png 32w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/detection-300x170.png 300w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/detection-768x435.png 768w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/detection-1024x580.png 1024w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/detection.png 1502w\" sizes=\"(max-width: 1030px) 100vw, 1030px\" \/><\/figure>\n<\/div>\n\n\n<p class=\"wp-block-paragraph\">The output is <code>vision_msgs\/msg\/Detection2DArray<\/code>. Pair a message with its source image and check the class, box coordinates, acquisition timestamp and <code>frame_id<\/code>. A processed frame with no targets should still publish an empty detections array, allowing downstream tracking to distinguish an empty observation from a stopped input.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If a topic exists but a subscriber receives nothing, inspect QoS. The course uses BEST_EFFORT sensor publishing; a subscriber requesting RELIABLE delivery may be incompatible. For detection quality, use labeled data. Precision asks how many reported objects are correct; Recall asks how many real objects were found. Report mAP with the dataset and IoU convention so that comparisons remain meaningful. An overlay alone does not measure accuracy.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Track the Same Object Across Frames with ByteTrack<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Detection supplies boxes for each frame. ByteTrack connects them to existing tracks: a Kalman model predicts motion, association matches observations, and track management handles creation, temporary loss and removal.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Its two-stage association first uses higher-score detections, then lower-score candidates to help recover some unmatched tracks. <strong>Those lower-score candidates must reach the tracker.<\/strong> The course&#8217;s detector confidence and tracker activation defaults are both <code>0.25<\/code>; upstream filtering can remove the candidates needed by the second stage.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Stop the standalone detection demo before running the tracking entry point, which starts its required detection\/tracking pipeline:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>.\/scripts\/m4\/run_m4_2_demo.sh\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Check <code>\/perception\/tracks<\/code> in another sourced terminal. It remains a <code>Detection2DArray<\/code>, now carrying track IDs in <code>Detection2D.id<\/code>.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1030\" height=\"563\" src=\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/tracking-1030x563.png\" alt=\"Recorded road-video tracking example with distinct IDs attached to vehicle detections\" class=\"wp-image-135699\" srcset=\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/tracking-1030x563.png 1030w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/tracking-32x18.png 32w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/tracking-300x164.png 300w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/tracking-768x420.png 768w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/tracking-1024x560.png 1024w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/tracking.png 1512w\" sizes=\"(max-width: 1030px) 100vw, 1030px\" \/><\/figure>\n<\/div>\n\n\n<h3 class=\"wp-block-heading\">Tune One Failure, Then Replay the Same Sequence<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The course configuration includes:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>track_activation_threshold: 0.25\nlost_track_buffer: 90\nminimum_matching_threshold: 0.8\nframe_rate: 30\nminimum_consecutive_frames: 1\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Increasing <code>minimum_consecutive_frames<\/code> delays confirmation and can filter short-lived false tracks. <code>lost_track_buffer<\/code> controls how long a missing track remains available. The library calculates that window as <code>int(frame_rate \/ 30.0 * lost_track_buffer)<\/code>: these settings give 90 processed frames, roughly three seconds at an actual 30 FPS or six at 15 FPS if the configuration stays unchanged.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In <a href=\"https:\/\/github.com\/roboflow\/supervision\/blob\/0.27.0\/supervision\/tracker\/byte_tracker\/core.py\">Supervision 0.27.0<\/a>, <code>minimum_matching_threshold<\/code> is a <strong>first-stage matching-cost ceiling<\/strong>. Increasing it allows higher-cost matches; it is not a minimum IoU threshold.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Replay a fixed sequence with entry, occlusion and exit. Note when detections disappear and when IDs change, then compare one parameter adjustment. Some <a href=\"https:\/\/github.com\/Seeed-Projects\/mobile-robot-full-stack-course\/blob\/main\/docs\/M04-AI-Vision-And-Edge-Acceleration\/code\/4.2-multi-object-tracking\/ros2\/bev_tracking\/bev_tracking\/tracking_node.py\">node parameter updates<\/a> rebuild the tracker and reset IDs; separate that reset from an identity switch during normal operation. This route does not enable ReID or global camera-motion compensation, and it does not automatically use the previous module&#8217;s IMU or odometry.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Use SegFormer to Separate Pixel Classes from Ground Candidates<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">SegFormer-B0 predicts a class at each pixel. The course configuration takes <code>[1, 3, 512, 1024]<\/code> input and returns <code>[1, 19, 128, 256]<\/code> logits for 19 Cityscapes classes.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The important order is <strong>restore score geometry, then choose the class<\/strong>. Remove letterbox padding and resize the class scores to the original-image geometry before applying pixelwise <code>argmax<\/code>. Choosing labels first and enlarging them can produce different boundaries because it propagates decisions already made.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The node publishes two outputs:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><code>\/perception\/semantic_mask<\/code>: predicted class IDs.<\/li>\n\n\n\n<li><code>\/perception\/drivable_mask<\/code>: configured classes written as <code>255<\/code>, with other pixels set to <code>0<\/code>.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">The default <code>drivable_class_ids: [0]<\/code> selects road. Adding class <code>1<\/code> includes sidewalk; it changes the selection rule, not the model weights or semantic predictions.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"880\" height=\"1030\" src=\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/08-segmentation-edited-880x1030.png\" alt=\"SegFormer workflow restoring logits to source-image geometry before argmax and then mapping selected classes to a separate ground-candidate mask\" class=\"wp-image-135701\" srcset=\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/08-segmentation-edited-880x1030.png 880w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/08-segmentation-edited-256x300.png 256w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/08-segmentation-edited-768x899.png 768w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/08-segmentation-edited-1312x1536.png 1312w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/08-segmentation-edited-27x32.png 27w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/08-segmentation-edited-1024x1198.png 1024w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/08-segmentation-edited.png 1520w\" sizes=\"(max-width: 880px) 100vw, 880px\" \/><\/figure>\n<\/div>\n\n\n<p class=\"wp-block-paragraph\">Prepare the segmentation engine, stop other demo modes, then run:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>.\/scripts\/m4\/run_m4_3_demo.sh<\/code><\/pre>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1030\" height=\"563\" src=\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/segmentation-1030x563.png\" alt=\"SegFormer course preview highlighting selected ground-candidate pixels over a road scene\" class=\"wp-image-135702\" srcset=\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/segmentation-1030x563.png 1030w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/segmentation-300x164.png 300w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/segmentation-768x420.png 768w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/segmentation-32x18.png 32w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/segmentation-1024x560.png 1024w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/segmentation.png 1512w\" sizes=\"(max-width: 1030px) 100vw, 1030px\" \/><\/figure>\n<\/div>\n\n\n<p class=\"wp-block-paragraph\">For a useful experiment, keep one road\/sidewalk frame fixed and compare selections <code>[0]<\/code> and <code>[0, 1]<\/code>. The semantic map should stay the same while the selected mask changes. A displaced boundary points toward geometry restoration; a wrong class points toward the model and scene; a correct class selected for the wrong purpose points toward the mapping.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Evaluation needs labeled pixels from the target scene. If 80% of an image is road, predicting road everywhere still gives 80% Pixel Accuracy while missing every other class. Use per-class IoU, mIoU and a confusion matrix to expose that failure.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Estimate a Known Object&#8217;s 6D Pose with FoundationPose<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A 2D detection cannot provide a known object&#8217;s full position and orientation. FoundationPose&#8217;s model-based route uses aligned RGB-D, camera intrinsics, a target-instance mask and a mesh. Keep the frame, mask, object dimensions and depth units consistent.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Start with the native Mustard recorded-data exercise. It lets you inspect the estimator before introducing live-camera synchronization and ROS transport. Prepare the documented NVlabs checkout, refiner\/scorer weights and sample assets first.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Register Once, Then Update the Pose<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">First-frame <code>register<\/code> generates hypotheses, refines them against the observation and scores candidates. Later <code>track_one<\/code> calls start from the previous pose and perform local updates:<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"772\" height=\"1030\" src=\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/09-foundationpose-edited-772x1030.png\" alt=\"Native FoundationPose lifecycle from consistent inputs and first-frame registration to repeated tracking updates using the previous pose\" class=\"wp-image-135704\" srcset=\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/09-foundationpose-edited-772x1030.png 772w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/09-foundationpose-edited-225x300.png 225w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/09-foundationpose-edited-768x1024.png 768w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/09-foundationpose-edited-1152x1536.png 1152w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/09-foundationpose-edited-24x32.png 24w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/09-foundationpose-edited-1024x1366.png 1024w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/09-foundationpose-edited.png 1520w\" sizes=\"(max-width: 772px) 100vw, 772px\" \/><\/figure>\n<\/div>\n\n\n<p class=\"wp-block-paragraph\">Run the maintained recorded-data entry point from the M04 code directory:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>bash scripts\/m4\/run_m4_5_native_mvp.sh --frames 8\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">In <code>output\/m4\/m45_native_mvp\/<\/code>, compare the first and last <code>frame_*_pose.png<\/code> overlays with the corresponding <code>pose_*.txt<\/code> matrices and <code>report.json<\/code>. Check outline alignment, axes and physical scale together.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"640\" height=\"480\" src=\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/native-pose.png\" alt=\"Native FoundationPose tracking output on the recorded Mustard RGB-D sequence, showing the estimated object box and axes\" class=\"wp-image-135705\" srcset=\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/native-pose.png 640w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/native-pose-300x225.png 300w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/native-pose-32x24.png 32w\" sizes=\"(max-width: 640px) 100vw, 640px\" \/><\/figure>\n<\/div>\n\n\n<h3 class=\"wp-block-heading\">Read the Matrix and the Timing Separately<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\"><code>T_camera_object<\/code> maps object coordinates into camera coordinates: <code>p_camera = R \u00d7 p_object + t<\/code>. A small read-only check helps inspect the exported matrix:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>from pathlib import Path\nimport numpy as np\n\nT = np.loadtxt(Path(\"output\/m4\/m45_native_mvp\/pose_0000.txt\"))\nassert T.shape == (4, 4) and np.isfinite(T).all()\nR, t = T&#91;:3, :3], T&#91;:3, 3]\nprint(\"translation:\", t)\nprint(\"rotation determinant:\", np.linalg.det(R))\nprint(\"R.T @ R:\\n\", R.T @ R)\nprint(\"homogeneous row:\", T&#91;3])\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">For a rigid transform, <code>R.T @ R<\/code> should be near identity, the determinant near <code>+1<\/code>, and the bottom row <code>[0, 0, 0, 1]<\/code>. These checks detect structural problems; accuracy still needs calibration or ground truth. If distance is wrong, check depth units, intrinsics and mesh scale. The course CAD example converts millimeter STEP geometry into a meter-scale mesh and records dimensions in <code>object.yaml<\/code>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In the report, <code>register_ms<\/code> describes initialization, <code>track_ms<\/code> contains update times, and <code>track_fps<\/code> is derived from tracking-stage latency. The course&#8217;s eight-frame Mustard record on J501 \/ AGX Orin 32GB, JetPack 6.2.1 and CUDA 12.6 reports <strong>11,050.90 ms<\/strong> for registration and approximately <strong>120.43 ms\/frame<\/strong> for updates (<strong>8.30 FPS<\/strong>). These are existing course measurements; the tracking figure is not the rate of a complete live RGB-D pipeline.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Transform the Pose into the Robot&#8217;s Frame<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The native <code>bev_pose<\/code> wrapper uses the prepared mesh with RGB, aligned depth, <code>CameraInfo<\/code> and an instance mask. It publishes <code>PoseStamped<\/code>, <code>Detection3DArray<\/code> and TF; the pose topic is <code>\/perception\/object_pose<\/code>. The file-producing MVP does not start this live wrapper.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">To express the object pose in the base frame, use the calibrated camera relationship under the column-vector convention:<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"906\" height=\"1030\" src=\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/10-pose-frames-edited-906x1030.png\" alt=\"Object-to-camera and camera-to-base coordinate transforms composed into an object-to-base transform under the column-vector convention\" class=\"wp-image-135707\" srcset=\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/10-pose-frames-edited-906x1030.png 906w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/10-pose-frames-edited-264x300.png 264w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/10-pose-frames-edited-768x873.png 768w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/10-pose-frames-edited-1351x1536.png 1351w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/10-pose-frames-edited-28x32.png 28w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/10-pose-frames-edited-1024x1164.png 1024w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/10-pose-frames-edited.png 1520w\" sizes=\"(max-width: 906px) 100vw, 906px\" \/><\/figure>\n<\/div>\n\n\n<p class=\"wp-block-paragraph\">Changing <code>frame_id<\/code> alone does not transform coordinates. Check direction, units, synchronization and source timestamps; ROS quaternions use <code>x, y, z, w<\/code> ordering. A valid message and an accurate physical pose require separate verification.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Where Isaac ROS Fits<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The Isaac ROS route teaches component graphs, engine profiles and ROS integration around the same pose task. For the course&#8217;s TensorRT 10.3 configuration, refine\/score use FP32. The 252-candidate and 42-candidate configurations need their matching engines and launch settings; score profile <code>1\/1\/252<\/code> specifies minimum, optimal and maximum candidate batch sizes.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">After preparing the container, engines and recorded assets, the official single-frame entry point is:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>M44_MODE=official .\/scripts\/m4\/run_m4_4_isaacros_quickstart.sh\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">It exposes <code>\/output<\/code> as <code>vision_msgs\/msg\/Detection3DArray<\/code>, rather than the native wrapper&#8217;s pose topic. The course records valid poses from the Mustard single-frame graph; continuous physical-camera tracking and accuracy acceptance remain further integration work.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Choose the Next Experiment and Find the Course Code<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Before moving on, save four results that you can explain:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>One detection frame:<\/strong> pair the source image with its message and check boxes, classes, timestamp and frame.<\/li>\n\n\n\n<li><strong>One tracking sequence:<\/strong> identify when a detection disappears, an ID changes or a parameter update resets the tracker.<\/li>\n\n\n\n<li><strong>One region boundary:<\/strong> compare the original image, semantic map and selected mask; distinguish geometry, classification and mapping errors.<\/li>\n\n\n\n<li><strong>One pose run:<\/strong> connect the inputs, first\/last overlays, pose matrices and timing report. State what still needs testing before live robot integration.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Keep the input fixed and change one setting at a time. Those saved results give you a useful baseline for the next experiment\u2014and enough context to discuss a problem with another developer.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Full installation instructions, model assets and advanced configuration are collected here:<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><thead><tr><th>Learning task<\/th><th>Maintained resource<\/th><\/tr><\/thead><tbody><tr><td>Prepare the workspace<\/td><td><a href=\"https:\/\/github.com\/Seeed-Projects\/mobile-robot-full-stack-course\/blob\/main\/docs\/M04-AI-Vision-And-Edge-Acceleration\/code\/README.md\">Dependencies, build and model preparation<\/a><\/td><\/tr><tr><td>Deploy YOLO<\/td><td><a href=\"https:\/\/github.com\/Seeed-Projects\/mobile-robot-full-stack-course\/blob\/main\/docs\/M04-AI-Vision-And-Edge-Acceleration\/4.1_YOLO_Object_Detection_and_TensorRT_Deployment\/README_en_US.md\">Detection, TensorRT and ROS 2 output<\/a><\/td><\/tr><tr><td>Investigate track IDs<\/td><td><a href=\"https:\/\/github.com\/Seeed-Projects\/mobile-robot-full-stack-course\/blob\/main\/docs\/M04-AI-Vision-And-Edge-Acceleration\/4.2_Multi-Object_Tracking\/README_en_US.md\">ByteTrack configuration and experiments<\/a><\/td><\/tr><tr><td>Interpret segmentation<\/td><td><a href=\"https:\/\/github.com\/Seeed-Projects\/mobile-robot-full-stack-course\/blob\/main\/docs\/M04-AI-Vision-And-Edge-Acceleration\/4.3_Semantic_Segmentation_and_Drivable_Area_Analysis\/README_en_US.md\">SegFormer, masks and evaluation<\/a><\/td><\/tr><tr><td>Start with recorded object pose<\/td><td><a href=\"https:\/\/github.com\/Seeed-Projects\/mobile-robot-full-stack-course\/blob\/main\/docs\/M04-AI-Vision-And-Edge-Acceleration\/4.5_Native_FoundationPose_6D_Pose\/README_en_US.md\">Native FoundationPose and ROS integration<\/a><\/td><\/tr><tr><td>Learn the accelerated component graph<\/td><td><a href=\"https:\/\/github.com\/Seeed-Projects\/mobile-robot-full-stack-course\/blob\/main\/docs\/M04-AI-Vision-And-Edge-Acceleration\/4.4_Isaac_ROS_FoundationPose_and_Acceleration\/README_en_US.md\">Isaac ROS FoundationPose<\/a><\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\"><strong><a href=\"https:\/\/github.com\/Seeed-Projects\/mobile-robot-full-stack-course\/tree\/main\/docs\/M04-AI-Vision-And-Edge-Acceleration\">Explore the complete M04 course and code<\/a>.<\/strong> Star or fork the repository to return to the lessons as your experiments develop. For a question or reproducible problem, check <a href=\"https:\/\/github.com\/Seeed-Projects\/mobile-robot-full-stack-course\/issues\">existing course issues<\/a> and include your versions, input, configuration and relevant output.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">The Hardware Behind the Experiments: reComputer Robotics J501<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The experiments in this module use <strong>reComputer Robotics J501<\/strong>, combining NVIDIA Jetson AGX Orin compute with GMSL camera expansion and robotics interfaces such as USB, Ethernet and CAN. This makes it a practical reference platform for projects that need multi-camera input, local AI inference and ROS 2 communication on an embedded computer.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><a href=\"https:\/\/www.seeedstudio.com\/reComputer-Robotics-J5011-with-GMSL-extension-board-p-6681.html\"><img loading=\"lazy\" decoding=\"async\" width=\"1030\" height=\"773\" src=\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/recomputer-robotics-j501-1030x773.png\" alt=\"Official reComputer Robotics J501 series product photograph showing the enclosure and front robotics interfaces reComputer Robotics J501 series. The course uses the 32GB module variant; select the matching configuration below\" class=\"wp-image-135708\" srcset=\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/recomputer-robotics-j501-1030x773.png 1030w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/recomputer-robotics-j501-300x225.png 300w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/recomputer-robotics-j501-32x24.png 32w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/recomputer-robotics-j501-768x576.png 768w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/recomputer-robotics-j501-1024x768.png 1024w, https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/recomputer-robotics-j501.png 1400w\" sizes=\"(max-width: 1030px) 100vw, 1030px\" \/><\/a><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">For the matching 32GB configuration, see <strong><a href=\"https:\/\/www.seeedstudio.com\/reComputer-Robotics-J5011-with-GMSL-extension-board-p-6681.html\">reComputer Robotics J5011 with GMSL expansion<\/a><\/strong>. The <strong><a href=\"https:\/\/wiki.seeedstudio.com\/ai_robotics_recomputer_j501_robotics_getting_started\/\">J501 Robotics Wiki<\/a><\/strong> covers flashing and hardware interfaces. If you are using another J501 configuration, check its module, camera and software requirements against the experiment before reusing the same settings.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The next stage of the course moves into SLAM and 3D reconstruction, building on the perception and sensor data established in the earlier modules. The same engineering habit carries forward: keep the input, coordinate frame, timing and output evidence visible.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Working on a related robot project? Join the <strong><a href=\"https:\/\/discord.com\/invite\/QqMgVwHT3X\">Seeed Studio Discord community<\/a><\/strong> and <a href=\"https:\/\/www.facebook.com\/groups\/rebotarm\">Facebook Group<\/a> to share your detection results, tracking sequences, segmentation masks or pose-estimation experiments. Include the input, configuration and a specific result so other developers can reproduce the issue and contribute useful feedback.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>From object detection and multi-object tracking to semantic segmentation and 6D pose estimation on NVIDIA<\/p>\n","protected":false},"author":3703,"featured_media":135711,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_lmt_disableupdate":"","_lmt_disable":"","_price":"","_stock":"","_tribe_ticket_header":"","_tribe_default_ticket_provider":"","_tribe_ticket_capacity":"0","_ticket_start_date":"","_ticket_end_date":"","_tribe_ticket_show_description":"","_tribe_ticket_show_not_going":false,"_tribe_ticket_use_global_stock":"","_tribe_ticket_global_stock_level":"","_global_stock_mode":"","_global_stock_cap":"","_tribe_rsvp_for_event":"","_tribe_ticket_going_count":"","_tribe_ticket_not_going_count":"","_tribe_tickets_list":"[]","_tribe_ticket_has_attendee_info_fields":false,"iawp_total_views":0,"footnotes":""},"categories":[5007,1],"tags":[5776,5775,2799,5772,5774,5631,5660,5633,5777,4420,5773],"class_list":["post-135676","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-feature","category-news","tag-6d-pose-estimation","tag-bytetrack","tag-edge-ai","tag-foundationpose","tag-jetson-ai-vision","tag-mobile-robotics","tag-recomputer-robotics-j501","tag-ros-2","tag-segformer","tag-tensorrt","tag-yolo11"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v24.0 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Jetson ROS 2 Computer Vision: YOLO, ByteTrack, SegFormer &amp; FoundationPose<\/title>\n<meta name=\"description\" content=\"Build AI vision on NVIDIA Jetson with ROS 2. Explore YOLO11 TensorRT deployment, ByteTrack object tracking, SegFormer semantic segmentation and FoundationPose 6D pose estimation.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.seeedstudio.com\/blog\/2026\/11\/11\/mobile-robot-full-stack-development-in-practice-m4-jetson-ai-vision\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Jetson ROS 2 Computer Vision: YOLO, ByteTrack, SegFormer &amp; FoundationPose\" \/>\n<meta property=\"og:description\" content=\"Build AI vision on NVIDIA Jetson with ROS 2. Explore YOLO11 TensorRT deployment, ByteTrack object tracking, SegFormer semantic segmentation and FoundationPose 6D pose estimation.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.seeedstudio.com\/blog\/2026\/11\/11\/mobile-robot-full-stack-development-in-practice-m4-jetson-ai-vision\/\" \/>\n<meta property=\"og:site_name\" content=\"Latest News from Seeed Studio\" \/>\n<meta property=\"article:published_time\" content=\"2026-11-11T06:22:18+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2026-11-11T06:28:33+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/MRm04featured.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1734\" \/>\n\t<meta property=\"og:image:height\" content=\"907\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Frank\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Frank\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"15 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.seeedstudio.com\/blog\/2026\/11\/11\/mobile-robot-full-stack-development-in-practice-m4-jetson-ai-vision\/\",\"url\":\"https:\/\/www.seeedstudio.com\/blog\/2026\/11\/11\/mobile-robot-full-stack-development-in-practice-m4-jetson-ai-vision\/\",\"name\":\"Jetson ROS 2 Computer Vision: YOLO, ByteTrack, SegFormer & FoundationPose\",\"isPartOf\":{\"@id\":\"https:\/\/www.seeedstudio.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.seeedstudio.com\/blog\/2026\/11\/11\/mobile-robot-full-stack-development-in-practice-m4-jetson-ai-vision\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.seeedstudio.com\/blog\/2026\/11\/11\/mobile-robot-full-stack-development-in-practice-m4-jetson-ai-vision\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/MRm04featured.png\",\"datePublished\":\"2026-11-11T06:22:18+00:00\",\"dateModified\":\"2026-11-11T06:28:33+00:00\",\"author\":{\"@id\":\"https:\/\/www.seeedstudio.com\/blog\/#\/schema\/person\/1dd116f442e8ed65ddbd35b6d3f28d8a\"},\"description\":\"Build AI vision on NVIDIA Jetson with ROS 2. Explore YOLO11 TensorRT deployment, ByteTrack object tracking, SegFormer semantic segmentation and FoundationPose 6D pose estimation.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.seeedstudio.com\/blog\/2026\/11\/11\/mobile-robot-full-stack-development-in-practice-m4-jetson-ai-vision\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.seeedstudio.com\/blog\/2026\/11\/11\/mobile-robot-full-stack-development-in-practice-m4-jetson-ai-vision\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.seeedstudio.com\/blog\/2026\/11\/11\/mobile-robot-full-stack-development-in-practice-m4-jetson-ai-vision\/#primaryimage\",\"url\":\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/MRm04featured.png\",\"contentUrl\":\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/MRm04featured.png\",\"width\":1734,\"height\":907,\"caption\":\"Mobile robot AI vision workflow on NVIDIA Jetson, showing object detection, multi-object tracking, semantic segmentation and 6D object pose estimation.\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.seeedstudio.com\/blog\/2026\/11\/11\/mobile-robot-full-stack-development-in-practice-m4-jetson-ai-vision\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.seeedstudio.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Mobile Robot Full-stack Development In Practice \u2163: How Does a Robot Understand What It Sees? YOLO, ByteTrack, SegFormer &#038; FoundationPose\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.seeedstudio.com\/blog\/#website\",\"url\":\"https:\/\/www.seeedstudio.com\/blog\/\",\"name\":\"Latest News from Seeed Studio\",\"description\":\"Emerging IoT, AI and Autonomous Applications on the Edge\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.seeedstudio.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.seeedstudio.com\/blog\/#\/schema\/person\/1dd116f442e8ed65ddbd35b6d3f28d8a\",\"name\":\"Frank\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.seeedstudio.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/1e890dec78807f51d88e5a819bc5b8ed232418f466c308274995ea7e7dbe0183?s=96&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/1e890dec78807f51d88e5a819bc5b8ed232418f466c308274995ea7e7dbe0183?s=96&r=g\",\"caption\":\"Frank\"},\"url\":\"https:\/\/www.seeedstudio.com\/blog\/author\/frank\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Jetson ROS 2 Computer Vision: YOLO, ByteTrack, SegFormer & FoundationPose","description":"Build AI vision on NVIDIA Jetson with ROS 2. Explore YOLO11 TensorRT deployment, ByteTrack object tracking, SegFormer semantic segmentation and FoundationPose 6D pose estimation.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.seeedstudio.com\/blog\/2026\/11\/11\/mobile-robot-full-stack-development-in-practice-m4-jetson-ai-vision\/","og_locale":"en_US","og_type":"article","og_title":"Jetson ROS 2 Computer Vision: YOLO, ByteTrack, SegFormer & FoundationPose","og_description":"Build AI vision on NVIDIA Jetson with ROS 2. Explore YOLO11 TensorRT deployment, ByteTrack object tracking, SegFormer semantic segmentation and FoundationPose 6D pose estimation.","og_url":"https:\/\/www.seeedstudio.com\/blog\/2026\/11\/11\/mobile-robot-full-stack-development-in-practice-m4-jetson-ai-vision\/","og_site_name":"Latest News from Seeed Studio","article_published_time":"2026-11-11T06:22:18+00:00","article_modified_time":"2026-11-11T06:28:33+00:00","og_image":[{"width":1734,"height":907,"url":"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/MRm04featured.png","type":"image\/png"}],"author":"Frank","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Frank","Est. reading time":"15 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.seeedstudio.com\/blog\/2026\/11\/11\/mobile-robot-full-stack-development-in-practice-m4-jetson-ai-vision\/","url":"https:\/\/www.seeedstudio.com\/blog\/2026\/11\/11\/mobile-robot-full-stack-development-in-practice-m4-jetson-ai-vision\/","name":"Jetson ROS 2 Computer Vision: YOLO, ByteTrack, SegFormer & FoundationPose","isPartOf":{"@id":"https:\/\/www.seeedstudio.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.seeedstudio.com\/blog\/2026\/11\/11\/mobile-robot-full-stack-development-in-practice-m4-jetson-ai-vision\/#primaryimage"},"image":{"@id":"https:\/\/www.seeedstudio.com\/blog\/2026\/11\/11\/mobile-robot-full-stack-development-in-practice-m4-jetson-ai-vision\/#primaryimage"},"thumbnailUrl":"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/MRm04featured.png","datePublished":"2026-11-11T06:22:18+00:00","dateModified":"2026-11-11T06:28:33+00:00","author":{"@id":"https:\/\/www.seeedstudio.com\/blog\/#\/schema\/person\/1dd116f442e8ed65ddbd35b6d3f28d8a"},"description":"Build AI vision on NVIDIA Jetson with ROS 2. Explore YOLO11 TensorRT deployment, ByteTrack object tracking, SegFormer semantic segmentation and FoundationPose 6D pose estimation.","breadcrumb":{"@id":"https:\/\/www.seeedstudio.com\/blog\/2026\/11\/11\/mobile-robot-full-stack-development-in-practice-m4-jetson-ai-vision\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.seeedstudio.com\/blog\/2026\/11\/11\/mobile-robot-full-stack-development-in-practice-m4-jetson-ai-vision\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.seeedstudio.com\/blog\/2026\/11\/11\/mobile-robot-full-stack-development-in-practice-m4-jetson-ai-vision\/#primaryimage","url":"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/MRm04featured.png","contentUrl":"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/MRm04featured.png","width":1734,"height":907,"caption":"Mobile robot AI vision workflow on NVIDIA Jetson, showing object detection, multi-object tracking, semantic segmentation and 6D object pose estimation."},{"@type":"BreadcrumbList","@id":"https:\/\/www.seeedstudio.com\/blog\/2026\/11\/11\/mobile-robot-full-stack-development-in-practice-m4-jetson-ai-vision\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.seeedstudio.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Mobile Robot Full-stack Development In Practice \u2163: How Does a Robot Understand What It Sees? YOLO, ByteTrack, SegFormer &#038; FoundationPose"}]},{"@type":"WebSite","@id":"https:\/\/www.seeedstudio.com\/blog\/#website","url":"https:\/\/www.seeedstudio.com\/blog\/","name":"Latest News from Seeed Studio","description":"Emerging IoT, AI and Autonomous Applications on the Edge","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.seeedstudio.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.seeedstudio.com\/blog\/#\/schema\/person\/1dd116f442e8ed65ddbd35b6d3f28d8a","name":"Frank","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.seeedstudio.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/1e890dec78807f51d88e5a819bc5b8ed232418f466c308274995ea7e7dbe0183?s=96&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/1e890dec78807f51d88e5a819bc5b8ed232418f466c308274995ea7e7dbe0183?s=96&r=g","caption":"Frank"},"url":"https:\/\/www.seeedstudio.com\/blog\/author\/frank\/"}]}},"modified_by":"Frank","views":6,"featured_image_urls":{"full":["https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/MRm04featured.png",1734,907,false],"thumbnail":["https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/MRm04featured-80x80.png",80,80,true],"medium":["https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/MRm04featured-300x157.png",300,157,true],"medium_large":["https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/MRm04featured-768x402.png",640,335,true],"large":["https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/MRm04featured-1030x539.png",640,335,true],"1536x1536":["https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/MRm04featured-1536x803.png",1536,803,true],"2048x2048":["https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/MRm04featured.png",1734,907,false],"visody_icon":["https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/MRm04featured-32x17.png",32,17,true],"magazine-7-slider-full":["https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/MRm04featured-1536x907.png",1536,907,true],"magazine-7-slider-center":["https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/MRm04featured-936x897.png",936,897,true],"magazine-7-featured":["https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/MRm04featured-1024x536.png",1024,536,true],"magazine-7-medium":["https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/MRm04featured-720x380.png",720,380,true],"magazine-7-medium-square":["https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2026\/11\/MRm04featured-675x450.png",675,450,true]},"author_info":{"display_name":"Frank","author_link":"https:\/\/www.seeedstudio.com\/blog\/author\/frank\/"},"category_info":"<a href=\"https:\/\/www.seeedstudio.com\/blog\/category\/feature\/\" rel=\"category tag\">Feature<\/a> <a href=\"https:\/\/www.seeedstudio.com\/blog\/category\/news\/\" rel=\"category tag\">News<\/a>","tag_info":"News","comment_count":"0","_links":{"self":[{"href":"https:\/\/www.seeedstudio.com\/blog\/wp-json\/wp\/v2\/posts\/135676","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.seeedstudio.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.seeedstudio.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.seeedstudio.com\/blog\/wp-json\/wp\/v2\/users\/3703"}],"replies":[{"embeddable":true,"href":"https:\/\/www.seeedstudio.com\/blog\/wp-json\/wp\/v2\/comments?post=135676"}],"version-history":[{"count":5,"href":"https:\/\/www.seeedstudio.com\/blog\/wp-json\/wp\/v2\/posts\/135676\/revisions"}],"predecessor-version":[{"id":135715,"href":"https:\/\/www.seeedstudio.com\/blog\/wp-json\/wp\/v2\/posts\/135676\/revisions\/135715"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.seeedstudio.com\/blog\/wp-json\/wp\/v2\/media\/135711"}],"wp:attachment":[{"href":"https:\/\/www.seeedstudio.com\/blog\/wp-json\/wp\/v2\/media?parent=135676"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.seeedstudio.com\/blog\/wp-json\/wp\/v2\/categories?post=135676"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.seeedstudio.com\/blog\/wp-json\/wp\/v2\/tags?post=135676"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}