{"id":77276,"date":"2023-03-30T13:52:27","date_gmt":"2023-03-30T05:52:27","guid":{"rendered":"https:\/\/www.seeedstudio.com\/blog\/?p=77276"},"modified":"2023-03-30T14:50:56","modified_gmt":"2023-03-30T06:50:56","slug":"yolov8-performance-benchmarks-on-nvidia-jetson-devices","status":"publish","type":"post","link":"https:\/\/www.seeedstudio.com\/blog\/2023\/03\/30\/yolov8-performance-benchmarks-on-nvidia-jetson-devices\/","title":{"rendered":"YOLOv8 Performance Benchmarks on NVIDIA Jetson Devices"},"content":{"rendered":"\n

This blog will talk about the performance benchmarks of all the YOLOv8 models running on different NVIDIA Jetson devices. We have specifically selected 3 different Jetson devices for this test, and they are the Jetson AGX Orin 32GB H01 Kit<\/a>, reComputer J4012 built with Orin NX 16GB<\/a>, and reComputer J2021 built with Xavier NX 8GB<\/a>.<\/p>\n\n\n\n

What is YOLOv8?<\/h2>\n\n\n\n

YOLOv8<\/a>, developed by Ultralytics, is a cutting-edge, state-of-the-art (SOTA) model that builds upon the success of previous YOLO versions and introduces new features and improvements to further boost performance and flexibility. YOLOv8 is designed to be fast, accurate, and easy to use, making it an excellent choice for a wide range of object detection, image segmentation, and image classification tasks.<\/p>\n\n\n\n

\"\"
Source: https:\/\/github.com\/ultralytics\/ultralytics<\/figcaption><\/figure>\n\n\n\n

YOLOv8 Models Types<\/h2>\n\n\n\n

YOLOv8 has different model types based on the number of parameters that will relate to the accuracy of the model. So, the bigger the model, the more accurate it is. For example, YOLOv8x is the biggest model and it has the highest accuracy out of all the models.<\/p>\n\n\n\n

\"\"
Source: https:\/\/github.com\/ultralytics\/ultralytics<\/figcaption><\/figure><\/div>\n\n\n\n

Why Do We Need to Benchmark Performance?<\/h2>\n\n\n\n

By running performance benchmarks, you can know how much of an inference performance you can obtain for a particular model type running on a particular device. This is more important for embedded devices such as the NVIDIA Jetson platform because if you know the exact model type you want to use for your application, you can decide which hardware will be suitable to run that model.<\/p>\n\n\n\n

Why Do We Need TensorRT Benchmarks?<\/h2>\n\n\n\n

TensorRT<\/a> is a library developed by NVIDIA to make inference faster on NVIDIA GPUs. TensorRT is built on CUDA and it can give more than 2 to 3 times faster inference on many real-time services and embedded applications when compared with running native models such as PyTorch and ONNX without TensorRT.<\/p>\n\n\n\n

Install YOLOV8 on Nvidia Jetson Devices<\/h2>\n\n\n\n

Step 1:<\/strong> Flash the Jetson device with JetPack as explained in this wiki<\/a>. <\/p>\n\n\n\n

Step 2:<\/strong> Follow the sections “Install Necessary Packages”<\/strong> and “Install PyTorch and Torchvision”<\/strong> of the above wiki to install YOLOv8 on the Jetson device<\/p>\n\n\n\n

How to Run the Benchmarks?<\/h2>\n\n\n\n

When you install NVIDIA JetPack<\/a> with SDK components on an NVIDIA Jetson device, there will be a tool called trtexec<\/a>. This tool is actually located inside TensorRT which comes with SDK components installation. This is a tool to use TensorRT without having to develop your own application. The trtexec tool has three main purposes<\/p>\n\n\n\n