We will be covering the following content and more!<\/p>\n\n\n\n
- What is a GPU Cluster?<\/li>
- What is an NVIDIA Jetson Cluster & What are its Benefits?<\/li>
- GPU Cluster Applications & Examples<\/li>
- Building an NVIDIA Jetson Cluster: Hardware Recommendations<\/li>
- Tutorial: Use Kubernetes with an NVIDIA Jetson Cluster for GPU-Powered Computing<\/li><\/ul>\n\n\n\n<\/div>\n\n\n\n
![\"\"](\"https:\/\/blog.seeedstudio.com\/wp-content\/uploads\/2021\/05\/NVIDIAJetsonClusterCover-1030x601.png\")
<\/figure><\/div>\n\n\n\n<\/div>\n\n\n\n
\n\n\n\n<\/div>\n\n\n\nWhat is GPU Clustering on the Edge?<\/strong><\/h2>\n\n\n\n
<\/div>\n\n\n\nEdge GPU clusters are computer clusters that are deployed on the edge<\/em>, that carry GPUs (or Graphics Processing Units) for edge computing purposes<\/strong>. Edge computing, in turn, describes computational tasks that are performed on devices which are physically located in the local space of their application<\/strong>. This is in contrast to cloud computing, where these processes are handled remotely.<\/p>\n\n\n\n
<\/div>\n\n\n\nSource: California Technical Academy<\/em><\/figcaption><\/figure><\/div>\n\n\n\n <\/div>\n\n\n\nFirstly, What is Cluster Computing<\/strong>?<\/h3>\n\n\n\n
<\/div>\n\n\n\nComputer clusters are defined as a group of computers that work together<\/strong> so that they can be viewed as a single system. A cluster is then typically used to process large workloads by distributing tasks across the multiple computers in the system<\/strong>. There are three major types of clusters<\/strong>, each serving different purposes and contributing their own set of benefits to cluster computing!<\/p>\n\n\n\n
<\/div>\n\n\n\n- High Availability<\/strong> \u2013 Ensures that applications are always available by rerouting requests to another node in the event of a failure.<\/li>
- Load Balancing<\/strong> \u2013 Spreads computing workloads evenly across slave nodes to handle high job volumes.<\/li>
- High Performance<\/strong> \u2013 Multiple slave nodes are used in parallel to increase computing power for tasks with high computing requirements.<\/li><\/ul>\n\n\n\n
<\/div>\n\n\n\n![\"\"](\"https:\/\/blog.seeedstudio.com\/wp-content\/uploads\/2021\/05\/110110991_2-1030x773.jpeg\")
NVIDIA Jetson GPU Cluster<\/em><\/figcaption><\/figure><\/div>\n\n\n\n <\/div>\n\n\n\nCluster Computing + GPUs = GPU Clusters<\/strong><\/h3>\n\n\n\n
<\/div>\n\n\n\nAs their name suggests, GPU Clusters<\/strong> consist of computers that are equipped with GPUs. GPUs, in contrast with CPUs or central processing units, specialise in parallel computing. They break complex problems into much smaller tasks in order to compute them all at once, in order to achieve high throughput computing<\/strong>.<\/p>\n\n\n\n
<\/div>\n\n\n\nSource: Apps4Rent<\/em><\/figcaption><\/figure><\/div>\n\n\n\n <\/div>\n\n\n\nTraditionally, graphics processing alone benefited from GPUs, since the rendering of textures, lighting and shapes had to be done simultaneously to produce smooth motion graphics. However, with modern GPU frameworks, their parallel computing capabilities can now be extended to other areas<\/strong> like data processing and machine learning – giving rise to General Purpose GPUs or GPGPUs!<\/p>\n\n\n\n
<\/div>\n\n\n\nI\u2019ve covered both Cluster Computing<\/a> & GPU Clusters<\/a> in depth in my previous articles, so be sure to check them out for more details!<\/p>\n\n\n\n
<\/div>\n\n\n\n
\n\n\n\n<\/div>\n\n\n\nWhat are NVIDIA Jetson Clusters & Why Use Them?<\/strong><\/h2>\n\n\n\n
<\/div>\n\n\n\nTraditionally, cluster computing was unique to cloud computing, but as advances in Single Board Computers (SBCs) & network infrastructure take strides, this is no longer the case. In this section, we will be looking at NVIDIA Jetson Clusters<\/strong>, which are GPU Clusters built from NVIDIA\u2019s Jetson modules, and why you should consider it for your edge computing applications!<\/p>\n\n\n\n
<\/div>\n\n\n\nIntroducing the NVIDIA Jetson Series<\/strong><\/h3>\n\n\n\n
<\/div>\n\n\n\nNVIDIA\u2019s Jetson modules are GPU-enabled compute modules that provide great performance and power efficiency<\/strong> to run software effectively on the edge. GPU capabilities allow these modules to power a range of applications that require higher performance, including AI-powered Network Video Recorders (NVRs), automated optical inspection systems in precision manufacturing, and autonomous robots!<\/p>\n\n\n\n
<\/div>\n\n\n\nFurthermore, NVIDIA\u2019s reputation in the semiconductor industry is outstanding. With their GPUs powering the majority of the global GPU applications, NVIDIA has for years consistently designed and produced high quality GPUs for a vast range of consumer, industry, and research applications. Naturally, the NVIDIA Jetson series is no exception, featuring great build quality and performance!<\/p>\n\n\n\n
<\/div>\n\n\n\nNVIDIA Jetson Nano System on Module (SoM)<\/em><\/figcaption><\/figure><\/div>\n\n\n\n <\/div>\n\n\n\nCUDA & NVIDIA Jetpack<\/strong><\/h3>\n\n\n\n
<\/div>\n\n\n\nBeyond NVIDIA\u2019s great hardware in their products, however, the greatest advantage of running an NVIDIA Jetson powered GPU cluster can be argued to lie heavily in their software capabilities, namely CUDA & NVIDIA Jetpack compatibility.<\/p>\n\n\n\n
<\/div>\n\n\n\nCUDA<\/strong><\/a> is NVIDIA\u2019s industry-leading proprietary GPGPU framework<\/strong> for programming and developing applications with GPUs. It features GPU-accelerated libraries, debugging and optimisation tools, a C\/C++ compiler and a runtime library that enables you to build and deploy your application on a variety of computer architectures. With excellent support, integration and development compatibility with numerous GPU applications, CUDA is in fact the preferred choice by developers worldwide.<\/p>\n\n\n\n
<\/div>\n\n\n\nNVIDIA\u2019s Jetpack SDK<\/strong><\/a> <\/strong>is a comprehensive solution from NVIDIA to build AI applications, and supports all Jetson modules & developer kits. It consists of a Linux operating system, CUDA-X accelerated libraries, as well as APIs for Deep Learning, Computer Vision, Accelerated Computing and Multimedia. Even better, it includes samples, documentation and developer tools to help you get developing easily!<\/p>\n\n\n\n
<\/div>\n\n\n\nThus, building a GPU cluster powered by NVIDIA Jetson products provides not only powerful hardware, but is also complete with also easy to use software and libraries that will be sure to accelerate your development process!<\/p>\n\n\n\n
<\/div>\n\n\n\nSource: NVIDIA Edge Computing<\/em><\/figcaption><\/figure><\/div>\n\n\n\n <\/div>\n\n\n\n
\n\n\n\n<\/div>\n\n\n\nBuilding NVIDIA Jetson Clusters with Jetson Mate<\/strong><\/h2>\n\n\n\n
<\/div>\n\n\n\nSo you\u2019ve decided to build your very own NVIDIA Jetson GPU cluster, and you\u2019re looking for a place to start. You may have already come across some examples that wire multiple Jetson Nano Development Kits<\/a> together. While that definitely works, the wiring can get rather complex, and even worse: that form factor is hardly suitable for field deployment – which is the whole point of an edge computing system!<\/p>\n\n\n\n
<\/div>\n\n\n\nTo tackle these challenges, Seeed is proud to share our complete edge GPU clustering solution with the Jetson Mate and NVIDIA\u2019s Jetson Nano \/ Xavier NX modules. Complete with a carrier board and the Jetson modules, you can easily get your hands on a complete NVIDIA GPU Cluster that is powered by NVIDIA\u2019s industry-leading GPUs for edge applications!<\/p>\n\n\n\n
<\/div>\n\n\n\n<\/figure><\/div>\n\n\n\n
<\/div>\n\n\n\nYou can now pick up the hardware for a complete edge GPU cluster from Seeed in two convenient packages:<\/p>\n\n\n\n
<\/div>\n\n\n\n- Jetson Mate Cluster Standard<\/strong><\/a> with 1 Jetson Nano<\/a> SoM and 3 Jetson Xavier NX<\/a> SoMs<\/li>
- Jetson Mate Cluster Advanced<\/strong><\/a> with 4 Jetson Xavier NX<\/a> SoMs<\/li><\/ul>\n\n\n\n
<\/div>\n\n\n\nContinue reading to learn more about the Jetson Mate, Jetson Nano and Jetson Xavier NX!<\/p>\n\n\n\n
<\/div>\n\n\n\nIntroducing the <\/strong>Jetson Mate<\/strong><\/a><\/h3>\n\n\n\n
<\/div>\n\n\n\nThe Jetson Mate carrier board is a comprehensive and reliable solution that has been specially designed for building NVIDIA Jetson clusters. Equipped with an onboard 5-port gigabit switch that enables up to 4 SoMs to communicate with each other, as well as independent power for 3 worker\/slave nodes, the Jetson Mate with its rich peripherals (CSI, HDMI, USB, Ethernet) and inbuilt fan is a complete solution for building NVIDIA GPU clusters on the edge.<\/p>\n\n\n\n
<\/div>\n\n\n\n\n\n![\"\"](\"https:\/\/blog.seeedstudio.com\/wp-content\/uploads\/2021\/05\/ezgif.com-gif-maker-1030x773.png\")
<\/figure><\/div>\n<\/div>\n\n\n\n\n![\"\"](\"https:\/\/blog.seeedstudio.com\/wp-content\/uploads\/2021\/05\/JetsonMate-1030x781.png\")
<\/figure><\/div>\n<\/div>\n<\/div>\n\n\n\n<\/div>\n\n\n\nThe Jetson Mate can house up to 4 of NVIDIA\u2019s very own Jetson Nano<\/a> \/ Xavier NX <\/a>SoMs in its compact form factor to deliver immense computing power on the edge. With an easy-to-build design that can be easily set up with our step-by-step guide, the Jetson Mate also offers high flexibility and performance for your GPU clusters.<\/p>\n\n\n\n
<\/div>\n\n\n\nTo learn more about the Jetson Mate, be sure to visit its product page<\/a> on the Seeed Online Store!<\/p>\n\n\n\n
<\/div>\n\n\n\nNVIDIA Jetson Nano Module<\/strong><\/a><\/h3>\n\n\n\n
<\/div>\n\n\n\nDesigned specially for AI applications with NVIDIA\u2019s JetPack SDK<\/a>, you can easily build, deploy and manage powerful machine learning applications at the edge with low power consumption with the Jetson Nano and its 128 NVIDIA CUDA Cores.<\/p>\n\n\n\n
<\/div>\n\n\n\n<\/figure><\/div>\n\n\n\n
<\/div>\n\n\n\nProduct Features<\/strong>:<\/p>\n\n\n\n
- Quad-Core ARM Cortex-A57 MPCore Processor<\/li>
- NVIDIA Maxwell GPU with 128 NVIDIA CUDA Cores<\/li>
- 4GB 64-Bit LPDDR4 Memory at 1600MHz 25.6GBps<\/li>
- 16GB eMMC Storage<\/li>
- NVIDIA JetPack SDK for AI Development<\/li><\/ul>\n\n\n\n<\/div>\n\n\n\n
Pick up your very own NVIDIA Jetson Nano Module on the Seeed Online Store<\/a>!<\/p>\n\n\n\n
<\/div>\n\n\n\nNVIDIA\u00ae Jetson Xavier\u2122 NX Module<\/strong><\/a><\/h3>\n\n\n\n
<\/div>\n\n\n\nWhile a tad bit pricier than the Jetson Nano, the Jetson Xavier NX module absolutely pulls out all the stops when it comes to GPU compute power. With 384 NVIDIA CUDA cores and 48 Tensor cores for machine learning, the Jetson Xavier NX is capable of up to a whopping 6 TFLOPS (trillion floating point operations per second) for FP16 values and 21 TOPS (trillion operations) for INT8 values. With the same NVIDIA Jetpack compatibility, the Jetson Xavier NX module will be sure to cover all the bases of your edge computing application.<\/p>\n\n\n\n
<\/div>\n\n\n\n<\/figure><\/div>\n\n\n\n
<\/div>\n\n\n\nProduct Features:<\/strong><\/p>\n\n\n\n
- Compact size SoM powerful enough for advanced AI applications with low power consumption<\/li>
- Supports entire NVIDIA Software Stack for application development and optimization<\/li>
- More than 10X the performance of Jetson TX2 <\/li>
- Enables development of AI applications using NVIDIA JetPack\u2122 SDK<\/li>
- Easy to build, deploy, and manage AI at the edge<\/li>
- Flexible and scalable platform to get to market with reduced development costs<\/li>
- Continuous updates over the lifetime of the product<\/li><\/ul>\n\n\n\n<\/div>\n\n\n\n
Pick up your very own NVIDIA Jetson Xavier NX Module on the Seeed Online Store<\/a>!<\/p>\n\n\n\n
<\/div>\n\n\n\n
\n\n\n\n<\/div>\n\n\n\nUsing NVIDIA GPU Clusters in Edge Computing<\/strong><\/h2>\n\n\n\n
<\/div>\n\n\n\nNVIDIA GPU Clusters powered by the Jetson Mate & Jetson SoMs can help you achieve limitless computing applications on the edge. Here are some example use cases where GPU clusters have become extremely important!<\/p>\n\n\n\n
<\/div>\n\n\n\nGraphics Rendering<\/strong><\/h3>\n\n\n\n
<\/div>\n\n\n\nWe\u2019ve talked about how GPUs have evolved to perform high-throughput general purpose computing, but that certainly isn\u2019t to say that its original role of graphics rendering is now obsolete! Photo, video editing, 3D modelling, virtual or augmented reality are just some of the many relevant modern applications that continue to rely on the traditional functions of GPUs. Unfortunately, laptops equipped with discrete GPUs are large and expensive, whereas desktop solutions lack the portability required in numerous situations.<\/p>\n\n\n\n
<\/div>\n\n\n\nFor a mobile and affordable solution, consider deploying an NVIDIA GPU cluster! Packed in a much smaller form factor yet still delivering significant amounts of power, you can offload graphics intensive workloads from your main computer to the edge GPU cluster to more efficiently process intensive graphics workloads.<\/p>\n\n\n\n
<\/div>\n\n\n\n3D Rendering in Autodesk, Source: Sculpteo<\/em><\/figcaption><\/figure><\/div>\n\n\n\n <\/div>\n\n\n\nMachine Learning on the Edge<\/strong><\/h3>\n\n\n\n
<\/div>\n\n\n\nNVIDIA Jetson clusters specialise in handling edge artificial intelligence (Edge AI) or edge machine learning workloads. Machine learning, or neural networks \/ deep learning in particular, require a considerable amount of computational power due to the great number of calculations that must be performed. As a result, powerful GPUs housed in data centres have long been indispensable for handling simultaneous calculations in machine learning workloads.<\/p>\n\n\n\n
<\/div>\n\n\n\nObject Detection with Machine Learning, Source: Medium<\/em><\/figcaption><\/figure><\/div>\n\n\n\n <\/div>\n\n\n\nToday, however, that is no longer necessarily the case. With an NVIDIA Jetson cluster, you can easily take advantage of powerful hardware acceleration and optimised machine learning libraries and APIs to develop effective solutions for machine learning on the edge. In this way, our edge devices can now be made smarter, with capabilities to perform complex tasks like make predictions, process complex data, and even administer solutions.<\/p>\n\n\n\n
<\/div>\n\n\n\nUsing an NVIDIA Jetson cluster is just one example of how you can take advantage of the shift to Edge AI. In fact, along with it come several key benefits, including reduced latency, reduced bandwidth requirement and cost, increased data security and improved reliability. Read about Edge AI and its transformative effects in Edge IoT in my previous article<\/a>!<\/p>\n\n\n\n
<\/div>\n\n\n\nDeploy Scalable Applications with Kubernetes<\/strong><\/h3>\n\n\n\n
<\/div>\n\n\n\nThese days, it\u2019s almost impossible to talk about cluster computing without mentioning Kubernetes<\/a>, which is an open-source platform for managing containerised workloads and services. While it\u2019s definitely not the only solution available, it is one of the most popular ways to deploy computer clusters in 2021. You can think of it as a management interface that helps you manage your clusters, scaling resources up or down as required to make the most efficient use of your NVIDIA GPU clusters and more!<\/p>\n\n\n\n
<\/div>\n\n\n\nSource: Kubernetes<\/em><\/figcaption><\/figure><\/div>\n\n\n\n <\/div>\n\n\n\n
\n\n\n\n<\/div>\n\n\n\nTutorial: Use Kubernetes with NVIDIA Jetson Clusters for GPU-Powered Computing<\/strong><\/h2>\n\n\n\n
<\/div>\n\n\n\nIn this section, I\u2019m going to show you just how easy it is to set up your very own NVIDIA Jetson Cluster running Kubernetes with the Jetson Mate and the powerful Jetson Nano modules. You can also read the complete tutorial on our Seeed Wiki page<\/a>.<\/p>\n\n\n\n
<\/div>\n\n\n\nRequired Materials<\/strong><\/h3>\n\n\n\n
<\/div>\n\n\n\nTo follow along with this tutorial, the following items are recommended. Take note that you will need to have at least two Jetson Nano modules, since we will need one to act as the master node, and the other to act as the worker node.<\/p>\n\n\n\n
- Jetson Mate Carrier Board<\/a><\/li>
- Jetson Nano Module<\/a> (at least 2)<\/li>
- Qualified Type-C Power Adapter (65W minimum) with PD Protocol<\/li><\/ul>\n\n\n\n
<\/div>\n\n\n\nNaturally, you can also skip the trouble of buying individual components by opting directly for the Jetson Mate Cluster Standard<\/a> or Jetson Mate Cluster Advanced<\/a>!<\/p>\n\n\n\n
<\/div>\n\n\n\nInstall & Configure Jetson OS<\/strong><\/h3>\n\n\n\n
<\/div>\n\n\n\nWe will have to configure the operating system for each of the modules using NVIDIA\u2019s official SDK manager.<\/p>\n\n\n\n
<\/div>\n\n\n\nFirst, choose the target hardware as shown below.<\/p>\n\n\n\n
<\/div>\n\n\n\n<\/figure><\/div>\n\n\n\n
<\/div>\n\n\n\nThen, choose the OS and Libraries you want to install:<\/p>\n\n\n\n
<\/div>\n\n\n\n<\/figure><\/div>\n\n\n\n
<\/div>\n\n\n\nDownload and install the files. While downloading, insert the Jetson Nano compute module into the main node of the Jetson Mate.<\/p>\n\n\n\n
<\/div>\n\n\n\n<\/figure><\/div>\n\n\n\n
<\/div>\n\n\n\nFollowing this, short the 2 GND pins <\/strong>according to the picture shown.<\/p>\n\n\n\n
<\/div>\n\n\n\n - Jetson Nano Module<\/a> (at least 2)<\/li>
- Jetson Mate Carrier Board<\/a><\/li>
- Jetson Mate Cluster Advanced<\/strong><\/a> with 4 Jetson Xavier NX<\/a> SoMs<\/li><\/ul>\n\n\n\n
- Load Balancing<\/strong> \u2013 Spreads computing workloads evenly across slave nodes to handle high job volumes.<\/li>
- High Availability<\/strong> \u2013 Ensures that applications are always available by rerouting requests to another node in the event of a failure.<\/li>
![\"\"](\"https:\/\/lh3.googleusercontent.com\/lP2XOSTPMOqg2R_DShFp04ug6UiZc1oWIu5Uoym_WLLxaxJQOqeNphtk49wcTJUZ6CVWXRWRC2mX0ItyNwCRS5QqqYakg-UR2fB1lDSLU_fVXOK4on6NsX1UQFLRJ9WcqRZRRs56\")
![\"\"](\"https:\/\/lh3.googleusercontent.com\/VhKHWfF7iu6uxD6GfQdYY2TgHo9iFnjTF0U1ae2h79Jz_BVLqSIuviDmoAQmTkinPDEBLNWOwPOW0D3x0lmAhSsfWg82OvSh0DXmorVLc78WXN5_PmFVawpDNqUCP1Umh9f7yYnS\")
![\"Jetson](\"https:\/\/lh6.googleusercontent.com\/BBEdGZ1Qkt0d7FQg0PGf8SRzTdw_-Iqa1QuiRCqB_QGRzwRuVJnXY4U4a0O5mcCoOvjG0_u29A7pqH8cFPp0kAuj7e9F-nZMcY22IzQNJrAH6YsGFdjT67BCYoTKDvpfHoBs0Ovk\" "\\"Jetson")
![\"embedded-jetson-sw-stack-diagram-update-white\"](\"https:\/\/lh4.googleusercontent.com\/uYGDyVBGq62tl54EvFOGGCPfBMICU8Z_GYnMlRb3XRuwAGDXuA4tdERmK2zt0rhnmwC1_JAw0rYb6_uBn_6Z9P8YcjazznB3-Cdt-Pd7_0HUwIAZ7cfBV9L3oBirS3LsqVVhxuwB\" "\\"embedded-jetson-sw-stack-diagram-update-white\\"\/")
![\"\"\/](\"https:\/\/lh4.googleusercontent.com\/7-RMFQQBePQ-sGgTHW20NIGjBnzYREFnkawN53ZUrQeJtxad_XJiMlg0wyxfcL1C2GADgKxLrc_XQDEKDyvfaIjpASO-C0FLnacNWK4ykbEGXUzeiRfTk-3h178VVjr9nupz1PNS\")
![\"\"](\"https:\/\/lh5.googleusercontent.com\/1wRm6xzYp6KRNjuwdVgGKVZ8NZFJSFCIBj2_A47PzwFP7SLcGWB7MfS25FM8A7j48btDpZ5srBX9Pl8cD9_CwEoZFawYgXf9mUprh1uTy7Zqcf9jUOtec1Fni2GTXrAlyLV0wUqy\")
![\"\"](\"https:\/\/lh3.googleusercontent.com\/I5y2rtr3eoA2uazE-U2UrVSZcb3tHxD8djG834B13lkEFbVS-qeW4mu9T3jSIpkYIJRzyothEIeHJao_-EdQl2EpgT1LmyHjEE7XHKIzBPRCkZgV0xcNmL4LTdbs_UIGzVYoIndM\")