Edge AI – What is it, Why is it important, and How to implement Edge AI?

Edge AI, short for edge artificial intelligence, is immensely popular right now. It’s the next frontier of development for Internet of Things (IoT) systems – but how much do you really know about it? Join me in this article to learn all about Edge ML and how industry leaders are using it to change the way we live and work.

This article will cover the following content:

  • What is Edge AI?
  • How does Edge AI Benefit Us?
  • Why is Edge AI Important?
  • TinyML at the Edge 
  • Choose hardware of SBC, GPU, TPU, and Microcontrollers for Edge AI
  • Edge AI solutions  
  • Getting Started with Edge AI – Device, Projects & More

What is Edge AI?

In the simplest terms, Edge AI refers to the use of artificial intelligence in the form of machine learning algorithms running directly on edge devices.

Machine learning is a broad field that has seen tremendous progress in the recent years. It is based on the principle that a computer can autonomously improve its own performance on a given task by learning from data – sometimes even beyond the capabilities of humans.

Today, machine learning can perform many advanced tasks, including but not limited to:

  • Computer vision: image classification, object detection, semantic segmentation
  • Speech recognition, natural language processing (NLP),  chatbots, translation,
  • Forecasting (Eg. Weather & Stock Markets), Recommendation systems
  • Anomaly detection, Predictive maintenance

You might wonder: Machine learning has been around for so long, what makes Edge AI suddenly so special? To shed some light on this, let’s first take a look at what edge in Edge AI really means.

Edge Computing vs Cloud Computing

In essence, both edge and cloud computing are meant to do the same things – process data, run algorithms, etc. However, the fundamental difference in edge and cloud computing is where the computing actually takes place.

In edge computing, information processing occurs on distributed IoT devices that are on the field and in active deployment (or on the edge). Some examples of edge devices include smartphones, as well as various SBCs and microcontrollers.

In cloud computing, however, that same information processing occurs at a centralised location, such as in a data centre.

Source: California Technical Academy

Traditionally, cloud computing has dominated the IoT scene. Because it is powered by data centres that naturally have greater computing capabilities, IoT devices  on the edge could simply transmit local data and maintain their key characteristics of low power consumption and affordability. Although cloud computing is still a very important and powerful tool for IoT, edge computing has been getting more attention lately for two important reasons.

Reason 1: The hardware on edge devices has become more capable while remaining cost competitive.

Reason 2: Software is becoming increasingly optimised for edge devices.

This trend is progressing tremendously that it is now possible to run machine learning, which has long been ‘reserved’ for cloud computing due to high computational requirements, on edge computing devices! And thus, Edge AI was born.

Edge AI: Bringing Cloud to the Edge to Evolve IoT

With Edge AI, IoT devices are becoming smarter. What does that mean? Well, with machine learning, edge devices are now able to make decisions. They can make predictions, process complex data, and administer solutions.

For example, edge IoT devices can process operating conditions to predict if a given piece of machinery will fail. This allows companies to perform predictive maintenance and avoid the larger damages and costs that would have been incurred in the event of a complete failure.

On the other hand, a security camera equipped with edge AI may no longer only capture video. We will be able to identify humans and count foot traffic. Or, with facial recognition, even identify exactly who has passed through an area and when. Very “big brother”-esque, I know; but still amazing nonetheless!

Source: Medium

As machine learning develops, many exciting possibilities will now extend to edge devices as well. But the crux of this paradigm shift is clear – more than ever, cloud capabilities are being moved to the edge; and for good reason.

Benefits of Edge AI

1.  Reduced Latency

The most direct advantage of processing information on the edge is that there is no longer a need to transmit data to and from the cloud. As a result, latencies in data processing can be greatly reduced.

In the prior example of preventative maintenance, an Edge AI enabled device would be able to respond almost immediately to, for instance, shut the compromised machinery down. If we used cloud computing to perform the machine learning algorithm instead, we would lose at least a second of time during to the transmission of data to and from the cloud. While that may not sound significant, every margin of safety that can be achieved is often worth pursuing when it comes to operation-critical equipment!

2.  Reduced Bandwidth Requirement and Cost

With less data being transmitted to and from edge IoT devices, there will be a lower requirement and thus costs in network bandwidth.

Take an image classification task for example. With a reliance on cloud computing, the entire image must be sent for online processing. But if edge computing was used instead, there would no longer be a need to send that data. Instead, we can simply send the processed result, which is often several orders of magnitude smaller than the raw image. If we multiply this effect by the number of IoT devices in a network, which can be as many as thousands or more, these savings are nothing to scoff at!

3.  Increased Data Security

A reduction in the transmission of data to external locations also means less open connections and fewer opportunities for cyber attacks. This keeps edge devices operating safely out of the reach of a potential intercept or data breach. Furthermore, since data is no longer stored in the centralised cloud, the consequences of a single breach are heavily mitigated.

4.  Improved Reliability

With the distributed nature of Edge AI and edge computing, operational risks can also be distributed across the entire network. In essence, even if the centralised cloud computer or cluster fails, individual edge devices are able to maintain their functions since the computing processes are now independent from the cloud! This is especially important for critical IoT applications, such as in healthcare.

Why is Edge AI Important?

While the tangible benefits of Edge AI are clear, its intrinsic impacts may be more elusive.

Edge AI Alters the Way we Live

For one, Edge AI represents the first wave of truly integrating artificial intelligence into daily living. While artificial intelligence and machine learning research have been around for decades, we are now just starting to see their practical uses in consumer products. Self-driving cars, for example, are a product of advancements in Edge AI. Slowly but surely, Edge AI is changing the way that we interact with our environment in many ways.

Edge AI Democratises Artificial Intelligence

The use and development of artificial intelligence is no longer exclusive to research institutions and wealthy corporations. Because Edge AI is designed to run on relatively affordable edge devices, it is now more accessible than ever for any individual to learn how artificial intelligence works and to develop it for their own uses.

More importantly, Edge AI makes it possible for educators around the world to bring artificial intelligence and machine learning into classroom learning in a tangible manner. For example, by providing students with hands-on experience with edge devices.

Edge AI Challenges the Way We Think

It’s commonly said that the potentials of AI and machine learning are only limited by the creativity and imagination of humankind – and this could not be more true. As machine learning becomes more advanced, many tasks that once only humans could do will become automated, and our internal notions of productivity and purpose will be heavily challenged.

While there’s no telling for certain what the future will hold, I’m generally optimistic about what Edge AI brings to the table as I believe that it will push us towards more creative and fulfilling jobs. What do you think?

Edge AI with GPU: NVIDIA Jetson embedded system

About NVIDIA Jetson Modules 

NVIDIA® Jetson™ brings accelerated AI performance to the Edge in a power-efficient and compact form factor. The Jetson family of modules all use the same NVIDIA CUDA-X™ software, and support cloud-native technologies like containerization and orchestration to build, deploy, and manage AI at the edge. Together with NVIDIA JetPack™ SDK, Jetson modules will help you develop and deploy innovative products across all industries.

Buy the latest NVIDIA Jetson Production module powered edge vices, carrier boards, and add-ons at Seeed

NVIDIA Jetson compatible Edge AI platform at Seeed

reComputer built with Jetson Nano/NX: real world AI at the Edge, starts from $199

Built with Jetson Nano 4GB/ Xavier NX 8GB/16GB

  • Edge AI box fit into anywhere
  • Embedded Jetson Nano/NX Module
  • Pre-installed Jetpack for easy deployment
  • Nearly same form factor with Jetson Developer Kits, with rich set of I/Os
  • Stackable and expandable

Edge AI with TPU: Google Coral series 

Google’s Coral series is equipped with their Tensor Processing Units (TPUs), which are purpose-built ASICs designed specially for neural network machine learning with TensorFlow Lite. The Coral Dev board is an all-in-one, TPU-equipped platform that allows you to prototype TFLite applications easily, which can even be scaled to production with its flexible SoM design!

Exclusive for educators: 

Join the Coral Global Education Program partnered with Seeed to receive an exclusive discount of 35% off of the Coral Dev Board 1GB list price of $129.99 to qualified applicants if you are a student or an educator. Please fill out the questionnaire to apply to the program and get an exclusive educational discount. Please also feel free to email [email protected] to find out more about the education program.

Product Features:

  • NXP i.MX 8M SoC (Quad Cortex-A53, Cortex-M4F) CPU
  • Integrated GC7000 Lite Graphics
  • Google Edge TPU coprocessor
  • 1 GB LPDDR4, 8GB eMMC
  • Suite of Interfaces: HDMI, MicroSD, WiFi, Gigabit Ethernet and more!

To learn more about the Coral Dev Board, please visit its product page on the Seeed Online Store!

Alternatively, you might be interested in the Coral USB Accelerator, which allows you to use Google’s Edge TPU with your existing development boards via USB! For embedded applications that require more power, the Coral M.2 Accelerator with Dual Edge TPU can be used with any system via the speedy M.2 interface, and sports two Edge TPUs for some serious machine learning capabilities!

Best SBCs for Edge AI

Edge AI is developing rapidly, and nobody wants to miss out. Fortunately, getting started with Edge AI is easier than ever with the wide variety of edge devices available. In this section, I’d like to share a few SBC recommendations for those of you who would like to get your toes wet with Edge AI!

Intel Celeron powered ODYSSEY X86 series

When it comes to general purpose computing, you’ll be hard pressed to find anything better than the ODYSSEY x86 series. Running on the powerful x86 CPU architecture, this SBC is capable enough to meet any edge computing requirement, or even serve as a mini PC to replace your desktop.

Product Features:

  • Intel® Celeron® J4125, Quad-Core 1.5-2.5GHZ
  • Dual-Band Frequency 2.4GHz/5GHz WiFi
  • Intel® UHD Graphics 600
  • Dual Gigabit Ethernet
  • Integrated Arduino Coprocessor ATSAMD21 ARM® Cortex®-M0+
  • Raspberry Pi 40-Pin Compatible
  • 2 x M.2 PCIe (B Key and M Key)
  • Support Windows 10 & Linux OS
  • Compatible with Grove Ecosystem

Interested to learn more? Learn more about the ODYSSEY x86J4158 on the Seeed Online Store now!

Beginner-Friendly: Raspberry Pi 4B

If you’re a beginner or if you’re looking for a beginner-friendly option for your child, you most certainly won’t go wrong with the Raspberry Pi 4B. Although the popular Raspberry Pi isn’t as powerful as the other recommendations I’ve shared so far, it is still a very capable SBC for anyone who is trying to learn Edge AI or general computing. You’ll also benefit greatly from the extensive range of projects and support from the Raspberry Pi community!

Edge AI solutions

Edge AI solution for Industry 4.0

With Edge AI, IoT devices are becoming smarter. What does that mean? With machine learning, edge devices are now able to make decisions. They can make predictions, process complex data, and administer solutions. For example, edge IoT devices can process operating conditions to predict if a given piece of machinery will fail. This allows companies to perform predictive maintenance and avoid the larger damages and costs that would have been incurred in the event of a complete failure 

Helmet Detection 

Edge devices embedded with AI are capable of monitoring PPE including hard hats compliance in the work environment in real-time and signaling any PPE violations to safety and maintenance. Computer vision combined with machine learning can automate the process of monitoring PPE compliance.

Check out our Helmet Detection Brochure by builing applications using Edge Impulse and YOLOv5.

Edge AI solution for Traffic Management 

Build intelligent transportation system for smart city and public safety

AI integrated cameras that can ease the bottlenecks and chokepoints that often hinder traffic in our cities. Traffic congestion occurs mainly due to the neglect of certain factors, such as the distance between two moving vehicles, traffic lights, road signs, and pedestrians at the crossroad. The intelligent transportation system (ITS) is a major computer vision applied field, including vehicle classification, traffic violation detection, traffic flow analysis, parking lot detection, license plate recognition, pedestrian detection, traffic sign detection, collision avoidance, and road condition monitoring, and in-vehicle driver attention detection.

Check out our Edge AI solution brochure of intelligent traffic management with examples of Deploying NVIDIA Jetson-powered applications into smart cities utilizing fast training & deployment platforms and pertained models. 

  • Pedestrian and Bicyclists detection Car and people detection using alwaysAI
  • License plate detection 

TinyML: Is Edge AI Possible on Microcontrollers?

Throughout this article, we’ve talked about how Edge AI is fitting artificial intelligence applications into smaller and less powerful computers. But really, how small can we go? How about microcontrollers that have only kilobytes of RAM?

As it turns out, the answer is a resounding YES – thanks to a new concept known as TinyML!

TinyML, short for Tiny Machine Learning, is a subset of machine learning that employs optimisation techniques to reduce the computational space and power required by machine learning models. Specifically, it aims to bring ML applications to compact, power-efficient, and most importantly affordable microcontroller units.

Further fuelling the TinyML movement, companies like Edge Impulse & OpenMV are helping to make Edge AI more accessible through user-friendly platforms. Now, anyone can deploy machine learning applications almost literally anywhere, even without prior expertise!

Best Microcontrollers for Edge AI

I know what you’re thinking. Being able to run artificial intelligence on a microcontroller that costs less than 5 dollars sounds like such a steal! Well, don’t worry because I’ve got you covered with the following microcontroller recommendations for Edge AI.

Seeeduino XIAO

The Seeeduino XIAO is the smallest Arduino compatible board in the Seeeduino Family. Despite its small size, the Seeeduino XIAO is equipped with the powerful SAMD21 microchip and a variety of hardware interfaces. It truly puts the tiny in TinyML!

Product Features:

  • ARM Cortex-M0+ 32bit 48MHz microcontroller (SAMD21G18) with 256KB Flash, 32KB SRAM
  • Compatible with Arduino IDE & MicroPython
  • Easy Project Operation: Breadboard-friendly
  • Small Size: As small as a thumb(20×17.5mm) for wearable devices and small projects.
  • Multiple development interfaces: 11 digital/analog pins, 10 PWM Pins, 1 DAC output, 1 SWD Bonding pad interface, 1 I2C interface, 1 UART interface, 1 SPI interface.

Keen to learn more about the Seeeduino XIAO? Visit its product page on our Seeed Online Store now!

Wio Terminal

The Wio Terminal is a complete Arduino development platform based on the ATSAMD51, with wireless connectivity powered by Realtek RTL8720DN. As an all-in-one microcontroller, it has an onboard 2.4” LCD Display, IMU, microphone, buzzer, microSD card slot, light sensor & infrared emitter. The Wio Terminal is officially supported by Edge Impulse, which means that you can easily use it to collect data, train your machine learning model, and finally deploy an optimised ML application!

Product Features:

  • Powerful MCU: Microchip ATSAMD51P19 with ARM Cortex-M4F core running at 120MHz
  • Reliable Wireless Connectivity: Equipped with Realtek RTL8720DN, dual-band 2.4GHz / 5GHz Wi-Fi (supported only by Arduino)
  • Highly Integrated Design: 2.4” LCD Screen, IMU and more practical add-ons housed in a compact enclosure with built-in magnets & mounting holes
  • Raspberry Pi 40-pin Compatible GPIO
  • Compatible with over 300 plug&play Grove modules to explore with IoT
  • USB OTG Support
  • Support Arduino, CircuitPython, Micropython, ArduPy, AT Firmware, Visual Studio Code
  • TELEC Certified

If you’re interested to pick up a Wio Terminal, please visit its product page on the Seeed Online Store!

Edge AI Projects & Getting Started

Edge AI is still relatively new, with different companies still in the process developing their own smart solutions. Despite this, there are already many resources to help you get started with building your very own Edge AI project!

Jetson AI courses and certifications

NVIDIA’s Deep Learning Institute (DLI) provides developers, educators, students, and lifelong learners with practical, hands-on training and certification in edge computing. It’s a great way to gain the critical AI skills you need to thrive and advance in your career. After completing these free open source courses, you can even earn a certificate to demonstrate your understanding of Jetson and AI. 

Use transfer learning along with Ultralytics YOLOv5 and Roboflow to train a dataset with very few samples.

Check out more details and 👉 try it follow our wiki!

In the wiki we showed the difference in training time between using a small dataset collected by ourselves and a large dataset available publicly. Also, we will use the trained model to perform faster inference on an edge device such as the NVIDIA Jetson platform with better accuracy.

Smart Weather Station with TFLite on the Wio Terminal

Weather stations are a popular project amongst the maker community. Why not take it one step further and add Edge AI capabilities to enable local weather predictions? This project by Dimitry Maslov does exactly that – visit the full article here for all the details!

Materials Needed:

This tutorial is also part of our Learn TinyML using Wio Terminal and Arduino IDE series. Be sure to check each of them out!

Machine Learning Powered Inventory Tracking with Raspberry Pi

This project uses machine learning powered object detection to count objects in a photo! The inventory numbers are then uploaded to Azure IoT Central so that the inventory can be monitored anytime, anywhere.

Materials Needed:

Keen to try this for yourself? Visit my complete step-by-step tutorial here!

Build Handwriting Recognition with Wio Terminal & Edge Impulse

Do you think it’s possible to perform handwriting recognition with just a single distance sensor? The answer to that is, well, sort of! This project uses machine learning on time series data from just one ToF sensor to recognise handwriting gesture patterns! While it’s very much a proof of concept project and far from actual implementation, I hope this inspires you to think of crazy ideas for your own project!

Materials Needed:

Like always, you can visit the full step-by-step tutorial here!

Summary & More Resources

Thanks for reading this article! I hope I’ve managed to shed some light on what Edge AI is and what it means for the future of IoT and even humankind. With SBCs and microcontrollers now joining the fray, there’s no better time to explore machine learning applications and build some Edge AI projects for yourself!

To wrap up, here are some more resources which you may find useful:


April 2021