Crazyflie AI-deck V1.1 – with GAP8 RISC-V MCU & ESP32 Wi-Fi

Features 

• Ultra-low-power GAP8 8+1 core RISC-V processor, 512 Mbit HyperFlash and 64 Mbit HyperRAM for faster speeds

• ESP32-based NINA-W102 Wi-Fi module to stream images to PC

• Delivers over 10 GOPS of computing power at exceptionally low power consumption

• Capture, analyze, and classify its environment with ULP camera

• Perfect for micro-sized area research with Crazyflie 2.X UAV.

• Perfect for complex artificial intelligence-based workloads like Convolutional Neural Networks to run onboard and path-finding projects

Description

What is Crazyflie?

Crazyflie is a versatile open-source flying development platform that only weighs 27g and fits in the palm of your hand. Crazyflie is equipped with low-latency/long-range radio as well as Bluetooth LE. This gives you the option of downloading an app and using your mobile device as a controller or, in combination with the Crazyradio PA, using your computer to display data and fly with a game controller. 

Description of Crazyflie AI-deck 1.1

Crazyflie AI-deck 1.1 enables low power on-board artificial intelligence capabilities for the Crazyflie using a GAP8 chip with RISC-V multi-core architecture, 512 Mbit HyperFlash and 64 Mbit HyperRAM. This lightweight and low-power combination opens up many research and development areas for the micro-sized Crazyflie 2.X UAV.

In addition, the AI-deck 1.1 has a Himax HM01B0 Bayer RGB camera with a NINA W102 Wi-Fi module to stream your images to a desktop as well as handling control.. These features fit the prerequisites of a convolutional neural network, but the AI-deck is not limited to the application of CNN's.

Workflow using GAPFlow

To design a neural network and deploy it on the AI-deck 1.1, you should know the workflow of the GAP8 SDK for AI applications that is provided by GreenWaves Technologies. A neural network can be designed, trained, and evaluated using Tensorflow and Keras in Python. To let this code be able to run on the AI deck 1.1, an automated process is executed by the GAPFlow of the GAP8 SDK.

What you should provide in this workflow is:

• dataset with labels

• neural network model

• GAP application code

• optional: own autotiler operator

Check out the GAPflow Model Extern explanation in the ai-example folder

 Tensorflow and Keras use

Keras is a framework within Tensorflow. In addition to providing a neural network framework, it also provides examples of common and simple neural networks and provides datasets along with it. Though when you want to make an application you might want to use other datasets that are relevant for your application. For this, you have to supply your own dataset with labels, augmentation and transformations when required.

https://keras.io/datasets/

 NNTool use

The NNTool makes use of post-training quantization and adjusts the quantized weights using a selection of images used for training the neural network.

https://github.com/GreenWaves-Technologies/gap_sdk/tree/master/tools/nntool

Block Diagram

Specifications

• Power supply 3V-5V @ VCOM up to 300mA

• 1-wire memory for automatic expansion board detection 

• UART connected between GAP8 and Crazyflie (RX1, TX1) 

• UART connected between ESP32 (RX2, TX2)

• ESP32 sysboot pin connected to Crazyflie (IO_1)

• Reset to GAP8 and ESP32 connected to Crazyflie (IO_4)

• SPI between GAP8 and ESP32

• GAP8 (B1) -> ESP32 (GPIO_5) IO

• ESP32 (GPIO_25) -> GAP8 (A13) IO

• 2 x Cortex-M 10-pin JTAG Interface to program on the GAP8 and ESP32

• Can be easily mounted on the Crazyflie V2.1 with the 2 x 10-pin male headers

• Button connected to ESP32 for UART bootloader or other actions

• Green LED status indicators for GAP8 and ESP32

Mechanical drawing

Technical Details

Part List