Tutorial: Run Yolo2 on Sipeed Maix Bit

Part 1: Train a Yolo2 objects detector

As demo in the class, you can train your own objects detector on your own dataset. This part mainly use MobileNet and Yolo2. You can also design the network or formulate the task by yourself. Here, I follow this tutorial to train a raccon detector.



Create a new conda environment and install the dependencies:

Git clone the code and dataset:

Change the images dirs inside the configs:

Train the model:

After training:


And you can find the saved model:


You can play with some hyperparameters in the config file such as the actual_epoch, batch_size, learning_rate, or anchors. They influence the final performance of your model. For example, if you use a larger dataset containing more training images and more detection classes, you should train your model by more epochs for better convergency. Or, if the targeting objects inside your image is typically small, you can use smaller anchors' parameters (it defines the prior anchors for detector. you can find more detail in FIg.2 in Yolo2 paper).

Part 2: Convert model to .kmodel

In this part, we use the Maix toolbox provided by Sipeed to convert the .tflite model into .kmodel which can be run on the board.



First of all, let's get the Maix toolbox from their GitHub repo:

Then, fix a bug:

Download the library:

In order to convert the model, we need some example images for inference. These example images have to be the same resolution as the input image's resolution to your model. In this case, we define the input size in the configuration file raccoon.json:

As a result, we need some images with 224x224 size for inference. I directly use the training images we used to train our racoon detector by Keras. To make sure those images are 224x224, I use Imagemagick package to directly resize them:

You can use your prefered way to do resizing.

Then, do convert:

If you successfully convert the model, you will see:


It shows you the detail of your model such as the I/O dimension of InputLayer, OutputLayer, and each Conv2d layer, as well as the memory usage of this converted model.

Part 3: Get your board ready

In this part, we burn Maixpy Firmware onto the board to run the micropython script. Here, I use the GUI package provide by Sipeed to burn the firmware. In order to use my laptop to demo this board at anywhere, I switch back to my MacBook. Because OSX is pretty similar to Ubuntu, it should be OK to keep using Ubuntu to complete this part.



First, get the firmware:

Or download from here.

Then, clone the Kflash GUI and install the requirements and packages (.bin file):

Run the Kflash GUI:

Let's burn the firmware:


If you successfully burn the firmware, you will see:


Instead of using this GUI, you can also use the kflash.py to burn the firmware. It is also inside the kflash_gui folder. However, you have to make sure you use the correct arguments for the process function.

In some case, you want to use light weight Maixpy Firmware, Sipeed also provide a minimum version:

For example, if you want to burn your .kmodel into the Flash memory simultaneously with the firmware, you have to burn a light weight firmware. By this light weight firmware, you cannot use the provided MaixPy IDE which I will introduce later to run a micropython script. Instead, you have to use Serial I/O to execute it (I will also intoduce it in the next part). Otherwise, either the some parts of firmware or your model would be earsed. Both case will cause you to fail to run the model. And there is no error message or warning message. Thus, I suggest to use a SD card to store the model and plug it into the board when you need it.

You can find different versions' firmware here. But in order to use the Maixpy IDE later, I suggest to use v0.4.0_52.

Part 4: Run the model

In this part, I use the provided MaixPy IDE and Serial I/O to execute the micropython script to run the .kmodel on the board. Instead of run the .kmodel by micropython, you can also run it by C. In this case, you have to use Arduino IDE or Platform I/O to burn your code onto the board.


Get the IDE:

Install it after downloading. Then, run it. First thing we need to check is the Board version:


Then, check if it can connect to the board:


If you cannot connect to your board, you may (1) select a wrong board version or (2) burn a wrong Maixpy Firmware. For example, you cannot use this IDE to connect the board when you burn the minimum version firmware. Double check you burn a correct version of Maixpy Firmware.

Open the Yolo-digit-detector/micropython_code/racoon_detector.py on this IDE and run it:


If you successfully run it, you will see:


The right part of this IDE shows the captured image and its RGB value in real-time.

Finally, you can test your racoon detector on some random racoon image from Google Image:


Let's see how to execute the micropython script by Serial I/O when you want to use a minimum version Maixpy firmware. In this example, I run another .kmodel which is a tiny-yolo2 trained on PASCAL dataset. Bsiaclly, use the screen to connect the board:

And, you will see:


Then do Crtl+E and copy-paste all the micropython script into it, and press Crtl+D to execute it. It basically put all the code in the script into one line then send them to the board:


From the script, you can find there are 20 classes (you can download from here): aeroplane, bicycle, bird, boat, bottle, bus, car, cat, chair, cow, diningtable, god, house, motobike, person, pottedplant, sheep, sofa, train, tvmonitor which are defined in the PASCAL dataset. And I put the .kmodel in the Flash memory starting from 0x500000.

Finally, let's do some test:



MaixPy IDE: http://dl.sipeed.com/MAIX/MaixPy/ide

MaixPy Firmware: http://dl.sipeed.com/MAIX/MaixPy/release/master/

MaixPy DOC: https://maixpy.sipeed.com/en/(https:/github.com/sipeed/MaixPy

MaixPy micropython example scripts: https://github.com/sipeed/MaixPy_scripts

Kflash tollbox: https://github.com/sipeed/kflash_gui

Sipeed BBS: https://bbs.sipeed.com

Train a flower classifier: https://iotdiary.blogspot.com/2019/07/maixpy-go-mobilenet-transfer-learning.html

Train a racoon detector: https://www.instructables.com/id/Object-Detection-With-Sipeed-MaiX-BoardsKendryte-K/

Train a MobileNet classifier: https://www.instructables.com/id/Transfer-Learning-With-Sipeed-MaiX-and-Arduino-IDE/

C code for running MobileNet classifier: https://github.com/AIWintermuteAI/transfer_learning_sipeed/tree/master/mobilenet_v1_transfer_learning

Run 20 classes TinyYolo2 on board: https://bbs.sipeed.com/t/topic/683

Yolo: https://pjreddie.com/yolo/

Yolo2: https://arxiv.org/pdf/1612.08242.pdf

MobileNet: https://arxiv.org/pdf/1704.04861.pdf

CSE599 G1 18au: https://courses.cs.washington.edu/courses/cse599g1/18au/

CS231n: http://cs231n.github.io