Machine learning possible on microcontrollers

ARM’s Zach Shelby introduced the use of microcontrollers for machine learning and artificial intelligence at the ECF19 event in Helsinki on last Friday. The talk showed that that artificial intelligence and machine learning can be applied to small embedded devices in addition to the cloud-based model. In particular, artificial intelligence is well suited to the devices of the Internet of Things. The use of machine learning in IoT is also sensible from an energy efficiency point of view if unnecessary power-consuming communication can be avoided (for example local keyword detection before sending voice data to cloud more more detailed analysis).

According to Shelby , we are now moving to a third wave of IoT that comes with comprehensive equipment security and voice control. In this model, machine learning techniques are one new application that can be added to previous work done on IoT.

In order to successfully use machine learning in small embedded devices, the problem to be solved is that it has reasonably little incoming information and a very limited number of possible outcomes. ARM Cortex M4 processor equipped with a DSP unit is powerful enough for simple hand writing decoding or detecting few spoken words with machine learning model. In examples the used machine learning models needed less than 100 kilobytes of memory.

zackdscf6473

The presentation can be now viewed on YouTube:

Important tools and projects mentioned on the presentation:

TinyML

TensorFlow Lite

uTensor (ARM MicroTensor)

TensorFlow Lite Micro

Articles on presentation:

https://www.uusiteknologia.fi/2019/05/20/ecf19-koneoppiminen-mahtuu-mikro-ohjaimeen/

http://www.etn.fi/index.php/72-ecf/9495-koneoppiminen-mullistaa-sulautetun-tekniikan

 

218 Comments

  1. Tomi Engdahl says:

    Using an Edge Impulse tinyML model and an OLED library from Adafruit Industries, Atul Yadav developed a Nano 33 BLE Sense system that detects and displays when a myna bird sings.

    Visual Indication for bird chirping (Mynah song detection)
    https://m.youtube.com/watch?v=9ibp7H7qPnk

    Reply
  2. Tomi Engdahl says:

    ML in Crop Quality and Environmental Tracking: A look at how many #environmental and #agricultural problems can be addressed within the same framework of audio analysis and environmental sensing through #machinelearning. https://bit.ly/3Ab89bf

    A Continuously Sprouting Project: ML in Crop Quality and Environmental Tracking
    https://www.sparkfun.com/news/3915?utm_content=171277991&utm_medium=social&utm_source=facebook&hss_channel=fbp-153488801415
    A look at how many environmental and agricultural problems can be addressed with the same framework of audio analysis and environmental sensing through machine learning!

    Reply
  3. Tomi Engdahl says:

    A FANtastic predictive maintenance project by Edge Impulse ambassador Andri Yadi! This system uses a tinyML model on an Arduino Nano 33 BLE Sense to classify fan operation and identify faults.

    https://m.youtube.com/watch?v=GB6ArqH_eLo&feature=youtu.be

    Reply
  4. Tomi Engdahl says:

    AIfES is a standalone AI framework that allows on-device training without a PC and works on almost any hardware. Even the 8-bit Arduino Uno!

    AIfES is an AI/ML framework written in C for even the smallest microcontrollers
    https://blog.arduino.cc/2021/07/06/aifes-is-an-ai-ml-framework-written-in-c-for-even-the-smallest-microcontrollers/

    Reply
  5. Tomi Engdahl says:

    Recognising Bird Sounds With A Microcontroller
    https://hackaday.com/2021/07/06/recognising-bird-sounds-with-a-microcontroller/

    Machine learning is an incredible tool for conservation research, especially for scenarios like long term observation, and sifting through massive amounts of data. While the average Hackaday reader might not be able to take part in data gathering in an isolated wilderness somewhere, we are all surrounded by bird life. Using an Arduino Nano 33 BLE Sense and an online machine learning tool, [Errol Joshua] demonstrates how to set up an automated bird call classifier.

    The Arduino Nano 33 BLE Sense is a fully featured little dev board that features the very capable NRF52840 microcontroller with Bluetooth Low Energy, and a variety of onboard sensors, including a microphone. Training a machine learning model might seem daunting to many people, but online services like Edge Impulse makes the process very beginner-friendly.

    a massive online library of bird calls from all over the world is available on Xeno-Canto.

    https://hackaday.io/project/180538-bird-sound-classifier-on-the-edge

    Reply
  6. Tomi Engdahl says:

    This device uses machine learning to detect the ripening stages of various fruits and vegetables by spectral color.

    Vegetables and Fruits Ripeness Detection by Color w/ TF © CC BY
    https://create.arduino.cc/projecthub/kutluhan-aktar/vegetables-and-fruits-ripeness-detection-by-color-w-tf-041f92

    Collate spectral color data of varying fruits and vegetables and interpret this data set with a neural network to predict ripening stages.

    Reply
  7. Tomi Engdahl says:

    RECOGNISING BIRD SOUNDS WITH A MICROCONTROLLER
    https://hackaday.com/2021/07/06/recognising-bird-sounds-with-a-microcontroller/

    Machine learning is an incredible tool for conservation research, especially for scenarios like long term observation, and sifting through massive amounts of data. While the average Hackaday reader might not be able to take part in data gathering in an isolated wilderness somewhere, we are all surrounded by bird life. Using an Arduino Nano 33 BLE Sense and an online machine learning tool, a team made up of [Errol Joshua], [Ajith KJ], [Mahesh Nayak], and [Supriya Nickam] demonstrate how to set up an automated bird call classifier.

    https://hackaday.io/project/180538-bird-sound-classifier-on-the-edge

    Reply
  8. Tomi Engdahl says:

    ‘Droop, There It Is!’ is a smart irrigation system that uses ML to visually diagnose drought stress
    https://blog.arduino.cc/2021/07/13/droop-there-it-is-is-a-smart-irrigation-system-that-uses-ml-to-visually-diagnose-drought-stress/

    Reply
  9. Tomi Engdahl says:

    Aksha is a Nano 33 BLE Sense-equipped pencil that uses tinyML to recognize different Hindi letters drawn in the air: https://medium.com/@naveenmanwani/aksha-an-arduino-based-ml-pencil-powered-by-tensorflow-lite-micro-e7ba854f42f3

    Reply
  10. Tomi Engdahl says:

    Using a tinyML model on the Nano 33 BLE Sense, this device automatically detects when someone is snoring and begins to vibrate as an alert.

    The Snoring Guardian listens while you sleep and vibrates when you start to snore
    https://blog.arduino.cc/2021/07/20/the-snoring-guardian-listens-while-you-sleep-and-vibrates-when-you-start-to-snore/

    Snoring is an annoying problem that affects nearly half of all adults and can cause others to lose sleep. Additionally, the ailment can be a symptom of a more serious underlying condition, so being able to know exactly when it occurs could be lifesaving. To help solve this issue, Naveen built the Snoring Guardian — a device that can automatically detect when someone is snoring and begin to vibrate as an alert.

    Reply
  11. Tomi Engdahl says:

    Can tinyML enable you to “hear” the difference between pouring hot and cold water? Marcelo Rovai and Marco Zennaro used the Nano 33 BLE Sense and Edge Impulse to find out.

    “Listening Temperature” with TinyML © GPL3+
    https://create.arduino.cc/projecthub/mjrobot/listening-temperature-with-tinyml-7e1325

    Can we “hear” a difference between pouring hot and cold water? Amazing proof-of-concept by a quick real deployment using Edge Impulse Studio

    Reply
  12. Tomi Engdahl says:

    A̶I̶ A-Thigh. This wearable system counts your squats using a TensorFlow Lite model on the Nano 33 BLE Sense.

    Squats Counter Using TensorFlow Lite and Tiny Motion Trainer © MIT
    https://create.arduino.cc/projecthub/Manasmw333/squats-counter-using-tensorflow-lite-and-tiny-motion-trainer-4d9dcf

    Squats counter that can detect squats by measuring the accelerometer readings and prints it on the Serial monitor.

    Reply
  13. Tomi Engdahl says:

    ICYMI! During last week’s Arm #AITechTalk, Massimo Banzi showed off our machine learning-capable boards, some new tools and frameworks to support your project, and a few example tinyML applications to help get you started.

    https://m.youtube.com/watch?v=mj2J445pdNc&feature=youtu.be

    Reply
  14. Tomi Engdahl says:

    Is it possible for a neural network to “hear” the difference between pouring hot and cold water? Marcelo Rovai and Marco Zennaro decided to use a sound classification model on the Nano 33 BLE Sense to find out.

    “Listening Temperature” with TinyML © GPL3+
    https://create.arduino.cc/projecthub/mjrobot/listening-temperature-with-tinyml-7e1325

    Can we “hear” a difference between pouring hot and cold water? Amazing proof-of-concept by a quick real deployment using Edge Impulse Studio

    Reply
  15. Tomi Engdahl says:

    This tutorial shows how to train an artificial neural network using the AIfES ML framework on a PC and run it on your Arduino board afterwards.

    How to Use AIfES on a PC for Training © GPL3+
    https://create.arduino.cc/projecthub/aifes_team/how-to-use-aifes-on-a-pc-for-training-9ad5f8

    This tutorial shows how to train an artificial neural network with AIfES on a PC and run it on your Arduino board afterwards.

    Reply
  16. Tomi Engdahl says:

    BABL is a tinyML-powered baby monitor that uses a Nano 33 BLE Sense with Edge Impulse’s new EON Tuner to distinguish cries from other household sounds.

    BABL x EON: A Baby Monitor Tuned by Edge Impulse! © MIT
    https://create.arduino.cc/projecthub/ishotjr/babl-x-eon-a-baby-monitor-tuned-by-edge-impulse-b8aa30

    Edge Impulse automatically improves model accuracy with one weird trick!

    In the original BABL project, we achieved 86.3% accuracy without a ton of effort, but what if we could improve that number – and do so without a ton of manual tweaking and training? That’s the concept behind Edge Impulse’s new EON Tuner, which automatically generates the optimal model based on your target and dataset type. Here’s a look at how EON Tuner can dramatically improve accuracy in just a few clicks!

    Reply

Leave a Comment

Your email address will not be published. Required fields are marked *

*

*