Letter three, Project one.
This is recognizing gestures with light sensor specifically for model training and development.
Now, in the last lesson, we get the data samples we need to train the model before it can be used
by a neural network.
Data needs to be cleaned up.
Sometimes you just change the range of data.
For example, we might change the range from zero to 1000 to zero to one.
Neural networks work better with smaller numbers than large ones.
People who work on tiny email often use more sophisticated pre-processing techniques to get so-called
features from real data, which speeds up the training process.
And in this lesson, we will learn more about pre processing functions available in edge impulse and
train simple, fully connected networks.
One preparation you should make is that you need to make sure you have Arduino IDE installed and working
with Real Terminal after you collected the samples.
It is time to design an impulse now on impulse.
Here is the word edge impulse used to denote data processing, training, pipeline bressan, create
impulse and set window length.
Four thousand ATMs and window length increased to 15 minutes.
Now you can change these settings to make sure that each time and inferences mean we are going to take
sensor measurements for 1000 amps.
It is set to 40 hertz during data collection, which means that 40 times a second, 40 times a minute.
So in a nutshell, your device is going to collect 40 data samples in a tiny window of a thousand miles.
It will take these data samples to process them and feed them to a neural network to get an answer.
We, of course, use the same window size while we are learning how to use it.
And now for this proof of concept project, we are going to try three different processing blocks with
default parameters, except for adding scaling flatten block, which takes computes average minimum
blocks and other functions of real data within time window.
Spectral features block, which extracts the frequency and power characteristics of a single.
Over time, we have two blocks one called raw data, which sends raw data to the end and learning block
and another called learning.
This is optionally normalizing the data.
Flooding block is the first thing we do today.
Add this block, then add neural network, which is us as a learning block.
We check the bulletin as input features and click on save impulse to save the changes you just made.
Choose the processing block you want to flatten, then go to the next one, which has the name of processing
block you choose.
Flatten and there is a scaling field where they put zero point zero zero one and leave the rest of the
scene when you're done.
Click Save parameters, then generate.
Then future visualization is particularly the useful tool in edge impulsive web interface, as it allows
users to get graphical insights into how the data looks after pre-processing.
For example, this is the data after flood in processing block.
Here you can see in documentation.
In this picture, we can see that the data points for different types of things are roughly divided
now there is a lot of overlap you can see here between rock and other types of things which will cause
problems and make the data less accurate for these types of things.
After you've made and looked at the features, go to the classifier.
Tom Gene, a simple, fully connected network with two hidden layers 20 and 10 neurons in each hidden
layer for 500 epochs with a learning rate of one each for this network has two hidden layers 20 and
10 neurons in each.
Test results will be shown in a conflict matrix, which looks something like this.
And then right after that, go back to create post-op, delete, flatten, block and just spectral features
block.
Generally, the features.
Now you need to remember that the set scaling to zero 0.1 and trained neural network on special features
need down here.
You should see slight improvement.
No, both flatten and spectral features blocks are actually not the best processing methods for Rock-Paper-Scissors
gesture recognition.
Think about it if we want to classify Rock-Paper-Scissors gestures.
We just need to keep track of how many times and how long the light sensor has been getting values that
are lower than normal.
If it's been a long time, then it's rock feast passing above the sensors.
There are seesaws.
If there are two times that if you see source beeper is anything more than that.
Sounds easy, but be serving time seriously.
That is really important for neural network to be able to learn this relationship in data points.
Now, flattening spectral features, processing blocks, they actually both remove the time relationship
in each window.
The flattened blocks simply turns the real values, which are in order into average, minimum and maximum,
and other values that are calculated in the values in the time window.
No matter how they are arranged, because this task was so simple, the spectral features block didn't
work as well as it should have, because the duration of each gesture was too short.
That means that the best way to get the best performance is to use your raw data block.
This will give the time series data safe.
Take a look at the sample project where we use raw data in a network that isn't connected as fully connected
network.
We were able to get ninety two point four percent of the same data right the first time around.
For now, we're going to use a simple, fully connected model and spectral features processing to meet
the model and look at the features and then both flatten and spectral features.
Blocks are actually not the best processing methods, as I've said earlier.
Then the final results after training were flat and FC.
We had a sixty nine point nine accuracy spectral features FC seventy point four percent accuracy.
Then the raw data called 1D ninety two point four per cent accuracy.
Then the next slide we will be discussing on convolutions and how useful they are in later articles
about sound processing.
For now, we're going to use simple, fully connected model and spectral features processing.
After you train the model, you can use the Lib Classification tab to see how well it works.
Now this will take a sample of data from your device and classify it with the model that's on edge impulse.
Now, we tried three different gestures and see that the accuracy actually somehow good enough to show
that the idea is real.
Now the next step is deployment on device here.
After clicking on deployment to choose Arduino Library and download it, extract the archive and place
it in your Arduino Libraries folder.
Make sure to open Arduino IDE and choose Static Buffer Sketch, which is located in file and then examples
that locate for name of your project.
And then you could see Static Buffer, which already has all the boilerplate code for classification
with your model place.
Now, the only thing for use to fill in this data acquisition and device will use a simple for loop
with delete to account for frequency.
And if you remember, we had 25 MHz delay when gathering data for training data.
Now, RMS stands for millisecond.
Let's practice for a drop.
If we were to do this, we could use a sensor data buffer that would let us do inference more often.
There will be more lessons in the course about that now, following your changes to this sample code
uploaded through real terminal and open a serial monitor.
When you move your hand while making a gesture, you can see the probability results on the serial monitor,
and that same goes for the paper and same goes for seesaws.
Now, when it was just a proof of concept demonstration, it actually really shows tiny animals up to
something big, right?
You probably knew it.
It is possible to record these gestures with a camera sensor.
Even image is downscaled a lot, but recognizing gestures with just one pixel is entirely different
level.
Now try increasing or decreasing number of neurons in first and second hidden layers and look or see
how that affects the accuracy and inference time.