In this video tutorial, we will learn how we can create a Streamlit application to do object detection,
instance segmentation and pose estimation using Ultralytics.
YOLO 11.
Here is a quick demo of the Streamlit application that we will be creating in this tutorial.
So let me first give you a quick demo of the Streamlit application that we are building in this tutorial.
So we can perform object detection, instance segmentation and pose estimation with the help of this
Streamlit application.
And we will be able to perform object detection, instance segmentation and pose estimation on images
and on videos.
You can upload any random image like.
You can click over here from Raw files and you can upload any image from here.
Okay, as you upload the image, an image will appear over here.
And if you want to perform a object detection, you can see over here we have by default the object
detection is being marked over here.
And then you can click on Detect object.
And over here you will see an output image.
Once the object detection is being performed.
So now you can see over here we have the output image.
And you can see we are able to detect bus person uh backpack handbags.
So you can see the detection results over here like you can see now uh and over here you can see we
have the, uh, bounding box coordinates over here in the form of tensors.
Okay, so we have the bounding box coordinates for each of the detected object in the form of tensors.
Okay.
So as you expand this, you will see all the bounding box coordinates.
And here you can see over here we have are able to detect a person backpack and bag person or bus and
all the things.
Okay so currently I'm using YOLO 11 nano model.
And if you use uh large models of YOLO 11 you will get definitely better results.
So going ahead you can also perform a segmentation as well.
So if you want to perform the segmentation on the same image, you can click on segmentation from here.
And you can just go down from here and click on Detect Object.
Okay.
So now you can see over here we are able to perform the segmentation.
We are able to take the person backpack.
And we are able to draw the mask as well.
So basically in instance segmentation we can draw a mask around each of the detected object.
And you can see over here we have we are able to draw a blue mask around each of the detected object
okay.
In the same way you can also perform pose estimation.
And if you click on pose estimation from here and going below, if you click on Detect object from here.
So now you can see over here we are able to perform the pose estimation.
And you can see that we have drawn the key points around each of the detected object.
Like you can see over here we are able to perform pose estimation.
And we are able to draw or draw the key points around each of the detected object.
Okay.
In the same way, if you want to upload some other image, you can just click on upload files from here
and you can upload this image from here.
Okay.
And after uploading this image, if you want to perform pose estimation, I have already selected pose
estimation.
And you can click on Detect Objects.
So now you can see over here we are able to detect the object.
Plus we are also able to draw the key points around each object object.
So we have performed pose estimation.
In the same way.
If you want to perform segmentation you can just click on segmentation from here and you will be able
to perform segmentation.
Now you can see over here we are able to detect the object as well as we are able to draw a mask around
each of the detected object over here.
Okay.
So we have seen that how we can perform object detection, instance segmentation and pose estimation
on the image.
Let's go to the video part.
So if you want to perform object detection, instance segmentation and pose estimation on video, you
can just select video from here as the source.
And you can see this is the default video.
Let's first see how we can perform object detection, instance segmentation and pose estimation on a
video.
So in my video directory I have added the two different videos.
You can add as many videos you want, so as many videos you add in your video directory, you will be
able to perform object detection, instance segmentation and pose estimation on those videos.
Okay, so if you select video one and if you want to perform object detection like let's click on Detect
Object.
From here.
And here you can see the output.
We are able to detect the person over here okay.
So like this is the wrong detection.
Currently I'm using the 11 nano model.
And if you want to get good results you can use a higher YOLO 11 models which include your 11 large.
Your or 11 extra large.
Uh, the reason I'm using a small yellow model, like 11 nano is that I'm currently running, uh, the
Streamlit application on my CPU, and I don't have GPU in my local machine, so YOLO 11 is the fast.
YOLO 11 nano is the fastest, but it is least accurate, while YOLO 11 Extra Large is the most accurate.
But it takes more inference time.
Okay, so there is a trade off between speed and accuracy.
Okay, so if you want to go with another video like you can see over here, you can click on Detect
Video Objects and you can see over here we are able to detect the person bicycle over here.
And detection results look quite promising.
Okay.
And if you want to perform segmentation you can click on segmentation from here and click on Detect
Video Objects.
And now you can see that we are able to perform segmentation.
We are able to draw a mask around each of the directory object.
Uh the person are in the blue mask which you can clearly see over here.
And if you want to perform pose estimation, you can just click on Pose Estimation and click on Detect
Video Objects.
And now you can see that we are able to detect the objects.
And we are also able to draw key points around each of the object over here.
You can also just a confidence threshold, like if you increase the confident threshold to 0.933 like
93%, you can see detection has um like there are no detections.
And if I just reduce the confidence to 70 let's see.
So now you can see that, uh, it will only detect the object that have a confidence score above 70%
or 0.7.
So all the objects that have confidence score above 70% will be detected.
So what does confidence score means?
Like the confidence score means how much the model is confident that this is a person or this is a bicycle
okay.
So if you just increase the confidence score there will be less false positives.
But if you decrease the confidence score like maybe to 25, or if you further increase the confidence,
go to 10% or 0.10.
You will get more detections, but there will be more false positives as well.
Okay, but if you increase the uh, confidence score, there will be more accurate detection and there
will be less false positives, but there will be few detections as well.
Okay.
So this is how our Streamlit application works.
And we will be creating this Streamlit application in this tutorial.
So let's get started with this.
So over here you can see that I have just created a project in PyCharm Community Edition.
And I have requirements.txt file.
And we are we have Ultralytics and Streamlit package over here.
So if you create this application we only require two packages.
One is Ultralytics and Streamlit.
And to install this is you can simply right pip install.
R requirements dot txt.
And if you run this you will be able to install these two packages.
So I've already installed these packages and you can see over here requirements already satisfied.
But if you are installing these packages for the first time this will take some time.
Next we will just create a new file like Main.py file.
In the step number one, we will import all the required libraries that we require in this project.
So we require the OpenCV Python package.
We require the Streamlit library.
To define the path.
Of our root directory we require the Pathlib primary.
From there we import path.
Then to require SSIS library.
Then to go to object detection using Ultralytics YOLO 11.
We require a YOLO package software report from Ultralytics.
We will import YOLO.
And to display any image we require image lab.
Okay.
So let's create a new page layout and with the help of Streamlit.
So we will be now creating an initial page layout.
So we'll write speed dot set dash page dash config.
And the page title will be.
YOLO 11.
We can also add an page icon.
Next we will define a header for our Streamlit application.
The header will be object detection level.
We can also create a sidebar.
We will have our all the settings like what task we want to perform, what would be the confidence threshold.
And if you want to do object detection on image or on video.
So you need to create a sidebar for this.
In the sidebar we will also add a header with the name model configurations.
In the sidebar.
We will also give the user the option to choose whether you want to perform object detection, instance
segmentation, or pose estimation.
Or the user will have the option to choose the model type.
We are just adding radio buttons for their.
Detection.
Segmentation.
Fault estimation.
We will also give the user the option to select the confidence threshold with the help of slider.
The confidence value will be in the form of float.
The minimum value of confidence will be 25.
The default value will be 40 and the maximum value will be 100.
We will divide it by 100 because our model takes the confidence value in the form of, uh, between
0 to 1, like 0.1 is 10%, 0.25 is 25%, 0.90 is 90%.
Okay.
So we write.
So let's see how each Streamlit application looks like.
So we'll write Streamlit run main.py.
So this will take some time.
Currently this will take some time.
Let me see if there is an error.
It looks fine.
Okay.
So here you can see we have our Streamlit application.
Like you can see over here we have the model configurations cost reduction.
Segmentation.
Cost estimation.
Okay.
Default value.
Why we are seeing this.
Okay.
So the maximum value is 100 and default value is 40.
I have just made this news.
Let's see.
Let me refresh this up.
Okay.
Again I think.
Oh yeah.
Let me refresh this up.
So you can see.
Default value is 40 and the minimum value is 25.
And maximum value is 100.
Okay.
So let's complete this application now.
First we need to get the absolute path of the current file.
Wide.
And we need to get the parent directory of the parent file.
Which is.
Next we will add the root path.
To the system dot.
Or if it's not already there.
I will just say if not in.
This pod.
Dot dot dot.
Append.
Okay.
Next we need to get the relative path.
Of the root directory.
With respect to the current working directory.
Okay.
So we have just got that part to the root directory with respect to the current current working directory.
Uh, that looks fine.
Next.
So we need to define the sources.
Uh, our we can pass our input in the form of image and as well as in the form of video.
Our input will be in the form of image or video.
So let's define a source list.
Or we need to define some default images that will appear if the user doesn't upload an image.
What should be the default image appearing?
I will set some image configurations over here.
So first we need to create a directory by the name images over here.
And inside this directory I will add some free images.
So this is the default detected image that will appear in our Streamlit application.
This is the image one.
This is the default image as well.
And this is another image which will appear in our images directory.
Okay.
So in the root over here we have defined a relative path of the root directory with respect to the current
working Writing directory.
And we need to define our default image over here.
So we can go to images directory.
And over here.
We have image one or jpg.
Then we have default detect image.
And we have detected image one dot okay.
Similarly we can do for the video as well.
For this I will just create another directory where the name videos over here.
And over here I will add two videos.
Okay.
Well first define video directory is equal to root and videos for the name of the directory is videos
okay.
And then we will create a dictionary.
Right.
Video one.
Video directory.
Video one dot.
Then we will write video two.
In your directory media two dot mp4.
Okay.
Next we need to define model configurations where we will create a weight folder over here.
And I will add my uh object detection instance segmentation and pose estimation over 11 models.
So I will just write model configuration.
So first what I will do is I will just create a new directory by the name weights over here.
And inside this I will add what I will do over here is uh we can go over here.
Okay.
So you can just go to your 11 GitHub.
up.
And if you want to download any object detection model or instance segmentation or pose estimation model,
uh, you can simply click over here and you can see that it start downloading.
And if you want to download any segmentation model, you can simply click over here.
And it will now start downloading.
And if you want to download Pose Estimation model you can simply click over here.
It will start downloading okay.
So I have already downloaded so I'm not downloading again.
So I'm just adding these three models over here.
So you can see we have your 11 pose estimation model and your 11 segmentation model.
So I will just write route.
And you just need to go to the weights directory over here.
So we just need to go to model directory.
And here we have the detection model as your 11 nano dash.
So I go to 11 nano that we got it.
So in case of your custom model.
You will write detection.
Dash model.
And here you will write model dash directory.
And over here you will add custom model with dot 80.
And you will just comment this line of code okay.
And uncomment this okay.
Next we need to define the segmentation model.
Next we need to also define the pose estimation model.
Okay.
So that things work.
So we are good for now.
Next uh, if the user uh we need to add the code.
If the user select the detection segmentation or the pose estimation model.
Now if the model die base.
Or a model die based detection.
Model path is equal to path.
Detection model as if.
If the model die based.
Segmentation.
Model path.
Path segmentation model and if model type is equal to for estimation.
On the label or.
For estimation.
On.
So next we will load the YOLO model.
When I try.
YOLO more than five.
And write except exception as e.
S g dot error.
You can see the write unable to load model.
Check uh s file path.
Okay.
Simply write this over here.
And we can also display the error in our Streamlit application.
Now we can just set image and video configuration.
Will give the user the option to select if you want to perform object detection, instance segmentation
and pose estimation on image or on video.
So we'll give the user the option.
We can start sidebar dot radio.
Like source.
Source list.
Sorry.
Initialize source image is equal to none.
So if that.
If the user selects an image.
We will ask the user to upload an image.
And the Streamlit application will accept the image of type.
JPG png.
JPEG.
BMP.
So I only know all these image types.
If you know some other, you can also add it over here as well.
Next we will create two columns which will contain a default image and other will contain the detected
image, or one will contain the.
Use the image that the user has uploaded and other is the our output image with the detections.
So first we will go towards the column one.
So if the source image is none like the user doesn't upload an image.
Then we will define a default image path over here.
And we will say.
This is our default image.
You can also open this image.
And we can also display this image in our Streamlit application using S3 dot image.
Right.
Default image path.
And our caption will be before image.
Else.
If the user uploads an image and source image is not on, then we will display the uploaded image.
So if the user uploads an image, then we will display the source image.
Caption will be uploaded.
Image.
Okay.
And now we will add accept as well.
Except exception as okay.
Okay.
What will be.
Any.
Error occurred.
While.
Opening that and we will.
Display the error as.
Now we'll create another column as well.
But if forcing it is not.
Then we'll define default detected image path over here.
Okay.
And here we'll pass the default detected image.
Path.
Caption will be detected.
Image.
Else if the source image is there, and user also has a sidebar button which is.
Detect objects.
That we will use model dot predict.
Okay.
And here we will pass the uploaded image.
And.
The first uploaded image over here.
Then.
But we also define the confidence value.
It'sand.
That's not the result as well.
And we'll also show the bounding box coordinates there as well.
Right.
With a dot expander.
And we will display the error.
And we can also add section over here as well.
Worthwhile.
Okay.
That works.
Now, let's test, uh, all our Streamlit application works on image.
And if it works fine, then we'll go ahead with the video as well.
Okay, so let's test our Streamlit application.
So here you can see that this is a Streamlit application that we have created so far.
So first uh, let's reduction I will just upload this image.
And if I click on detect object from here.
So this is the uploaded image.
Okay.
So now you can see over here we are able to detect the person purse backpack.
And here you can see we have the bounding box coordinates for each detected object in the form of a
tensor.
We can also do segmentation and let's click on detect object from here.
So now you can see that we are able to form instance segmentation.
And we have drawn mask around each of the detected object.
We can also perform pose estimation.
And let's click on detect object from here.
So now you can see we are able to perform pose estimation.
And we have drawn key points around each of the detected object.
And here you can see we have the bounding box coordinates appearing over here for each of the object.
Okay so we are done with the image part.
Let's go ahead and move towards the uh video part as well.
Okay.
So we can write and if.
Source radio is equal to radio.
Copy.
Right.
Source video is equal to ask not sidebar or select box.
It was a video.
In your dictionary dot keys.
And right will open with your dictionary.
And what source video or read it.
As I so we need to convert the video into bytes so we can perform object detection on the video.
Or S3 dot video will display the video into our Streamlit application.
So if the user click on this button right here.
And.
Then we'll add a try and accept permission.
Okay.
So we'll create an empty frame from now.
Now we'll just create a while loop over here.
And if we are able to read the frame then.
We'll just resize the frame over here.
Next we will predict the objects in the image using YOLO 11.
Right result is equal to model dot.
Predict image is equal to confidence is equal to confidence value.
Next we will plot the detected objects on the radio Rain.
Uh, we already have created an empty frame.
We will be plotting our detected objects on that frame.
And if the video frames end, then we'll release all the frames and we will break the while loop.
And we can also add an accept conditional here.
Around loading video.
Okay so and we will display the error over here as well.
So now we have created our Streamlit application.
Now it's time to test our Streamlit app application.
So we can simply write over here Streamlit run main.py.
Okay so let's test our Streamlit application and let's see if we are able to do detection on image and
video as well.
We have tested for image already now.
Now we will mainly test on video now.
So first of all I will just click on video from here.
And um I can just call it video two and click on Detect Video Objects from here.
And now you can see over here we are able to detect a person bicycle from here.
And the detection results look quite promising.
If you want to perform segmentation you can select segmentation.
From here.
You can select any video that you have saved in the video directory.
And then you will run Detect video objects.
Now you can see that we are able to perform instance segmentation.
We are able to detect a traffic lights bicycle person.
And you can see over here we have drawn a mask around each of the detected object.
So this is what we do.
In instance segmentation.
In instance segmentation, we draw a mask around each of the detected object.
Similarly, we can perform pose estimation and they can detect 3D objects from here.
So now you can see we have performed pose estimation.
And we have drawn key points around each of the detected object.
Similarly you can adjust the confidence threshold to 79 as well.
And you can see now as I increase the confidence threshold the detection has decreased.
But now the detections are more accurate.
But if I decrease the confidence score there can be some false positives as well.
But there have been more number of detections.
Similarly you can do it for image as well.
We have already tried on images.
And detect objects.
And let me see if I have some other image over here as well.
Let's use this image and let's click on detect objects.
So now you can see over here we are able to detect the objects.
And we are also able to draw the key points around each of the detected object because we have selected
for the estimation.
And if I just like segmentation, you can see we are able to draw a mask around each of the detected.
So now you can see over here with the help of instance segmentation, we are able to draw a mask around
each of detected object.
And if I click on detection from here and click on detect object.
From here you can see that I am able to detect a different objects like person bus.
And over here if I just expand this I have all the bounding box coordinate informations in the form
of tensors over here.
So that's all from this tutorial.
In this tutorial we have learned how we can create a Streamlit application to do object detection,
instance segmentation and pose estimation with the help of ultralytics YOLO 11 on images as well as
on videos.
That's all from this video.
Thank you for watching.