In this session, we will build the fire detection frontend using Next.js with TanStack Query and polling.
First, we will refer to the official documentation to install and configure TanStack Query.
Create a providers.tsx file in the same directory as basic layout.tsx and write the following code.
We will refer to the example in the official documentation to write the `providers` file.
I accidentally omitted this, but you must include `use client` at the top.
Wrap the `children` part of the layout file with the provider component.
Run the project and start writing the code in earnest.
We will first write the UI structure in JSX and then add detailed functionality.
Now, I will explain the part where the connected camera list is displayed on the screen.
our code is a system that detects fires using a webcam.
In this process, we need to select one of the multiple cameras connected to the computer.
The implementation of fetching camera information will be done later.
This select tag displays the list of cameras.
The user can select the desired camera from the list.
Create a `selectedDevice` state to store the selected camera.
Set the `onChange` function so that when a camera is selected from the `select` tag, the value updates accordingly.
This code uses an array called `devices`.
`devices` contains information about all available cameras.
Specify the `MediaDeviceInfo` type to enable type support.
This array is iterated using the map function, And each camera's information is displayed as an option tag.
Set a `key` for each `option` in the `map` and assign a `value` to each `option`.
Here, `device.label` represents the camera name.
If there is no name, it defaults to displaying Camera along with part of its ID.
Create a `state` to store the status of the `stream`.
Display the `Start` and `Stop` buttons based on the state of `isStreaming`.
When `isStreaming` is `true`, display `Stop`, and when `false`, display `Start`.
I will modify the `onClick` part later.
I will write the `onClick` functions for `Start` and `Stop`.
Handle errors using `try-catch`.
Use `getUserMedia` to fetch the camera stream for a specific `deviceId`.
`selectedDevice` is the ID of the camera selected by the user.
In other words, the video stream is retrieved based on the `id` of the camera selected in the `select` dropdown.
Now, I will explain the key part of the code that starts the webcam.
First, `videoRef.current` is a reference object pointing to the `` tag.
This means we can directly manipulate the video element on the screen.
Declare a videoRef using useRef.
The `stream` obtained from the webcam is assigned to `videoRef.current.srcObject`.
This allows the video captured by the webcam to be displayed on the screen.
Next, `await videoRef.current.play();` is executed.
This is the code that starts video playback.
Since the `await` keyword is used,
It waits until the video actually starts before executing the next line of code.
`setIsStreaming(true);` is executed.
This indicates that the webcam is currently running.
Finally, `setIsPolling(true);` is executed.
This signals that frames should now be periodically sent to the server.
Log a message to verify that the webcam has started successfully.
Now, I will explain the part of the code that stops webcam streaming.
First, the `stopPredict` function is responsible for stopping the webcam.
To turn off the webcam, we need to clean up the currently running video stream.
we check whether `videoRef.current` and `videoRef.current.srcObject` exist.
This ensures that the video element exists and that the webcam is currently running.
Next, we use `getTracks()` to get all currently active tracks.
Webcam video data is managed in units called tracks.
This code stops all tracks.
In other words, it prevents the webcam from sending any more video.
Then, we set `videoRef.current.srcObject` to `null`.
This removes the webcam stream from the video element.
As a result, the video disappears from the screen.
Finally, `setIsStreaming(false);` → Sets the state to indicate that the webcam has stopped.
Then, `setIsPolling(false);` → Signals that the process of sending data to the server should also stop.
I will now modify the `onClick` function that was left unchanged earlier.
Connect the `startPredict` function to the `Start` button and the `stopPredict` function to the `Stop` button.
Now, I will explain the part that displays the webcam video on the screen.
First, the `` tag is responsible for displaying the video received from the webcam on the screen.
Here, `ref` is used to connect `videoRef` to the video element, allowing direct manipulation.
This enables JavaScript code to control the video.
The `autoPlay` option makes the video play automatically.
Once the webcam is connected, the user does not need to press a play button manually.
The `playsInline` setting ensures that the video does not switch to fullscreen on mobile devices but plays inline instead.
This helps users watch the video seamlessly within the webpage.
I will make some simple CSS adjustments.
Now, I will explain the code that searches for camera devices.
This code uses `useEffect` to run only once when the page is first loaded.
The `enumerateDevices` method retrieves all media devices connected to the computer.
Among them, only video input devices, meaning cameras, are selected through filtering.
The retrieved camera list is then stored using `setDevices`.
Additionally, if at least one camera is connected, The first camera is selected by default.
In summary, this code retrieves and stores the list of available cameras, And automatically selects the default camera.
Finally, we call the `getDevices` function.
Now, I will explain the code that captures a video frame.
This function converts the current webcam screen into an image.
First, it checks whether the video size is correct.
If there is no size, capturing is not possible, so the process stops.
Next, a canvas element is created, And its width and height are set according to the video size.
`getContext('2d')` is used to prepare for drawing 2D graphics.
Then, `drawImage(videoElement, 0, 0);` is executed, Drawing the video frame onto the canvas.
Finally, `canvas.toBlob()` is used To convert the image into a JPEG-format Blob data.
In summary, this code allows the current video frame to be saved as an image.
Now, I will explain the code that sends a video frame to the server.
First, it checks whether the webcam is running.
If `videoRef.current` does not exist or `isStreaming` is `false`, It means the webcam is not running, so the process stops.
Next, `captureFrame(videoRef.current)` is called, To convert the current video frame into an image.
If the conversion fails, it does not send the data to the server and stops.
Now, prepare to send the image to the server.
Create a `FormData` object, And append the image with the key `'file'`.
Now, use `fetch` to send the data to the server.
The request is sent to `"http://localhost:8000/predict_fire"`.
The request method is `POST`, And the request body contains `formData`.
Finally, convert the server response to JSON and log it.
In summary, this code captures a video frame, And sends it to the server to request fire detection.
Now, I will explain the code that sends automatic requests using TanStack Query.
First, `queryKey` is used for caching data.
Here, the key `"fireDetection"` is used to manage the results.
`queryFn` is the function that will be executed.
This executes the `sendFrameToServer` function, Which sends a video frame to the server and retrieves the fire detection results.
`enabled` determines whether the request should be executed.
It only runs if the webcam is active and data transmission is enabled.
`refetchInterval` sets the interval for automatically refreshing the data.
Here, it sends a request to the server every 5 seconds.
Finally, `retry` determines whether to automatically retry the request if it fails.
In this case, retries are disabled.
In summary, this code ensures that when the webcam is running,
It automatically sends an image to the server every 5 seconds,
And retrieves the fire detection results.
Check the `data` in the `console.log`.
To resolve CORS issues when making API requests,
we added the following code to the `main.py` file in FastAPI.
We used `CORSMiddleware` to allow requests from all domains.
Requests are allowed from all origins,
with all HTTP methods and headers permitted.
This configuration should only be used in a development environment,
And in a production environment,
it is recommended to apply restrictions as needed.
Finally, I will correct a few mistakes and wrap things up.
Add `"use client"` at the top of the `providers` file.
I will update the API request URL.
I will check the functionality in the browser.
If a fire is detected, `"fire detection"` is displayed in the `message`.
Next time, we will implement a simple modal and alert notification when a fire is detected.