Hi everyone.
Welcome back.
In this lecture we are going to explore one of the most exciting additions to the Java Stream API,
which is the stream gatherers.
Until Java 21 streams API were limited to predefined operations like map, filter and flatMap for intermediate
operations.
But now starting from Java 22, Java gives us a brand new way of building custom intermediate stream
operations.
And that's exactly what stream gatherers do.
They let you manipulate the data flowing through a stream in ways that were previously impossible,
like working with sliding windows, calculating running totals, or processing elements concurrently.
We'll explore all of them in action.
In the following lectures, you can find all the built in gatherers in the Java.util Stream Dot gatherers
class.
We'll explore that also in this lecture.
So in this section we'll go step by step and explore what gatherers are and why we need them and how
to use them effectively in your code.
Let's talk about why do we need gatherers in the first place?
The traditional stream model in Java is pull based.
Each stage in the pipeline asks for data from the stage before it.
For example, in this case, when you call collect or for each, the downstream operation pulls the
data from the upstream.
So in this case, the list of fruits is going to be passing the elements in the collection one by one.
After we call the foreach operation.
But gatherers flip this behavior, they make the stream push based.
This means the upstream can now decide when and how to send elements downstream.
It's not going to be the terminal operation that's going to be pulling the elements one by one.
It's going to be the gatherer operation that's going to be pushing the elements downstream.
Let me quickly show you an example to understand this behavior.
So in this example here we have the gather operation which is an intermediate operation.
And what it does is that here we have gatherers dot windows fixed of size three.
This means that it's going to be gathering the elements of size of three elements, and then push those
three elements downstream.
So in this case to the for each operator.
This gives us much more control, especially for stateful or batch oriented processing like grouping
elements into windows, computing running totals or averages, or performing event streaming analytics.
In short, gatherers are going to make the streams more powerful and expressive for real world data
problems.
We are going to be exploring some real time examples.
Once we explore those examples, this concept will be much more clearer.
So in this slide I'm going to be talking about the first gatherer, which is window fixed.
This is exactly what we saw in the previous slide also.
But in this slide I'm going to go through those windows fixed operator in detail.
So this one groups the elements into fixed size batches.
So basically when we call the gatherer operation it's going to be gathering the results from the original
collection into elements of three.
That's what you see over here.
So here this is the movies is the actual collection.
And then from the collection we do the gatherer dot window fixed of size three.
This is going to be collecting the elements from the original collection into batches of three.
So in this case let's say you have this movies collection of size ten.
This is going to create four batches.
Three batches will have three elements.
And the last batch will have one element.
And once that is collected then that list will be passed down to the for each operation.
So in this case you can see that once the list of three elements of movie is collected, then it will
push the batch to this for each operation.
Now you have access to the whole list over here.
Once we have this list, then we can actually use that list.
And actually we are printing that in the console over here.
So this is going to emit a window containing a list of type whatever that your actual stream is containing
a subset of stream elements.
So this is again going to create a stream.
That's really important because you have a stream and that's going to be pushed down to the for each
operation.
So this is a concept of gathering it and pushing it downstream.
I hope the explanation is clear.
Now let's jump into the code and understand the gather API in detail and implement a basic example using
the windows fixed gather operator.
So I'm going to go back to the IntelliJ.
So we are going to be working on this Stream Gatherers demo class.
So that's what this one is.
So I'm going to be searching for that gatherer API first gatherer.
You can actually include this non-project items.
There you go.
This is the gatherer operator.
So in this case you can see that.
So this is gatherer interface.
It accepts three elements actually the type of the input element.
And then the potential mutable state type of gatherer operation.
And then the output of the gatherer operation.
Internally gatherers work through an integrator which receives this input streams.
Basically you have the original stream.
An integrator is the one which is going to be holding the actual intermediate state, and this is the
one which will also decide when to emit the results downstream.
So think of gatherers as like a smart transformers sitting in your stream pipeline.
They can accumulate, hold and release elements based on the custom logic.
This is what makes them more flexible than normal map or filter operations.
Now let's switch back to the code and then code this very first function, which is going to be the
demonstrate windows fixed function.
And the data that we are going to be using for this particular example is the list of movie domain.
Lets me go ahead and then show you what this domain is.
So this domain has five properties.
Number one is a title of the movie, the genre and the release date and the rating of the movie and
the duration.
And if we go back.
So here we have this create sample movies function.
So this function has a list of bunch of movies of different genres.
So here you have drama movies, action movies science fiction romance.
So these are the different genres.
And each and every movie has different dates rating and duration.
So the first thing that we are going to be exploring is the windows fixed.
Okay.
Let's go back to this one.
And this is getting the input of list of movies.
Let's go ahead and use that one.
And then I'm going to be calling the stream function.
So after the stream function we have access to the gather function call which is an operator.
And this gather operator accepts a gatherer, actually.
Okay.
So that's what it accepts.
If we go to the gather function again it accepts a gatherer.
So we are going to be using some inbuilt gatherer to start with.
We will build some custom gatherers as we go down in the future part of the course.
Let's go back to the code here.
And the gatherer that we are going to use is gatherer dot windows fixed.
And then I'm going to give the value as three.
So what this is going to return.
So this is going to give you access to the stream of list of movies actually.
So that's going to hold batches of three movies from the original movie collection.
So this is going to be an intermediate operation creating a fixed windows of size threes okay.
And what this is going to do, this is going to be passing it down to this for each function.
And we are going to be accessing to that data.
So I'm going to give a window.
As you can see this is going to give you access to the list of movies.
And after that I'm actually getting the access to the window.
And then I'm actually printing each and every movie over here.
Okay, let me put this on top.
Okay.
So this is our very first gatherer function.
What this one does is that it's going to be acting on the original movies collection, and we're streaming
on it.
We are doing a gather over here.
This gather is going to take care of gathering the movies of.
List of three elements as a stream.
And then when we call foreach, this gets pushed down to the.
For each foreach is a terminal operation.
And now this has access to the list of movies.
And we are basically printing them in the console.
Let me go ahead and execute this program.
There you go.
So now this is the actual movie list that you can see.
And then we are actually gathering a list of three movies.
In this case if we go to the demonstrate windows fixed, we are actually printing each window.
And each window has access to list of three elements.
And as you can see that it's going to be printing all the elements in the console.
So this is the beauty of gatherers.
You have the ability to batch the result and then pass it downstream.
So if you think about it, this helps you create windows of exactly the specified size and windows do
not overlap.
And final window may contain fewer elements.
If the stream size is not divisible by the window size, what does it mean actually?
So in this case, let's say you have a collection of ten movies.
Okay, now it will be broken down into four windows of size three, and the last window is going to
be of size one.
I think it's three windows actually not four three windows of size three.
And the last window of size one.
I hope you all have a pretty good idea about how gatherers work, and you can see the result one more
time.
So in the result we have windows fixed three fixed size windows.
So we have window one has three elements.
Window two has three elements window four and window.
So totally there are four windows.
Basically we have 12 movies in the original collection.
I hope the explanation is clear.
Let's go back to the slide and quickly highlight the differences between the streams and gatherers.
So collectors are used at the end of the stream, right when you have processed everything, you are
ready to combine all the results into one final output, for example a list or map.
So this is how we do the collectors.
Actually, in the case of gatherers, they are intermediate operations and they are push based and they
have the ability to emit the intermediate results so it can collect the result and then emit the intermediate
result to the downstream.
But ultimately you need a downstream operation like in collector in order to receive the gathered results.
So in this example, if you take a look at it, we have gathered the result here.
But if you do not have a terminal operation, there is no way you can access this gathered result.
So if you need fine grained control while elements are flowing through your pipeline gatherers or the
tool to use.
This marks the end of this lecture.
Thank you for watching.