The document class is one of Link Chain's core building blocks, and it serves as a standard container
for handling text.
And it's designed to package a piece of text along with contextual information.
So every document object consists of two main parts.
We have the page content, and this holds the actual text like a paragraph from an article or a page
from a PDF or any other text.
And we have the metadata.
And this is a dictionary that stores additional details about the text like the source where it's coming
from, maybe the file name, or maybe a URL, or maybe page number or some custom tags that we want
to add it.
And this structure is super important for our link chain Rag workflows.
So when we load data from various sources which LinkedIn supports, it supports tons of integrations.
It's converted into a list of documents objects.
And these can be broken down into smaller chunks later which themselves are documents instances.
So the attached metadata is super useful when we want to enable some filtering or retrieval logic later.
The pipeline, which is very crucial for advanced applications.