Skip to content

[Feature Request]: Metadata in documents for improve the prompt #3690

@devMls

Description

@devMls

Is there an existing issue for the same feature request?

  • I have checked the existing issues.

Is your feature request related to a problem?

I have several documents with different information , and I need refer the information with the context of his document, for achieve that LLM understard better the information

Describe the feature you'd like

Add metadata to documents in database documents

Describe implementation you've considered

with the document metadata, the prompt can be structured like that:

I think that the metadata of the documents and the metada of the knowledge database should be inserted in the prompt. In this way:

The knowledge database ${DATABASE_NAME} that has a description ${DATABASE_DESCRIPTION} has this documents with information:

Document ${DOCUMENT_NAME} with description ${DOCUMENT_DESCRIPTION}. This document has following attributes
${METADATA1_NAME} : ${METADATA1_VALUE}
${METADATA2_NAME} : ${METADATA2_VALUE}
This document ${DOCUMENT_NAME} has the following information:
${CHUNK 1}
${CHUNK 2}
End of ${DOCUMENT_NAME}

Document ${DOCUMENT_NAME} with description ${DOCUMENT_DESCRIPTION}. This document has following attributes
${METADATA1_NAME} : ${METADATA1_VALUE}
${METADATA2_NAME} : ${METADATA2_VALUE}
This document ${DOCUMENT_NAME} has the following information:
${CHUNK 3}
${CHUNK 4}
End of ${DOCUMENT_NAME}

End of ${DATABASE_NAME}

This other database ${DATABASE_NAME} that has a description ${DATABASE_DESCRIPTION} has this documents with information:

Document ${DOCUMENT_NAME} with description ${DOCUMENT_DESCRIPTION}. This document has following attributes
${METADATA1_NAME} : ${METADATA1_VALUE}
${METADATA2_NAME} : ${METADATA2_VALUE}
This document ${DOCUMENT_NAME} has the following information:
${CHUNK 5}
${CHUNK 6}
End of ${DOCUMENT_NAME}

Document ${DOCUMENT_NAME} with description ${DOCUMENT_DESCRIPTION}. This document has following attributes
${METADATA1_NAME} : ${METADATA1_VALUE}
${METADATA2_NAME} : ${METADATA2_VALUE}
This document ${DOCUMENT_NAME} has the following information:
${CHUNK 7}
${CHUNK 8}
End of ${DOCUMENT_NAME}

End of ${DATABASE_NAME}

Documentation, adoption, use case

No response

Additional information

I am not sure how that can affect to rerank process.

Metadata

Metadata

Assignees

Labels

💞 featureFeature request, pull request that fullfill a new feature.

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions