[C-API] Initial support for accessing file system - #19086
Merged
Merged
Conversation
duckdb-draftbot
marked this pull request as draft
September 23, 2025 09:39
Member
Author
|
Since we always have a filesystem in the client context (even if all are disabled, we still have a |
Maxxen
force-pushed
the
c-api-filesystem
branch
from
September 23, 2025 09:41
7a9df84 to
b761e54
Compare
Maxxen
marked this pull request as ready for review
September 23, 2025 10:24
duckdb-draftbot
marked this pull request as draft
September 23, 2025 11:29
Maxxen
marked this pull request as ready for review
September 23, 2025 12:50
taniabogatsch
left a comment
Member
There was a problem hiding this comment.
Thanks for the changes! Left more comments, mostly nits. :)
carlopi
reviewed
Sep 23, 2025
duckdb-draftbot
marked this pull request as draft
September 23, 2025 20:49
Maxxen
marked this pull request as ready for review
September 23, 2025 20:50
Member
|
thanks! |
krlmlr
added a commit
to krlmlr/duckdb-r-old
that referenced
this pull request
Oct 21, 2025
[C-API] Initial support for accessing file system (duckdb/duckdb#19086) Skip serialization of row_start in DataPointer when targeting latest storage (duckdb/duckdb#19111) Allow directory override for extension build (duckdb/duckdb#19110)
krlmlr
added a commit
to krlmlr/duckdb-r-old
that referenced
this pull request
Nov 1, 2025
[C-API] Initial support for accessing file system (duckdb/duckdb#19086) Skip serialization of row_start in DataPointer when targeting latest storage (duckdb/duckdb#19111) Allow directory override for extension build (duckdb/duckdb#19110)
krlmlr
added a commit
to krlmlr/duckdb-r-old
that referenced
this pull request
Nov 2, 2025
[C-API] Initial support for accessing file system (duckdb/duckdb#19086) Skip serialization of row_start in DataPointer when targeting latest storage (duckdb/duckdb#19111) Allow directory override for extension build (duckdb/duckdb#19110)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds initial support for accessing DuckDB's filesystem layer from the C-API.
With the current state of the C-API this is primarily useful when writing table functions that scan some external file format, as it allows you to make use of DuckDBs existing cross platform filesystem, as well as any loaded extension file systems (like httpfs), without having to reimplement the equivalent functionality yourself.
I've tried to keep this initial version of the interface as small and simple as possible, while also leaving room to expand it in the future. The most complex part is the error handling, but I think I've solved it relatively neatly by having both the
duckdb_file_systemandduckdb_file_handlestore their last occurred error internally, which can then be converted toduckdb_error_datathrough a pair of separate functions. Other than that the read/write functions follow posix semantics, returning the number of bytes read/written, 0 on EOF and a negative value on error. In practice the error value is always-1, but I've left it unspecified so that we can potentially return specific error codes in the future (like something like EAGAIN, when we get asyncio), without forcing the user to allocate, inspect and destroy aduckdb_error_dataobject to handle "common" IO errors.