Add extra element to offsets - read path - part1 (Arrow compatibility) - #1932
Merged
Merged
Conversation
Shelnutt2
approved these changes
Nov 24, 2020
joe-maley
reviewed
Nov 24, 2020
ypatia
force-pushed
the
yt/3890_add_offset_extra_element_arrow
branch
2 times, most recently
from
November 27, 2020 13:33
5448da9 to
a23b3b4
Compare
ypatia
force-pushed
the
yt/3890_add_offset_extra_element_arrow
branch
from
November 27, 2020 18:14
a23b3b4 to
315176c
Compare
joe-maley
reviewed
Nov 29, 2020
joe-maley
left a comment
Contributor
There was a problem hiding this comment.
Looks great! I have a few style-related comments below, but it looks functionally correct to me.
Member
Author
|
The openAPI specification part of the serialization changes can be found here: TileDB-Inc/TileDB-Cloud-API-Spec#217 |
Contributor
|
The new serialization changes LGTM. Let's hope this fixes the CI issue. Thanks! |
This was referenced Dec 22, 2020
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is part of a series of tickets that are going to address compatibility with Arrow in the area of offsets.
We have already implemented support for element offsets (instead of the default byte offsets).
This PR is addressing the first part (Read path only - simple version) of supporting an extra element in the offset array that refers to the size of the returned data. In other words the end of each successful read (INCOMPLETE or COMPLETED), we are appending an extra offset that points to the end of the buffer.
However in this first part if the user's buffer has no space for the extra element we are just erroring-out the read, instead of handing it gracefully. Proper handing of the memory issue will be part of another ticket.