Throw when trying to consume over 128 byte decimals #10601
Conversation
src/function/table/arrow.cpp
Outdated
| // Find the positions of the commas | ||
| size_t pos1 = parameters.find(','); // Position of the first comma | ||
| size_t pos2 = parameters.find(',', pos1 + 1); // Position of the second comma | ||
|
|
There was a problem hiding this comment.
D_ASSERT(pos1 != std::string::npos)
|
@Mytherin I think the errors are related to secrets, maybe we can restart the CI? |
src/function/table/arrow.cpp
Outdated
| } else if (format == "g") { | ||
| return make_uniq<ArrowType>(LogicalType::DOUBLE); | ||
| } else if (format[0] == 'd') { //! this can be either decimal128 or decimal 256 (e.g., d:38,0) | ||
| std::string parameters = format.substr(format.find(':')); |
There was a problem hiding this comment.
These format strings come to us from an external source, so I think it would make sense to be defensive here. We never know if some library could produce just d as a schema. Could we check that format.find(':') and parameters.find(',') return valid positions and throw if they do not?
There was a problem hiding this comment.
Perhaps using StringUtil::Split is easier to use in a defensive way since then you don't need to do all this arithmetic on string positions which is somewhat error prone
Restarting won't fix the CI since these issues are present in the |
|
Thanks! |
Merge pull request duckdb/duckdb#10601 from pdet/throw_256 Merge pull request duckdb/duckdb#10768 from Tishj/pandas_analyze_overflow
Fix: #9201