added netloc support for hdfs URIs#168
Conversation
…DFS URIs which was against the URI specification
|
@vvaten Thank you for your pull request. It looks good to me. @menshikh-iv Might be a good idea to merge this after our HDFS integration tests are up. What do you think? |
| * file:///home/user/file | ||
| * file:///home/user/file.bz2 | ||
|
|
||
| NOTE: hdfs://path/file does no longer work as it is against the URI |
There was a problem hiding this comment.
Can you add more information to comment (when this happens, what HDFS version affected, etc)?
There was a problem hiding this comment.
This is a generic change that affects all HDFS versions. They do support hdfs://host/path/file URI format. Using hdfs://path/file in smart_open violates this and also violates the URI specification (RFC3986) where the hostname part is always after the '://'. The correct way to refer to local content is hdfs:///path/file instead of hdfs://path/file.
|
Hello is there any way this can be merged ? As part of my work I was trying to read and write to hdfs and traced the bug to this |
|
Hi @vvaten 👋 I know it's been a while. It looks like this PR is still valid, only outdated. Do you want to pull |
removed the special handling of HDFS URIs which was against the URI specification
fixes #162