Skip to content

Fix Windows encoding problems#2146

Open
Xanders wants to merge 3 commits into
pry:masterfrom
Xanders:fix/windows_encoding
Open

Fix Windows encoding problems#2146
Xanders wants to merge 3 commits into
pry:masterfrom
Xanders:fix/windows_encoding

Conversation

@Xanders

@Xanders Xanders commented Jul 24, 2020

Copy link
Copy Markdown

Hi! Thanks for the greatest project!! I just realized there are some bugs with filesystem encoding on Windows. Dir.pwd gives me string in Windows-1251 encoding on my machine, so Pry starting process is broken with incompatibility exception. Also whereami command is broken too. So I think it's safe to perform encode('UTF-8') on some strings from filesystem to ensure they are compatible with other. Everything works as expected on my machine after fixes.

I'm using Pry 0.13.1 on Ruby 2.7.0 on Windows 10.

@kyrylo kyrylo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you post the full exception(s)? FWIW we don't really test on Windows because we lack manpower. This solution doesn't look too clean to me.

@Xanders

Xanders commented Nov 28, 2020

Copy link
Copy Markdown
Author

@kyrylo, thank you for reply! There is full exception for me:

Encoding::CompatibilityError: incompatible character encodings: UTF-8 and Windows-1251
from C:/tools/ruby27/lib/ruby/gems/2.7.0/gems/pry-0.13.1/lib/pry/code/code_file.rb:102:in `expand_path'

And:

Encoding::CompatibilityError: incompatible character encodings: IBM866 and UTF-8
from C:/tools/ruby27/lib/ruby/gems/2.7.0/gems/pry-0.13.1/lib/pry/commands/show_info.rb:112:in `+'

Steps to reproduce:

  1. Start Windows Sandbox, install Chocolatey and run choco install ruby, gem install pry
  2. Create folder and Ruby-file with non-ASCII symbols in path and content:
def тест
end

require 'pry'
Pry.start
  1. Run the file and hit show-method тест

The root of the problem is Windows default filesystem, please see below:

C:\Data\Тест>docker run --rm ruby ruby -e "p Dir.pwd.encoding"
#<Encoding:UTF-8>

C:\Data\Тест>ruby -e "p Dir.pwd.encoding"
#<Encoding:Windows-1251>

C:\Data\Тест>docker run --rm ruby ruby -e "p Dir['*'].first.encoding"
#<Encoding:UTF-8>

C:\Data\Тест>ruby -e "p Dir['*'].first.encoding"
#<Encoding:IBM866>

In Ruby we cannot mix non-ASCII symbols with different encodings. So on Windows we should convert non-UTF-8 strings to UTF-8 before first use. This does not affect strings that are already in UTF-8.

I just realized there was several places in code where converting is required too and pushed a new commit. And also another one to fix Rubocop warning.

@Xanders Xanders requested a review from kyrylo January 15, 2021 21:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants