Repository navigation
IDLE's deficiency in the completion of file names (Python 32, Windows XP) #59142
Description
| BPO | 14937 |
|---|---|
| Nosy | @loewis, @terryjreedy, @serwy |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2013-09-09.22:18:34.264>
created_at = <Date 2012-05-28.17:17:24.670>
labels = ['expert-IDLE', 'type-bug']
title = "IDLE's deficiency in the completion of file names (Python 32, Windows XP)"
updated_at = <Date 2013-09-09.22:18:34.263>
user = 'https://bugs.python.org/fgracia'bugs.python.org fields:
activity = <Date 2013-09-09.22:18:34.263>
actor = 'terry.reedy'
assignee = 'none'
closed = True
closed_date = <Date 2013-09-09.22:18:34.264>
closer = 'terry.reedy'
components = ['IDLE']
creation = <Date 2012-05-28.17:17:24.670>
creator = 'fgracia'
dependencies = []
files = ['25751', '25802', '25944']
hgrepos = []
issue_num = 14937
keywords = ['patch']
message_count = 21.0
messages = ['161795', '161796', '161808', '161812', '162167', '162168', '162169', '162171', '162172', '162175', '162196', '162203', '162204', '162692', '162789', '162804', '172699', '172721', '196517', '196599', '197409']
nosy_count = 6.0
nosy_names = ['loewis', 'terry.reedy', 'roger.serwy', 'westley.martinez', 'python-dev', 'fgracia']
pr_nums = []
priority = 'high'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue14937'
versions = ['Python 3.3', 'Python 3.4']
Activity
fgracia commentedon May 28, 2012
I find specially nice the completion feature for filenames of IDLE when they include long paths, something that is more mand more frequent with big disks and infinite amounts of files.
But there is a small problem with it. If the name of any of the directories that intervene in the path to the file has any special character in it (meaning by that I suppose any non ASCII character), the completion mechanism ceases to work.
For instance, if the path were:
the cesation of the help would occur after the incorporation of *Técnica*, because its *e* is accented.
There is plenty of cheap software which shows this minor deficiency, but I consider that IDLE for Python 3 should not be in that group.
loewis commentedon May 28, 2012
Would you like to contribute a patch?
fgracia commentedon May 28, 2012
I would be delighted, but unfortunately I am a very poor programmer and do not have the slightest idea of how all this works.
serwy commentedon May 28, 2012
The "open_completions" method in AutoComplete.py is where the bug exists. When mode == COMPLETE_FILES, the code searches for characters within the ASCII set, plus a few others contained in FILENAME_CHARS.
Attached is a patch to also include characters beyond the ASCII set during the backward search. If anyone knows of a better technique, please comment.
Fransisco, thanks for reporting this issue. If you find more, please let us know.
westleymartinez commentedon Jun 2, 2012
I think a better technique would be to expand FILENAME_CHARS to include more characters.
westleymartinez commentedon Jun 2, 2012
Also, shouldn't the space character ' ' be included?
westleymartinez commentedon Jun 2, 2012
Ahh okay, sorry for the triple post, I have an idea. On UNIX, the function should accept any character except: \0 /, and on Windows should accept any character except: \0 \ / : * ? " < > | On classic Macintosh, : is invalid. However, I do not know what characters are valid on other systems. (I believe Mac OS X is the same as UNIX, but am not sure.)
Included is a patch with FILENAME_CHARS left in in case anything still needs it.
serwy commentedon Jun 2, 2012
I agree that chr(32) should be included in FILENAME_CHARS.
The algorithm for backward searching checks that each character is contained in FILENAME_CHARS. I'm concerned about running time, as expanding FILENAME_CHARS to include all valid Unicode would be a large set to search. Do you know a better algorithm for determining when to perform a file-system completion when in a string?
Also, are there any Unicode code points above 127 that are not valid in a filename? I appreciate that different file systems can have different constraints but I don't know all these subtle points.
serwy commentedon Jun 2, 2012
Westley, I was responding to msg162168 and didn't see msg162169 yet.
PEP-11 mentions MacOS 9 support was removed in 2.4. Is ":" still invalid in OSX?
I'll need to think about the approach of using an "INVALID_CHARS" list. It looks like it might be a better way to do this. However, here are some issues with the patch.
I received this error:
I fixed this minor mistype, but noticed that file auto-completion no longer works for the following:
Pressing tab after the forward slash should show a drop-down with the root filesystem. Changing INVALID_CHARS to be:
allows the root filesystem to show.
fgracia commentedon Jun 2, 2012
Is there any necessity at all for the IDLE to test the validity of the filenames?
I mean: the file specification is provided by the underlying operating system, so by definition it has to be taken as valid. Testing for its validity is superfluous and, in my opinion, a fault in the general conception of the procedure. IDLE's only role should be to present the data it receives in the best possible form and silently integrate the election made by the user.
Excuse me if I speak nonsense, but I got this idea from studying *Autocomplete.py* and its dependencies. It seems to me that the difficulty resides in submitting natural language strings to the judgement of *PyParse* and *Hyperparser*, which are not competent for it because its purpose is to check the validity of Python code and Python *identifiers*. This confussion is what in my opinion causes this bug (and probably others related to the treatment of natural language text files by the IDLE, as it could also happen with bpo-14929).
westleymartinez commentedon Jun 3, 2012
You're right. The code shouldn't *have* to check if the name is valid. It should just accept that the name is already valid. This would simplify things.
Here's the problem: the code needs to find the index of where the string with the filename starts. The way the code does it now by checking for a quote in a rather obfuscated way (while i and curline[i-1] in FILENAME_CHARS + SEPS:). So, changing that line to say curline[i-1] != "'" or curline[i-1] != '"' would work (in theory) but I'm really hoping there's a better way.
python-dev commentedon Jun 3, 2012
New changeset 41e85ac2ccef by Martin v. Löwis in branch '3.2':
Issue bpo-14937: Perform auto-completion of filenames in strings even for non-ASCII filenames.
http://hg.python.org/cpython/rev/41e85ac2ccef
New changeset 9aa8af0761ef by Martin v. Löwis in branch 'default':
Merge 3.2: issue bpo-14937.
http://hg.python.org/cpython/rev/9aa8af0761ef
loewis commentedon Jun 3, 2012
I have now fixed it by looking for the beginning of the string, and not checking for file name characters at all. There was a related issue that the auto-complete window would disappear if you type a non-ascii character; I have fixed that as well.
serwy commentedon Jun 12, 2012
There's a small logic bug with Martin's patch. Opening up a new editor window and pressing Tab causes a traceback. Attached is a new patch to fix the problem.
python-dev commentedon Jun 14, 2012
New changeset 62030ebb2b01 by Martin v. Löwis in branch '3.2':
Issue bpo-14937: Fix typo. Patch by Roger Serwy.
http://hg.python.org/cpython/rev/62030ebb2b01
loewis commentedon Jun 14, 2012
Thanks, fixed
terryjreedy commentedon Oct 11, 2012
This patch (I suspect it is this one) disabled the use of '/' in filenames on windows when using filename completion.
'c:\ <wait, tab, ^space bring up box> in 3.2.3 and 3.3.0>
(If there is no 'r' prefix, it really should require '\\' to be safe.)
"c:/ ditto for 3.2.3.
In 3.3.0, (and, I presume, 3.2.4 if not changed) tab inserts spaces to next tab stop, while ^space is ignored.
Since using '/' is recommended over unsafe '\', this is an undesireable regression.
fgracia commentedon Oct 12, 2012
In my machine with Windows XP and Python 3.3.0 both variants work, the only
difference being that
'c:/
brings up the selection box authomatically and
'c:\
requieres that it be summoned with the tab key, as indicated.
2012/10/12 Terry J. Reedy <report@bugs.python.org>
westleymartinez commentedon Aug 30, 2013
On Thu, Oct 11, 2012 at 3:06 PM, Terry J. Reedy <report@bugs.python.org> wrote:
+1 for requiring \\.
I'll test this tomorrow and report back the behaviour. On Linux it
seems that the window only pops up when you press tab. I think
behaviour should be as identical on all platforms as possible.
westleymartinez commentedon Aug 30, 2013
On 3.4.0a1 on Windows it seems to come up automatically with \ or \\. A single \ will only pop up with the tab key; I think it's good that way. Special characters (i.e. æ) work normally. All this said, I think this issue is fixed.
terryjreedy commentedon Sep 9, 2013
I am not currently seeing the problem I reported, so closing. If I type past / and hit ^space, I sometimes get an empty listbox, but I do not yet have a consistent failing case. Will open a new issue if/when I get one.