Repository navigation
[css-text-3] treat all-neutral lines same as empty ones for plaintext alignment #4405
Description
[note added 9 Oct @15:35 by @r12a
For an overview of the thread leading up to this see here]
From Aharon Lanin https://lists.w3.org/Archives/Public/public-i18n-bidi/2016JulSep/0028.html
I do not think that there is a problem with the directionality algorithm. Aside from alignment, it makes no difference whether that isolate paragraph is displayed as LTR overall or RTL overall. And for much more important and common cases of all-neutral text - phone numbers and, to a smaller extent, dates, times, and signed numbers - its being LTR by default is extremely important.
I do think that there is a problem with the definition of start alignment, which is not controlled by Unicode, but by various other specs, such as CSS. The problem affects the alignment of all-neutral paragraphs generally, like the examples cited above, not just the corner case of an isolate. Start alignment is generally defined as "left" for an LTR paragraph, and "right" for an RTL paragraph, and this means that an all-neutral paragraph, which is LTR by the UBA, is left-aligned - even if the alignment outside that paragraph is right. This makes the all-neutral paragraph needlessly different from its surroundings and usually looking pretty bad.
The solution is to make start alignment match the alignment outside the paragraph if the paragraph's directionality is determined from its content and its content is all-neutral.
In the context of CSS, the relevant spec is that of the start and end edges of a line box whose containing block has ‘unicode-bidi: plaintext’ (https://www.w3.org/TR/css-text-3/#bidi-linebox). It already makes an exception for an empty line box, which "takes its inline base direction from the preceding line box (if any), or, if this is the first line box in the containing block, then from the ‘direction’ property of the containing block." I think that this exception should be broadened to an all-neutral line box.
Activity
css-meeting-bot commentedon Jan 24, 2020
The CSS Working Group just discussed
all-neutral lines.The full IRC log of that discussion
<TabAtkins> Topic: all-neutral lines<astearns> github: https://github.com//issues/4405
<TabAtkins> fantasai: Issue raised by Aharon
<TabAtkins> fantasai: When you have a plaintext block, like a <pre> with unicode-bidi:plaintext
<TabAtkins> fantasai: It resolves the para base direction of each "paragraph" (things separated by forced line breaks) according ot the first strong character in the paragraph.
<TabAtkins> fantasai: This is a heuristic, not always accurate, but it's what the plaintext bidi algo does.
<TabAtkins> fantasai: Interesting to Aharon is alignment
<TabAtkins> fantasai: We take advantage of a para-based direction, that's not 'direction' proper, but we use that direction per-line to decide what "start" text-alignment means.
<TabAtkins> fantasai: Problem arises if you ahve a paragraph that's completely neutral chars, like punctuation.
<TabAtkins> fantasai: Or numbers.
<TabAtkins> fantasai: What he noticed is that you have a document that's entirely arabic, but it has a line of text with no letters in it, it ends up being aligned to the left (wrong side for arabic)
<TabAtkins> fantasai: We already make an exception for empty line boxes; we take the direction from preceding line. Suggestion is to apply that exception to all-neutral lines too.
<TabAtkins> florian: Currently it's udnefined?
<TabAtkins> fantasai: Currently the defualt is ltr if you can't find a direction.
<TabAtkins> jfkthame: I think the terminology is a little wrong; numbers are weak, not neutral.
<TabAtkins> fantasai: Sure. I don't recall the exact spec wording, it's probably more accurate.
<TabAtkins> fantasai: So if you have arabic text and underlined it with a line of hyphens, for example
<TabAtkins> s/fantasai/faceless/
<TabAtkins> fantasai: Yeah, that's a case here
<TabAtkins> koji: So I guess options are previous paragraph, or 'direction' property. Any reason you chose preivous para?
<TabAtkins> fantasai: That was the suggestion from Aharon.
<TabAtkins> fantasai: We're specifically in a unicode-bidi:plaintext context, which imlpies the 'direction' doesn't know what it should be.
<TabAtkins> koji: But then 'direction' should specify the ui or document direction, so it's a more natural fallback to me.
<TabAtkins> florian: Not necessarily - if writing content like an email, the UI might be in English but the context is Arabic. Using 'unicode-bidi:plaintext' on the email textarea because you know the lang direction is unknown.
<TabAtkins> koji: Agree, this is heuristic, we could argue both cases.
<TabAtkins> koji: I just think author intention is more natural fallback.
<TabAtkins> myles: Not sure I agree with that.
<TabAtkins> myles: If there's a quote snippet, you might want to remove some middle paragraphs, replacing with [...] to indicate it's been snipped
<TabAtkins> myles: If you do that as currently described, it'll be on the other side of the screen and look wrong.
<TabAtkins> koji: If you have English quote in arabic text, and there's underline, you want it on arabic direction
<TabAtkins> koji: If you ahve hyphens *above*, it'll be arabic-aligned; if you ahve hyphens *below*, it'll be English-aligned.
<TabAtkins> koji: So it's a question of which is better heuristic.
<TabAtkins> fantasai: I think you're both making good points.
<TabAtkins> fantasai: It's not clear which is clearly better.
<TabAtkins> fantasai: I think what is clear is that we shouldn't use ltr as the generic default.
<TabAtkins> koji: Have you tested what browsers do today?
<TabAtkins> faceless: My understanding of "plaintext" is that you really can't rely on the direction property.
<fantasai> testcase - http://software.hixie.ch/utilities/js/live-dom-viewer/?%3C!DOCTYPE%20html%3E%0A%3Cpre%20dir%3Dauto%20style%3D%22direction%3A%20rtl%22%3E%0A%5E__%24%0A%D9%84%D8%B9%D8%B1%D8%A8%D9%8A%D8%A9%0A%5E__%24%0Aabcde%0A%5E__%24%0A%3C%2Fpre%3E%0A
<TabAtkins> TabAtkins: My intution says that most of the time you'll be looking at single-language text, so the neutral para should align with that, and the heuristic will work. When you ahve mixed text, Koji's argument that it's uncleaer and might be wrong is valid, but the "previous para" heuristic should be right about half the time. So that's not bad!
<TabAtkins> fantasai: Testcase shows that each paragraph is handled individually; if it's all neutral, it's treated as ltr even if "direction:rtl" (this is exacty what the spec currently specifies).
<TabAtkins> fantasai: Interesting case - neutral text like ASCII art in a "plaintext" context, and there's some rtl preceding it, you don't want it swapped direction.
<TabAtkins> myles: ASCII art is pretty niche. I don't think it should affect this decision.
<TabAtkins> fantasai: So I think if the *entire paragraph* is neutral, we shouldn't look at previous para; we should just use the "previous line".
<TabAtkins> koji: What do you do at the beginning of the block?
<TabAtkins> fantasai: Probably just default to what unicode says, which is ltr
<TabAtkins> myles: I'm sure we're not the first ot have this idea. Do we know why Unicode gives this advice?
<TabAtkins> fantasai: Unicode doesn't define alignment.
<TabAtkins> fantasai: They do require that the *base diretion* is ltr (affecting bidi reordering); we don't want to touch that. This is just about alignment.
<TabAtkins> koji: Unicode bidi l1 requires applying reordering to trailing spaces; without bidi l1 and l0 affects reordering.
<florian> q?
<florian> q+
<TabAtkins> fantasai: We can tweak the alignment; I don't think we should be making tweaks to the ordering of characters.
<astearns> Zakim, open queue
<Zakim> ok, astearns, the speaker queue is open
<florian> q+
<TabAtkins> koji: I'd rather fix paragraph level rather than fix text-align
<TabAtkins> fantasai: So that would be to close this issue no-change?
<TabAtkins> fantasai: The issue is just about text-alignment; saying it's wrong for these cases.
<TabAtkins> fantasai: You're saying let's not do that. Taht's an okay proposal!
<TabAtkins> koji: No, just say that text-align uses base direction, and CSS can just say that if there's no strong characters we inherit from the previous line.
<TabAtkins> fantasai: bidi reordering relies on base direction, that'll change the ordering of characters.
<TabAtkins> fantasai: Like base direction determines where your period goes at the end of your sentence.
<TabAtkins> jfkthame: Not sure I'm totally comfortable with changing base direction for reordering purposes and for alignment.
<TabAtkins> fantasai: I'm just relaying fro mthe issue
<TabAtkins> fantasai: We can decide to reject the proposal.
<astearns> ack florian
<TabAtkins> florian: Originally I was enthusiastic, and maybe given it's from Aharon my concerns are unfounded; in Unicode there of course is no alignment, it comes from *somewhere*. Currently in plaintext applications everything is effectively start-aligned.
<TabAtkins> florian: So aren't those apps already effectively doing the same for alignment and base direction?
<TabAtkins> koji: The feedback is from 2016, mind checking with Aharon again?
<TabAtkins> koji: He's been responsive to me more recently.
<TabAtkins> fantasai: Ok, I'll summarize and take it back to himl
<TabAtkins> fantasai: I'm opening some plaintext editors, and it looks like alignment is based on the whole file.
<TabAtkins> fantasai: Even having differing-direction entire paragraphs doesn't affect it.
fantasai commentedon Apr 3, 2020
Formal minutes of the 24 January 2020 discussion: https://lists.w3.org/Archives/Public/www-style/2020Feb/0019.html
fantasai commentedon Apr 3, 2020
Replied to the original post on public-i18n-bidi here: https://lists.w3.org/Archives/Public/public-i18n-bidi/2020AprJun/0000.html
[edited by @r12a on 9 Oct: including text of email]
The CSSWG tracked your issue here at
#4405
and it was discussed at the last F2F, minutes here:
https://lists.w3.org/Archives/Public/www-style/2020Feb/0019.html
With regards to concerns with your last suggestion [apply resolved
directionality of the previous line box to an all-neutral line box], one
concern was inconsistency in handling all-neutral content before and after a
strong paragraph (particularly at the start of the plaintext block); and
another concern was the divergence of the base directionality wrt bidi
reordering vs the base directionality for alignment.
I was asked to reach out to you to ask if this is still something you want
changed, and if so, if you would mind reading through the concerns and explain
what you think is the best way forward and why.
(And of course, if anyone else on this thread has feedback on the issue, CSSWG
would be interested to hear that, too.)
amire80 commentedon Apr 4, 2020
fantasai commentedon Apr 6, 2020
amire80 commentedon Apr 6, 2020
faceless2 commentedon Apr 6, 2020
fantasai commentedon Jul 5, 2020
Aharon's reply: https://lists.w3.org/Archives/Public/public-i18n-bidi/2020JulSep/0000.html
[edited by @r12a on 9 Oct: including text of email below]
Let me handle this one first.
Such divergence is not optimal, but it is actually quite common, being the
least of all evils. I am referring to setting text-align to left (not
start) in an LTR page (and right in an RTL page) on a field whose direction
is opposite to that of the page, in order to get the overall layout of the
page to work well. This is usually the best thing to do for fields
whose content is significantly shorter than the field's width. It is
definitely the wrong thing to do for fields whose content is longer than
the field's width and is allowed to wrap, since wrapped content is
definitely less readable when it is aligned to its end side. Ah, so what
does one do when one doesn't know whether the field's content will be short
or wrapped? There is a neat trick: put the content in an element with
direction set to the content's directionality, text-align:start, and
layout:inline-block. Then wrap that element in another whose text-align
matches that of the page and whose width is as desired. The outcome is that
short content appears aligned to the page's start (and thus to its own
end), while wrapped content appears aligned to its own start, and everyone
is happy.
Unfortunately, this trick does not work for a field that contains multiple
paragraphs of content (wrapped when longer than the field's width) whose
directionality may not match that of the page or even one another. The
canonical solution for such fields is unicode-bidi:plaintext, and it is for
these fields that I made my suggestion. In such fields, one generally wants
RTL paragraphs to be aligned right, and LTR paragraphs to be aligned left,
since each may be wrapped. However, it is better for the all-neutral LTR
paragraphs to match the alignment of their surroundings. Yes, that means
that their alignment may not match their directionality, but as I explained
above, this is not unusual and is probably the least of all evils.
It is easier to talk about specific examples. Koji gave one in a private
communication:
[start of example]
Then the first stand-alone ">" goes one way, while the second stand-alone
">" goes the other way.
[end of example]
My take on this example is that I don't have a big problem with that
outcome, given that the > on the line with the Arabic on it will be < and
right-aligned too. In fact, I am not even sure that the current way comes
out better. Generally speaking, nothing is going to be perfect, but I think
that my proposal will fix more problems than it creates.
We can tweak the proposal to look at the paragraph(s) after the all-neutral
one if there are no non-neutral paragraphs before it (or consider other
tweaks or even entirely different proposals, if there are any).
7 remaining items
[css-text-3] plaintext line boxes without a strong direction use dire…
[css-text-3] Fix earlier neutral plaintext line wording: bidi switche…
Test for paragraph directionality with plaintext alignment
Test for paragraph directionality with plaintext alignment
Bug 1808000 [wpt PR 37704] - Test for paragraph directionality with p…
Bug 1808000 [wpt PR 37704] - Test for paragraph directionality with p…
Bug 1808000 [wpt PR 37704] - Test for paragraph directionality with p…