Default line break handling is inconvenient

It is bad because quoting you:

The browser forcefully wraps at hardwrapped places via the insertion of <br> tags when it might be too short or too long for the viewport, due to the varieties in ways in which the text can be displayed over various styling, systems and devices. Therefore the wrapping should be left to the browser to handle it.

Guys, there is no “right” answer on this one. However it is abundantly clear that original intent of Markdown was to wrap lines ala HTML, e.g.

this text
is written on two lines

(renders as)
this text is written on two lines

and CommonMark is designed to be as true as possible to the original philosophy of Markdown, only making necessary decisions in the many, many places where the spec was ambiguous.

As noted by @riking earlier:

It is an explicit goal to support a common “switch” that allows linebreaks to be treated as <br> since that is what users tend to expect.

That’s how it works
here too. :smile:

The compromise we have in is that you can indicate linebreak with a backslash character now too e.g.

this text\
is written on two lines

(renders as)
this text
is written on two lines

and at least you can now see why, rather than having to somehow magically infer that there are two invisible spaces at the end of the line…

4 Likes

Exclude from what? It’s a bit like saying you’re excluded from using QWERTY keyboards. It’s not “excluding” to say that a tool is designed for a certain task. It’s just a practical realization that a single tool can’t work great for whatever task. That’s why we have non-QWERTY keyboards, and that’s also why there should probably be a different flavour of Markdown for other kinds of writing systems.

No, you’re deliberately misconstruing what I said. It’s like saying that I’m “excluding” towards screws when I’ve designed a nail gun to efficiently nail stuff to the wall.

You’re mistaken. While Github uses the hars linebreaks for comments, it’s not used for Markdown files and the like.

1 Like

If you feel offended, my apologies, that was not my intention. Your original comments comes across that you were trying to say that certain users should not use markdown. That would contradict the idea to create a standard or “common” mark spec.

I would like conclude my contribution to the discussion with one last remark. After that, I have said all I have to say.

Referring to this wikipedia article

The Unicode standard defines a number of characters that conforming applications should recognize as line terminators:[3]

LF: Line Feed, U+000A
VT: Vertical Tab, U+000B
FF: Form Feed, U+000C
CR: Carriage Return, U+000D
CR+LF: CR (U+000D) followed by LF (U+000A)
NEL: Next Line, U+0085
LS: Line Separator, U+2028
PS: Paragraph Separator, U+2029

This may seem overly complicated compared to an approach such as converting all line terminators to a single character, for example LF. However, Unicode was designed to preserve all information when converting a text file from any existing encoding to Unicode and back. Therefore, Unicode should contain characters included in existing encodings. NEL is included in EBCDIC with code (0x15). NEL is also a control character in the C1 control set.[4] As such, it is defined by ECMA 48,[5] and recognized by encodings compliant with ISO-2022 (which is equivalent to ECMA 35).[6] C1 control set is also compatible with ISO-8859-1 .[citation needed] The approach taken in the Unicode standard allows round-trip transformation to be information-preserving while still enabling applications to recognize all possible types of line terminators.

As we can see here, the way line breaks are handled is already quite complex. It is my belief that adding a backslash or two spaces at the end of a line will not make life easier for future generations.

I think you’re missing the point here. For the purpose markdown was designed for – that is, writing prose in English or similar languages, users do not want or need hard line breaks.

So if you have large groups of people on your site that write with asian scripts or poety, or some other case where hard breaks will suit your needs better, you will want to use Github Flavoured Markdown or any markdown implementation that implements the aforementioned flag to enable hard breaks.

Or not use markdown at all. No one’s forcing you to, after all.

That is fine.

What, however, would not be fine, would be to change an already established standard in a backwards-incompatible way, requiring manual reformatting of millions of already existing markdown documents.

1 Like

Reading all the examples posted it seems there could another approach to this - require the line to be of certain length for the newline in source to be removed. For example, if the newline character is before position 40 or 60, it is converted to HTML line break, otherwise it is not. This would actually handle every case mentioned above correctly.

I like your compromise proposition, I’m just afraid that as a basic user, that distinction would be too “obscure” for me to know it and to craft my text accordingly.

Note that this was discussed for pandoc

Heated topic, this. Still, I feel that a few useful things have not been said yet, so I’ll see if I can get them across properly.

First off, terms:

  • I’ll use hardwrapping and hardwrapped files to refer to files that use linebreaks in the middle of paragraphs. These linebreaks are placed to make the plaintext / markdown look good, they have no semantic meaning (it’s just one paragraph).
  • I’ll use softwrapping to refer to using linebreaks in the markdown source only where they are semantically relevent as linebreaks or paragraph separations. In particular, this means that paragraphs end up as a single long line in the markdown source.
  • I’ll refer to rendering a linebreak soft as meaning that a linebreak in the markdown does not produce a linebreak in the rendered HTML, and rendering hard means it does result in a linebreak. I think this is the same as what the spec uses.

Then, my own position: I’m a vim user that really wants to hardwrap all the things. I certainly think that rendering linebreaks soft should be the default, as it does not prevent people from using softwrapping. I do think some extra markup could make all of our lives easier. On to the details.

About hardwrapping. People have mentioned that “if the user pressed enter, that should produce a line break in the rendered output”. However, some editors (most notably vim) take care of this automatically - once you go over 72 characters (by default), it inserts a linebreak. It also has some commands to efficiently “reflow” a paragraph when needed. When editing plaintext files, this is convenient.

The result is a hardwrapped file, which:

  • Looks exactly like the author intended, even on dumb editors
  • Is convient to use with diff software
  • Easier to work with in hard-wrapping editors

The opossite, softwrapping, or using a single line for each paragraph, is:

  • Has configurable line width, regardless of what the author liked
  • Easier to work with in editors which do not support handwrapping
  • Markdown source doesn’t look so good without Markdown-aware wrapping

One interesting observation here is that the latter, using a single line for each paragraph, is rendered the same way, regardless of wether linebreaks are rendered hard or soft. So for regular paragraphs, which would be the brunt of markdown text, rendering linebreaks hard is more flexible (it supports either approach). The difference is in other context, such as addresses, where the linebreaks in the markdown do have semantic meaning (in particular, where linebreaks are needed, but there are not multiple paragraphs involved).

Above, I said softwrapping doesn’t look so good without Markdown-aware wrapping in the editor. This is probably best illustrated by an example. For ease of illustration, I’ll assume a very small (let’s say 40-character wide) editor screen, but everything I say applies to any width.

First off, a typical short paragraph. When it’s hardwrapped at 40-chars, the markdown source looks like:

Lorem ipsum dolor sit amet, consectetur
adipisicing elit, sed do eiusmod tempor
incididunt ut labore et dolore magna
aliqua.

When softwrapped, the source is just one line, but when viewed in our 40-character editor, it also looks exactly the same:

Lorem ipsum dolor sit amet, consectetur
adipisicing elit, sed do eiusmod tempor
incididunt ut labore et dolore magna
aliqua.

So, no difference there (actually, bonus points for softwrapped - if I resize the editor, the wrapping changes).

Now, suppose I add a list and some quoting to the equation. This is a single list item, containing a quote and some normal text, using hardwrapping:

 - > Lorem ipsum dolor sit amet,
   > consectetur adipisicing elit, sed
   > do eiusmod tempor incididunt ut
   > labore et dolore magna aliqua.

   Ut enim ad minim veniam, quis nostrud
   exercitation ullamco laboris nisi ut
   aliquip ex ea commodo consequat.

Now, if I use softwrapping, these end up as 2 long lines, with an empty line in between. If I let a regular editor apply linewrapping to that, it will look like this:

 - > Lorem ipsum dolor sit amet,
consectetur adipisicing elit, sed do
eiusmod tempor incididunt ut labore et
dolore magna aliqua.

   Ut enim ad minim veniam, quis nostrud
exercitation ullamco laboris nisi ut
aliquip ex ea commodo consequat.

Which, IMHO, is not quite readable, especially compared to the hardwrapped version.

Of course a smart, markdown-aware editor could certainly improve this, by adding indents and other prefixes to the wrapped lines based on the context.

But I think this example actually shows that Markdown lends itself well for hardwrapping the source code (just like email).

Now, people argued that if linebreaks do appear in the source document, they’re probably intended as such and should be present in the output. For example, comment #35 argues that a hard-wrapped paragraph will still look proper when the linebreaks are rendered hard (e.g. actually become linebreaks in the rendered output). However, there’s a few problems with that:

  • When rendered to HTML, this approach prevents reflowing the text dependent on the output width, line length depends on the source formatting.

    I realize that this is a bit of a double sided blade, since the same argument could be made for softwrapping (and I actually mentioned this as an advantage of softwrapping above)

    However, I think that this argument has more weight for the rendered HTML output, since people expect their line width to fill the available space (or whatever the layout and CSS rules dictate). If the text is rendered really narrow (which is usually the case when the source was hardwrapped at 80 or 120 characters) it just looks wrong.

  • When rendering, typically a proportional font is used. Depending on what letters are used on which lines, this could cause the line endings to look uneven. This is not a problem when the markdown source is viewed, since that typically happens in a non-proportional font.

  • When rendering, parts of the markdown might end up significantly shorter in the rendered output (this is actually just a special case of the previous point, I guess). Best illustrated with an example.

    Consider this (hardwrapped) markdown source:

    [Lorem ipsum](http://www.lipsum.com/)
    dolor sit amet, consectetur adipisicing
    elit, sed do eiusmod tempor incididunt
    ut labore et dolore magna aliqua.
    

    This looks balanced when viewing the markdown source, but when rendered the extra link markup disappears. When linebreaks rendered hard, this ends up as:

    Lorem ipsum
    dolor sit amet, consectetur adipisicing
    elit, sed do eiusmod tempor incididunt
    ut labore et dolore magna aliqua.

    Which isn’t quite what you’d want.

So, that suggest that you want to render linebreaks hard, at least by default. However, even for people like me, that use hardwrapping, sometimes other pieces of text come up, that are not regular, flowing paragraphs. Addresses seem like a perfect example of this. I’d like to have the rendered output look like this:

Mr. Example
Example street 1
12345 Someplace

But none of these markdown syntaxes for this look good:

Mr. Example  
Example street 1  
12345 Someplace  

Mr. Example\
Example street 1\
12345 Someplace\

Mr. Example

Example street 1

12345 Someplace

The first uses double trailing spaces, which look fine, but are not visually distinguishable from regular paragraphs. Trailing backslashes are better in this regard, but look quite ugly. Finally, making each line a separate paragraph looks bad and is semantically incorrect as well.

For and address like this, you really do want to render linebreaks hard (yes, even me).

The current spec says “A renderer may also provide an option to render soft line breaks as hard line breaks.”, which I take to mean that a rendere might have an option, like a commandline or configuration option, to render hardbreaks instead of softbreaks. Even though this seems like a solution, it’s not, due to lack of granularity.

More specifically, this would mean you could turn this option on or off for a single user or perhaps even system. However, a single user might have different needs in different documents and, as shown above, even different needs within a document.

If we could somehow detect between the different types of “paragraphs” (or rather, distinguish between a paragraph and a group of lines) without adding markupt for that this would be great. Comment #41 has a proposal for this, but I simply don’t think we can come up with something that works in enough cases, so the second-best option would be to somehow mark a single paragraph, or a block of markdown, to use hard-rendered linebreaks?

I’m thinking about something along the lines of the ``` syntax that can be prefixed and postfixed to a list of blocks to make linebreaks inside them rendered hard.

This also means that if a document needs all their linebreaks rendered hard, it could be possible to just wrap the entire document in this new syntax.

At first glance, such a syntax would solve all problems adequately (or did I miss anything). The only challenge left would be to come up with some decent markup that can be used, preferably something that gives some idea of what it does when reading it).

Thinking on this a bit more - this is actually a weakened version of the ``` syntax. Where ``` implies:

  • (typically) Proportional font / code block
  • No inline markdown formatting
  • Hard linebreaks

This new syntax would only need to imply the last point. This raises the question: Is there any usecase for other blocks with just one or two of the above points? E.g. a block that is shown in a proportional font, but renders linebreaks soft and supports inline formatting? Or a block that doesn’t support inline formatting, but doesn’t get rendered in a proportional font? Doesn’t sound all that bad, though this might be overengeneering things.

Thoughts? Flames? Comments?

2 Likes

+++ Matthijs_Kooijman [Oct 06 14 22:53 ]:

If we could somehow detect between the different types of “paragraphs”
(or rather, distinguish between a paragraph and a group of lines)
without adding markupt for that this would be great. [4]Comment #41 has
a proposal for this, but I simply don’t think we can come up with
something that works in enough cases, so the second-best option would
be to somehow mark a single paragraph, or a block of markdown, to use
hard-rendered linebreaks?

Here I really like reStructuredText’s line blocks:

 | Mr. Example
 | Example street 1
 | 12345 Someplace

This is very readable and doesn’t look marked up. Note that
line blocks also preserve leading space, so they’re ideal
for poetry as well. Pandoc already supports this extension.

4 Likes

Right, that actually looks good to me as well. One downside of a prefix (as opposed to begin/end markers) is that you can’t easily wrap your entire document in them, but that wouldn’t be a problem for me personally (I’m just thinking some people that posted in this thread might have liked this.

1 Like

Could see this as being handy in terms of styling a printed document. E.g.

 | Mr. Example
 | Example street 1
 | 12345 Someplace
 {.address}

Where the .address style makes it float to the top right of the document.

(Plus if you support YAML autodetection, within those block. You can make for some pretty smart visible metadata )

 | name:     Mr. Example
 | tel.work: (703) 902-6923
 | web:      www.bah.com
 | email:    joseph@exampe.com
 {.details}

I’m kinda amazed at the overwhelming number of people here who either have never heard of hard-wrapped text (which is extremely common in the Linux world), or who think its no big deal to insert a bunch of unintended (and unsemantic) br tags in a hard-wrapped paragraph.

What I still don’t get is why people need so many br tags that they want the default behavior changed. When I write HTML, br is probably one of the least-used tags. Its basically only useful for addresses, song lyrics, poetry, and such. Are people using br to start new paragraphs? Is there some Markdown poetry forum that you are all coming from?

3 Likes

+++ mightymax [Oct 07 14 12:10 ]:

I’m kinda amazed at the overwhelming number of people here who either
have never heard of hard-wrapped text (which is extremely common in the
Linux world), or who think its no big deal to insert a bunch of
unintended (and unsemantic) br tags in a hard-wrapped paragraph.

What I still don’t get is why people need so many br tags that they
want the default behavior changed. When I write HTML, br is probably
one of the least-used tags. Its basically only useful for addresses,
song lyrics, poetry, and such. Are people using br to start new
paragraphs? Is there some Markdown poetry forum that you are all coming
from?

I agree completely. There are very few cases where good typography would call for a hard line break inside a paragraph. That’s why I’m surprised that sites like github (and this forum) want implementations that make all newlines into br tags. The result is usually typographically unappealing. My guess is that the people who are requesting this feature expect a new paragraph to start when they hit newline, but that is something entirely different.

3 Likes

This is configurable, you can enable the hard wrapping in Site Settings: traditional markdown linebreaks , I would recommend you do so here as its probably a better fit for this community.

We find that when you give Markdown to the non technical audience the general expectation is that what you type in the box in the left is somewhat similar to the box in the right. Hard wrapping is an enigma as is the trailing double space.

3 Likes

I’ve seen this and also an attempt to create an unordered list with line breaks. The writer then wondered why the list items didn’t appear on separate lines.

1 Like

For example, this interaction designer

http://talk.commonmark.org/t/lists-without-a-list-indicator-character/861

A solution would be to make line break handling default, selectable by admin. And also to be able to override line break handling via document declaration. Here is a mockup of first few line of document declaration, with a “Line_Break” field:

!CommonMark: 0.1.23-github.username.projectname
 Title:      Title for the top bar of any browser
 Date:       32-4-2002
 Line_Break: On

Even if line break lists were possible they would not create semantic HTML lists. While convenient for the writer, it is not convenient for the web designer (or blind users) who may need that HTML so that the list may be presented differently. For example, the web designer may wish to use a special style of bullet point for all unordered lists.

Requiring lists to be explicitly declared requires the writer to care more about the reader of the document.

Another application where line breaks is needed is lyrics

Take for instance this lyric (Is there a commonmark equiv to [Verse 1:] or is that just standard header):

"Not A Bad Thing"

[Verse 1:]
Said all I want from you is to see you tomorrow
And every tomorrow, maybe you'll let me borrow your heart
And is it too much to ask for every Sunday
And while we're at it, throw in every other day to start

[Pre-Chorus:]
I know people make promises all the time
Then they turn right around and break them
When someone cuts your heart open with a knife, now you're bleeding
But I could be that guy to heal it over time
And I won't stop until you believe it
'Cause baby you're worth it

[Chorus:]
So don't act like it's a bad thing to fall in love with me
'Cause you might look around and find your dreams come true, with me
Spent all your time and your money just to find out that my love was free
So don't act like it's a bad thing to fall in love with me, me
It's not a bad thing to fall in love with me, me

[Verse 2:]
Now how about I'd be the last voice you hear tonight?
And every other night for the rest of the nights that there are
Every morning I just wanna see you staring back at me
'Cause I know that's a good place to start

[Pre-Chorus:]
I know people make promises all the time
Then they turn right around and break them
When someone cuts your heart open with a knife, now you're bleeding
Don't you know that I could be that guy to heal it over time
And I won't stop until you believe it
'Cause baby you're worth it

[Chorus:]
So don't act like it's a bad thing to fall in love with me
'Cause you might look around and find your dreams come true, with me
Spent all your time and your money just to find out that my love was free
So don't act like it's a bad thing to fall in love with me, me
It's not a bad thing to fall in love with me, me
Not such a bad thing to fall in love with me
(Not such a bad thing to fall in love with me)

[Bridge:]
No I won't fill your mind
With broken promises and wasted time
And if you fall, you'll always land right in these arms
These arms of mine

[Chorus:]
Don't act like it's a bad thing to fall in love with me
'Cause you might look around and find your dreams come true, with me
Spent all your time and your money just to find out that my love was free
So don't act like it's a bad thing to fall in love with me, me
It's not a bad thing to fall in love with me, me
Not such a bad thing to fall in love with me

So I understand if the status quo should be kept. But at least please allow for switching break handling at document/header declaration.

1 Like

I think what is needed for lyrics and poetry is a special environment that preserves not only line breaks but initial indentation and interior spaces. reStructuredText has “line blocks” for this purpose (and for things like addresses), and I have adopted them in pandoc’s markdown. They look like this:

| The limerick packs laughs anatomical
| In space that is quite economical.
|    But the good ones I've seen
|    So seldom are clean
| And the clean ones so seldom are comical

| 200 Main St.

Since the cases where hard line breaks are typographically appropriate are relatively uncommon, special cases, there should be a special environment (like the above line blocks) that triggers them.

4 Likes