Is CommonMark really Markdown "compatible"?

As stated earlier, it is literally impossible to implement Markdown reliably because the current spec is so ambiguous – and the last official code release was in 2004. Thus, every single implementation of Markdown deviates from “Markdown” in unknowable ways.

We generally define “compatible” as “meets consensus in Babelmark”, e.g. the unavoidable diverging implementations of Markdown (as the spec was ambiguous-by-design) agree most on this set. See for yourself.

This is where the word common comes from, as in “In this set of markup, Markdown is commonly interpreted to mean…”

There is definitely a tiny bit of unavoidable editorial choice in there, since there might be two “reasonable” divergences that you have to decide among. Wherever choices were made we tried to stay in harmony with the original goals of Markdown.

3 Likes

We generally define “compatible” as “meets consensus in Babelmark”, e.g. the unavoidable diverging implementations of Markdown (as the spec was ambiguous-by-design) agree most on this set. See for yourself.

But you’re trying to redefine what Markdown is (which I think is why John Gruber keeps getting upset). Markdown has a spec which CommonMark unambiguously violates. I happen to agree with your changes, but they’re not just “Markdown without ambiguity”.

1 Like

As I said, every implementation of Markdown violates the current “spec” because it is so ambiguous. So this is kind of a meaningless statement to make.

1 Like

As I said, every implementation of Markdown violates the current “spec” because it is so ambiguous.

This doesn’t follow. If the spec is ambiguous, then different implementations can choose to implement the ambiguous behavior in different (incompatible) ways, but that doesn’t mean that they’re not compliant with the spec. “Undefined behavior” in C and C++ is a good example of this. Do you mean that Markdown has conflicting requirements?

I could see what you mean if Markdown has conflicting requirements and your changes were just to pick one which one is more important, but are you really arguing that adding fenced code blocks is an example of removing ambiguity in the original spec?

1 Like

Absolutely. Give an average user the task of putting code inside a numbered list and watch them struggle with the hidden 8 space rule. Very, very few people ever figure this out.

On top of that, there are also many places where the 4 space indent rule for code introduces additional ambiguities in output. So having a fenced code block that starts ‘flat’ against the left side

```
like so
```

versus

    like so

does allow you to express Markdown in a way that is less prone to ambiguities of interpretation. For example, try this.

is this a code
    block?

And existing code blocks continue to work fine, of course. That’s the “compatible” bit.

2 Likes

You feel it’s constructive to describe other contributor’s observations as ‘meaningless’ ?

Your own claim is certainly not ‘meaningless’, but its meaning does seem arguable:

every implementation of Markdown violates the current “spec” because it is so ambiguous

Well, John MacFarlane openly acknowledges that there are:

places where this spec says things that contradict the canonical syntax description

Do you disagree then, with John’s distinction between contradictions and variations authorised by ambiguity ?

Unless you do, it might well seem just a little careless or aggressive to claim that all variations within the scope of ambiguity “violate” the current spec, and perhaps a little combative or defensive to inform others (on demonstrably shaky ground) that their very reasonable and helpful statements are ‘meaningless’.

In the last analysis, any technical standard (or indeed constitution or law) expresses a compromise between differing interests, and the nature of the compromise is effectively shaped by the prevailing balance of forces at the time.

If this standard is to enjoy respect and confidence, contributors do need to feel that their views and needs are respected, and that the shots are not being called exclusively by those who happen to own the platform of discussion, or are given to egregiously self-righteous or aggressive rhetoric.

In short, they need to feel that the forum is not a charade, a theatre set up by powerful interests who are determined to shape events in the manner which best suits their own interests and businesses.

The tension between:

  1. Claims that the initial project is to add rigour to the existing core, and
  2. the fact that it already incorporates changes which in John MacFarlane’s words “contradict the canonical syntax description”, and which are closely associated with particular and powerful players,

is already expressive of a balance of forces which might well risk undermining credibility, and alienating potential adopters.

A pity, because the product of these efforts could be valuable.

PS

I look forward (in 5-10 years time) to reading a Wiki entry on the prevailing Standard Plain Text Markup, which comments that it emerged out of an earlier standard, now forgotten, which was called Markdown.

It would be good to think that something like that, by any other name, could emerge from all of the goodwill and thoughtful contributions that are accumulating here.

But counter-productive preoccupation with variants of the word ‘markdown’, and with securing victory in various over-publicised personal conflicts, can only weaken that prospect.

2 Likes

Personally, I feel the best strategy for CommonMark is to distance itself from Markdown.

Not attempt to build “a canonical Markdown” but instead attempt to build something better specified, more extensible that will last us for the next 10 years influenced heavily by Markdown, but not married to it.

It should also have a very aggressive goal of displacing Markdown and its 50 differing implementations.

I do feel that extensibility and in particular extensibility “standards” are at odds with Markdown, there is zero consumer value in having 700 different “table” plugins, CommonMark should be specifying this stuff.

Similarly certain controversial decisions need to be made at the core, like newline handling and so on.

I really hope that the answer to “Is CommonMark really Markdown “compatible”?” Becomes, you can easily drop in CommonMark and expect the vast majority of users to notice no change, except less odd bugs.

6 Likes

The OP has one point that seems to have fallen under the table: why keep the Markdown logo?

From a consumer standpoint alone it’s totally confusing if Markdown and CommonMark use the same logo.

That being sad, who actually knows what the Markdown logo looks like? I happen to know it, but I suspect a vast majority of e.g. GitHub or StackExchange users don’t (and may not even know they’re typing Markdown). What I’m trying to say is: I don’t believe the Markdown logo is very recognizable, so sticking to it has little gain but the above mentioned drawback.

I believe a new CommonMark logo would be better.

4 Likes

I’m in agreement with Sam that CommonMark should distance itself from Markdown for a few reasons:

  1. To put the naming controversy behind us. (Also why we need a new logo).
  2. To allow CommonMark to go its own direction and supersede Markdown in its own right.
  3. Extensibility - if we want to add something that the founders of the CommonMark spec don’t feel should be included, and was never part of Markdown originally, we should be able to say “this is an implementation of CommonMark with Tables and ID headers” and people will know what we mean.

That is the goal. More extensibility is further on, since we’re currently building on quicksand and that has to be addressed first.

The same reason USB devices have the USB logo – it indicates compatibility.

“oh, I can plug my USB cable in here and it will work”

“oh, I can type Markdown in here and it will work”

That’s an awfully … convenient … place to clip the quote. Let’s look at the full quote:

Because Gruber’s syntax description leaves many aspects of the syntax undetermined, writing a precise spec requires making a large number of decisions, many of them somewhat arbitrary. In making them, I have appealed to existing conventions and considerations of simplicity, readability, expressive power, and consistency. I have tried to ensure that “normal” documents in the many incompatible existing implementations of Markdown will render, as far as possible, as their authors intended. And I have tried to make the rules for different elements work together harmoniously. In places where different decisions could have been made (for example, the rules governing list indentation), I have explained the rationale for my choices. In a few cases, I have departed slightly from the canonical syntax description, in ways that I think further the goals of Markdown as stated in that description.

There are only a few places where this spec says things that contradict the canonical syntax description:

Here is what you quoted:

places where this spec says things that contradict the canonical syntax description

What happened to “only a few”? Was quoting a complete sentence not possible?

I suggest reading the paragraph before which provides the complete context.

3 Likes

We can take as large a sample of John’s texts as we like (the larger the better) and we still find him drawing a important distinction between:

  1. Variants authorised by ambiguity
  2. Contradictions of the original spec.

This (correct) distinction is incompatible with your (repeated) claim (see above):

As I said, every implementation of Markdown violates the current “spec” because it is so ambiguous

The point is not whether the number of contradictions of the Gruber spec is small or large.

The point is that there is a difference between legitimate variation within the scope of ambiguity, and straightforward contradiction. John seems relaxed about acknowledging that.

Is there some value in plastering over it, or trying to skate over it rapidly ?

Either the new spec is ‘Markdown’ or it isn’t.

If it contains any contradictions of the spec at all, even one, (and John acknowledges that there is more than one) then it may well be better than Markdown for some purposes, (I think it is), but it is not ‘compatible’ with it.

Hanging on to the apron-strings of the ‘Markdown’ name, while developing a spec which contains contradictions of it, has a cost, but no value.

The cost is of creating an impression of being more preoccupied with effective ownership and control of a name than with simply developing a good and widely respected spec.

Surely an enriched and more rigorous spec is more valuable and more motivating than a costly and constraining ‘Markdown’ fetish ?

Let me see if have understood your position.

You believe that every implementation ‘violates’ Gruber’s Markdown spec, but that CommonMark is ‘compatible’ with it ?

So there is only one class of implementation – those that violate ? No distinction between those that vary within the scope of ambiguity and those that contradict ?

Are you then inviting us to speak a language in which ‘violates’ is a synonym of ‘compatible with’ ?

Is that the sense in which you speak of CommonMark being ‘compatible with’ Markdown ? (‘compatible with’ = ‘violates’) ?

I have to say that I find it easier to make sense of John’s distinction, which gives us two classes of implementations:

  1. Those that vary within the scope of ambiguity, and
  2. those that contradict.

‘Compatible’ seems a good name for the first class of implementation.

(But an unconvincing and rather unhelpful name for the second class)

No, I don’t think such concrete clarity is possible here. Markdown is used by many thousands of people, of those, I’d think less than 10% use (or have ever used) the original markdown.pl implementation (Perl isn’t commonly used in application development, whether deskop, mobile or web).

So for all those who’ve learned markdown through PHPMarkdown or Discount or any of the zillion other implementations – markdown for them is that specific implementation.

That’s why CommonMark(down) uses Babelmark to determine the consensus of all the mainstream markdown implementations, and generally uses the consensus as the standard.

So to the most people, this will be hopefully turn out to be “markdown” in their eyes. A laudable goal, I think.

4 Likes

I get the idea, but can you provide an example of where the Markdown logo is actually used in such a way?

I checked StackOverflow and GitHub and the Markdown logo is nowhere to be found.

On StackOverflow, when writing a question, there is a “How to format” help box, but it doesn’t even mention that the format being used is Markdown – the word “Markdown” first appears when you follow the formatting help link, but still no logo in sight.

This brings me to my initial point: the Markdown logo isn’t (very) recognizable, the comparison with the USB logo falls short because I personally haven’t seen the logo used in this way.


Aside from that, what about features exclusive to CommonMark, like fenced code blocks?

As a user, I would appreciate it if I knew I could enter CommonMark and not just Markdown, simply by glancing at the CommonMark logo.

If the Markdown logo alone is used, I have to try whether or not fenced code blocks (or any other non-standard feature) is supported. I honestly don’t get why CommonMark should cling to the Markdown logo (or the name “Markdown”).

Well, maybe that’s because the logo is as old as CommonMark? It may or may not become widely adopted. Only time will tell. But if it does, it will indicate that “this place is compatible with CommonMark, so everything that goes in CommonMark, goes here”.

Or, at least, that’s how I’ve understood it.

Actually, no. The logo has been around since 14 Mar 2012, the work of Dustin Curtis, and is found in places like WhatIsMarkdown, one “evangelist’s” attempt to promote its use. More at the Github repo.

1 Like

You should use this mark to identify user input areas which support Markdown-compiled HTML output or to identify general Markdown support. Although you may modify this mark any way you like, without restriction, please respect the following guidelines

CommonMark certainly counts as “general Mn"; "general Mn” is kind of the whole mission statement.

1 Like

Look more closely:

1 Like

My bad, I only looked at the source editor of markdown files, where I can’t spot it::

Is it somewhere on StackOverflow too? It should be, being the USB-logo equivalent and all.

But this discussion is pointless anyways if just a tiny bit of my argument is picked out and proven (partially?) wrong while the rest is ignored. The logo isn’t widely adopted yet so there’s no point to cling to it. To me this feels like the name war is being continued here for the same childish reasons. (Don’t bother “replying”, the topic’s clear to me now)