# Mermaid - Generation of diagrams and flowcharts from text in a similar manner as markdown

@jgm:

You wrote: “Or it may occur inside an HTML comment.” — I have never thought of that one! And out of curiousity I tried it out with zhtml:

Text here,

<!--
%%Z
vanish!
%%
-->

and text there.


And indeed, the %%Z block inside the HTML comment gets converted to a bunch of HTML, which then vanishes inside the comment when the final HTML document is rendered! Gee!

But honestly, I won’t loose much sleep about this fancy incident: the text which zhtml sees is the hand-written typescript created by a human author (ie: me), and why would an author (me) do something like the above?

Unless of course, one wants to comment out some Z block, a possibility that never occured to me either …

The whole thing would blow up if the converted Z block would contain two HYPHEN-MINUS characters in a row, ie “--”, because that would destroy the HTML comment and render the final HTML document invalid.

Luckily there is no way I can think of to produce such a “--” sequence from inside a Z Notation block

I thought that a cheap cop-out would be to require %%Z to be preceded by an empty line: so there would be no HTML block for Markdown to preserve, but even with blank lines before %%Z and after %% the comment survives cmark and makes it into the HTML document. Is that intended? A similar construct

<DIV

class="hi-there"

>


does not “survive” parsing! It gets converted into this mistake:

<DIV
<P>class=&quot;hi-there&quot;</P>
<BLOCKQUOTE>
</BLOCKQUOTE>


which does not even resemble HTML any more. (And zhtml didn’t touch the <DIV> at all, using cmark -t xhtml only gives the same result!)

Is a HTML comment treated differently than a “stretched-out” regular element like this <DIV>? Does the CommonMark specification allow a screwed output like the one for the <DIV> example? Why?

I am about to embark on a project to implement a “general” post-processing structure using the idea of “transport elements” in which plain text marked-up in a “foreign” syntax gets sent to the appropriate processor.

See my Announcement and Request for Comments post in the Implementation category.

The approach of post-processing literal nodes (code and codeblock) with an identifier in the info-string is that which I have taken with the design of Typedown. Once you have the AST it is straightforward to delegate to various plugins / special content parsers to transform the AST further, as your application dictates.

1 Like

Is the content-type-declaration in Typedown (e.g. !import) located only within the info text? (if using your example in that page, maybe it would look like:)

 !import
src: path/to/imported/file.md


When I first implemented it, yes. Then I realized (obvious really) I could put the content-type-declaration within the info text line, and make the ! optional, making this very similar to normal syntax highlighting-like usage, like:

So…


!import
src: file.md



…becomes…

 !import
src: file.md



…, and in that case the exclamation becomes optional…

import
src: file.md



… and attributes are specified as a yaml flow block (YAML’s version of JSON), where the braces are implied. So:

import skip: false, title: "This file will be imported into the current document"
src: file.md



… is the same as:

import { skip: false, title: "This file will be imported into the current document" }
src: file.md



If the yaml-block content (not the stuff in the info string) is parsed as YAML also, the YAML file separator --- becomes a record separator within the block, so:

author id: authlist, description: "This is a list of authors", type: "!Array<{name:string}>"
name: Foo
---
name: Bar
---
name: Baz



(in this example, the type attribute is a @typedef using closure compiler annotation syntax)

1 Like

Would it be a problem to overload the code block syntax without the ! differentiator?

What if the author intend for import below to be just plain code blocks, but with import syntax highlighting etc…

import
src: file.md



I would suggest to encourage developers and users to stick to !, and keep the optional exclamation mark as an optional feature. It doesn’t seem much of an overhead to require the usage of !.

On additional thought, the bit after the content-type-declaration as you describe it. I think we don’t really need to enforce usage of YAML, since we can just pass the info string directly to whoever needs to handle it. However we can encourage best practices by providing a library that can parse the infotext in a consistent manner, but still allow for the developers to use the infotext in anyway they want.

It could be possible to restrict the info string between closing code fence brackets, to allow for additional keys via consistent attribute syntax as discussed here.

e.g.

 !import <infotext for !import handler>  { key=value }
src: file.md



Oh and just in case. The concept here is that the content inside the fenced block is not necessarily only YAML encoded. It’s can be anything. It’s up to the external plugin/preprocessor/postprocessor/AST/etc… to deal with.

I think you did mean “… the info string between openening code fence brackets, …”, right?

Why not put both the attribute list (enclosed in “{” “}” anyway, and thus recognizable) and after it the regular info string?

So any one of

{ key = value}


or

info-string


or

{ key=value } info-string


would be unambiguous.

Hey folks - did we come to a conclusion around rendering foreign input such as mermaidjs within commonmark?

!mermaid
graph TD;
A-->B;
A-->C;
B-->D;
C-->D;


Seems the easiest convention to implement and reads pretty nicely. What we’d probably do is actually generate the contents with-in a iframe so that we can delegate rendering of these components rather than trying to generate SVG’s etc with-in the commonmark renderer. That would also allow us to mix and match between client side of server side rendering depending on the external format being rendered inline.

1 Like

Sorry, this is a bit long, but I think context is important here.

At Discourse we support 2 means of extending the a Discourse installation.

1. Themes/components - these are JS and CSS packages you can add and remove while an installation is running

2. Plugins - These are JS / CSS and Ruby packages that require a full rebuild of your instance

Recently we had some inquiries from people to get mermaid support and they only had the freedom of using method (1)

Traditionally we used BBCODE to decorate blocks, so our original go-to here was to add support for:

[mermaid]
...
[/mermaid]


However, this is a bit of a breaking change to the platform. If we were to unconditionally fiddle with html for unknown bbcode tags we would get:

[test]
thing
[test]


Likely render as:

thing

(due to a new wrapping div)

[test]
thing
[test]

Like it does today.

We could add even more complexity to our engine and allow for:

<p data-wrap-type="test">
<span class="tagname">[test]</span><br>
test
<span class="tagname">[test]</span><br>
</p>


This would work, but it starts getting very kludgy.

@vitaly noticed BBCODE as a weaker part of our extensibility. We are building now on 9 years of history, we don’t want to make large breaking changes. That said the graphing problem is very interesting. (mermaid, graphviz, svgbob, charts)

We ended building this support which will land into core:

This allows us to follow the now forming “industry standard” which is

mermaid
graph here



It will also allow us to support this if we wish, but I worry that the industry has not yet adopted !, I guess GitHub could help push for this, I certainly see merit in calling this out, and it also allow for syntax highlighting of mermaid, something that the current ! less solution does not have.

!mermaid
graph here



We also opted to support attributes like so:

mermaid height=200,width=150



Finally, one thing that is still missing is some sort of parity for inline graphs, which this pattern does not support. BBCODE looks like the only easy way to support it from what I can tell.

I am an inline [mermaid height=100].....[/mermaid]


Mermaid does not have a compelling argument for inline stuff, but other things like mathjax and so on do have that, dealing generally with inline stuff is tricky and nothing in the spec helps with it, nor are there any areas that would be easy to expand to add this support.

Overall, I think the call of adding the ! into the spec so we can differentiate between syntax highlighting and block extensions has lots of merit.

3 Likes

IMO ! does not help with readability or programming. In theory, it could help to avoid collisions between language name & extension name, but i don’t know real world examples of that.

IMO

quote
text



looks more natural than

!quote
text



The last looks like pushing users think like programmers

If you wish block name to be !mermaid, you can do it right now, without spec change.

You are right. inline markup has no such simple principle to extend, as blocks. But AFAIK, at current moment only math equations are a real pain.

I’d propose:

• Create a list of demanded inline extensions (except math). If those are rare, inline bbcode would be not too horrible.
• Land ASAP math block/inline syntax (as separate well known problem)
2 Likes

Regarding

A:

mermaid


vs B:

!mermaid


Looks like the consensus is just to stick with A (without the !). Will ensure that GitHub aligns with that as we introduce mermaidjs support.

I was thinking that the !language approach helps differentiate between block rendering/execution vs straight syntax highlighting and would also have the advantage of not adjusting any existing examples of folks inlining mermaid code examples, but folks today don’t do !mermaid for those examples, they do javascript. Similarly even things like inline SVG would be xml in syntax highlighting terms.

Let’s kick off a separate thread for math?

3 Likes

I agree it is kind odd to be forced to do:


mermaid
graph TD;
A-->B;
A-->C;
B-->D;
C-->D;


mermaid-hl
graph TD;
A-->B;
A-->C;
B-->D;
C-->D;




That said, I get @vitaly’s argument here that we would be hurting readability for an edge case here.

Teaching people about something like mermaid-hl etc. when they need to apply highlighting to mermaid syntax is probably easier than teaching people about a new ! mark that would be required.

What about attributes? I am not sure we are aligned as an industry, but I think maybe GitHub should support something like this: (there is a forest theme here: https://github.com/mermaid-js/mermaid/tree/master/src/themes)

mermaid theme=forest



We have theoretical support for that scheme at Discourse now, just need to implement the component.

1 Like
mermaid



Under CommonMark this would display the source code of a Mermaid diagram if no extensions are applied. I do not think it should render the output of that code as this behaviour would be inconsistent from the behaviour of other code blocks with syntax highlighting. If the behaviour is inconsistent between different types of code blocks it doesn’t follow the principle of least surprise.

CommonMark renders to an HTML <code> element. The HTML Standard’s definition is “The code element represents a fragment of computer code.”

If we need to render the output of source code with a CommonMark document, perhaps it would be better to go with a dedicated syntax?

2 Likes

I have no personal preference about attrs format, and dont’ know reasons why something should be prefered/avoided.

I this case, following this logic, we will need new syntax for EVERY new block renderer. That’s overkill.

May be, i could agree, using fenced syntax to wrap things with nested md markup like quotes may be unusual, but i see no problem with code-like blocks, as mermaid.

Fallback of mermaid to code text block when no extension installed looks natural, IMO.

Also, spec (markdown) is for humans, not humans for spec. We should not push users follow blind with abstract rules, if result does not looks “natural”.

Let’s be realistic - doing nothing will cause waste of several years more. I’d be happy if process of math markup stabilization can move forward.

2 Likes

One more possibility - tag name can be more descriptive:

draw-mermaid
content



I’m not sure how nice and good is that. Just sharing idea.

1 Like

Yes it’s too far late to dictate a rule that says a fenced code block without some new signifier must only render as source. And adding modifiers to the first info string token such as !mermaid will break too many things as well.

The simplest most backward compatible solution would be to establish a standard around an optional second token of the info string that, if used, makes explicit whether the code block should be rendered as source code or “executed”. It could work like this:

1. an = second token is an explicit declaration that the content should be rendered literally, as source code.

 mermaid =
show the mermaid diagram source code here,
perhaps with syntax highlighting


2. a ( or () second token is an explicit declaration that the content should be “executed” (interpreted, rendered or otherwise processed) if possible.

 mermaid ()
render the diagram described here


 markdown ()
render the markdown source here


3. If neither of the above tokens occurs in the second position, you get todays behavior, thus backward compatibility.

 mermaid theme:dark
whatever happens today for the above info string



The above form also serves as the “user friendly” form, meaning that, for the given content type named by the first token, the most natural thing should happen. For Mermaid, what most users expect is that a diagram is rendered.

4. The remainder of the info string is passed thru to the syntax highlighter or extension determined by the first token, i.e. with the = or ( and ) removed. Calls to existing libraries will continue to work without changes.

 mermaid ( width:300px height:300px )
invoke the diagram renderer with the
following args:
width:300px height:300px


 javascript = numberedLines:true
configure the syntax highlighter with:
numberedLines:true



#### backward compatibility

The only case where backward compatibility might be lost is when all of the following are true:

• the renderer doesn’t know about the above tokens and does not remove them from the info string before passing it to the extension or syntax highlighter
• that extension looks beyond the first token
• and has brittle expectations for the second token (it isn’t designed to skip unknown tokens in the info string) and fails hard

I think this will be rather rare and mostly limited to power users, who will figure it out and update their software or demand that it gets updated.

The other proposals mod the first token. They won’t degrade gracefully.

1 Like

Let me summarize “problem”.

1. It’s not specific to mermaid, it’s about extending fenced-like markup for blocks, to land new renderers.
2. Currently, spec says fenced blocks are for code only. But IMO this approach has bigger potential than initially expected.
• I would not use alternate guards (::: and so on). IMO “” could be preferable (subject to discuss).
3. There are two main directions, where to extend fenced blocks:
a) Self-sufficient isolated content with alternate renderers (diagrams, math and so on)
b) Blocks with nested markdown content (quotes, asides, notes…)
4. It’s not mandatory, but would be nice to have recommendation about fenced params format.
5. IMO, the most influencing parties for CM promotion are Discourse & Gihtub.
• From my past experience, guys from Discource use markdown-it, and are very responsive. No problems at this side.
• Communication with Github is not clear. They publish own spec (CM + addons), but i don’t know contact, who can say explicit their plans and intents about future [syntax] extensions. It would be nice if we could move forward in sync.
6. IMO, extending fenced blocks for isolated content (see [3. a]) is not difficult and does not require too much time to collect & balance opinions. But for “solid” result, feedback from Github is mandatory (see [5]). I’d like to know exactly their opinion, if possible, before start. Of cause, we could do everything with Discource only, but that’s not good for CM spec in long term. I understand importance of unified spec, and would like to avoid any possibilities of future splits.
2 Likes

Could you clarify what breakage do you mean?

IMO, the only possible breakage is collision with language name for highlight, but i don’t know any examples from real world. In worst case, this can be solved by human-friendly tag name & recommendation how to create those.

For example:

• draw-mermaid
• draw-<anything_you_wish>

That similar to !, but more clear for human & still easy to type.

Alternative is to just use mermade and add mermade-code for crazy case, if anyone really wish to show source with highlight.

1 Like

I mean there is already much extant content that expects the following to result in rendered mermaid diagrams:

mermaid
diagram code



All I’m saying is that it’s too late to dictate that the above should always be rendered as source code.

How do you like my updated syntax for execution (mermaid ())?

2 Likes

I clarified the sentence you quoted.

1 Like