Mermaid - Generation of diagrams and flowcharts from text in a similar manner as markdown

The approach of post-processing literal nodes (code and codeblock) with an identifier in the info-string is that which I have taken with the design of Typedown. Once you have the AST it is straightforward to delegate to various plugins / special content parsers to transform the AST further, as your application dictates.

1 Like

Is the content-type-declaration in Typedown (e.g. !import) located only within the info text? (if using your example in that page, maybe it would look like:)

``` !import
src: path/to/imported/file.md
```

When I first implemented it, yes. Then I realized (obvious really) I could put the content-type-declaration within the info text line, and make the ! optional, making this very similar to normal syntax highlighting-like usage, like:

So…

``` 
!import
src: file.md
```

…becomes…

``` !import
src: file.md
```

…, and in that case the exclamation becomes optional…

```import
src: file.md
```

… and attributes are specified as a yaml flow block (YAML’s version of JSON), where the braces are implied. So:

```import skip: false, title: "This file will be imported into the current document"
src: file.md
```

… is the same as:

```import { skip: false, title: "This file will be imported into the current document" }
src: file.md
```

If the yaml-block content (not the stuff in the info string) is parsed as YAML also, the YAML file separator --- becomes a record separator within the block, so:

```author id: authlist, description: "This is a list of authors", type: "!Array<{name:string}>"
name: Foo
---
name: Bar
---
name: Baz
```

(in this example, the type attribute is a @typedef using closure compiler annotation syntax)

2 Likes

Would it be a problem to overload the code block syntax without the ! differentiator?

What if the author intend for import below to be just plain code blocks, but with import syntax highlighting etc…

```import
src: file.md
```

I would suggest to encourage developers and users to stick to !, and keep the optional exclamation mark as an optional feature. It doesn’t seem much of an overhead to require the usage of !.


On additional thought, the bit after the content-type-declaration as you describe it. I think we don’t really need to enforce usage of YAML, since we can just pass the info string directly to whoever needs to handle it. However we can encourage best practices by providing a library that can parse the infotext in a consistent manner, but still allow for the developers to use the infotext in anyway they want.

It could be possible to restrict the info string between closing code fence brackets, to allow for additional keys via consistent attribute syntax as discussed here.

e.g.

``` !import <infotext for !import handler> ``` { key=value }
src: file.md
``````````````````````````````````````````````

Oh and just in case. The concept here is that the content inside the fenced block is not necessarily only YAML encoded. It’s can be anything. It’s up to the external plugin/preprocessor/postprocessor/AST/etc… to deal with.

I think you did mean “… the info string between openening code fence brackets, …”, right?

Why not put both the attribute list (enclosed in “{” “}” anyway, and thus recognizable) and after it the regular info string?

So any one of

````{ key = value}

or

````info-string

or

````{ key=value } info-string

would be unambiguous.

Hey folks - did we come to a conclusion around rendering foreign input such as mermaidjs within commonmark?

```!mermaid
graph TD;
    A-->B;
    A-->C;
    B-->D;
    C-->D;

Seems the easiest convention to implement and reads pretty nicely. What we’d probably do is actually generate the contents with-in a iframe so that we can delegate rendering of these components rather than trying to generate SVG’s etc with-in the commonmark renderer. That would also allow us to mix and match between client side of server side rendering depending on the external format being rendered inline.

2 Likes

Sorry, this is a bit long, but I think context is important here.

At Discourse we support 2 means of extending the a Discourse installation.

  1. Themes/components - these are JS and CSS packages you can add and remove while an installation is running

  2. Plugins - These are JS / CSS and Ruby packages that require a full rebuild of your instance

Recently we had some inquiries from people to get mermaid support and they only had the freedom of using method (1)

Traditionally we used BBCODE to decorate blocks, so our original go-to here was to add support for:

[mermaid]
...
[/mermaid]

However, this is a bit of a breaking change to the platform. If we were to unconditionally fiddle with html for unknown bbcode tags we would get:

[test]
thing
[test]

Likely render as:

thing

(due to a new wrapping div)

Instead of:

[test]
thing
[test]

Like it does today.

We could add even more complexity to our engine and allow for:

<p data-wrap-type="test"> 
   <span class="tagname">[test]</span><br>
   test
   <span class="tagname">[test]</span><br>
</p>

This would work, but it starts getting very kludgy.

@vitaly noticed BBCODE as a weaker part of our extensibility. We are building now on 9 years of history, we don’t want to make large breaking changes. That said the graphing problem is very interesting. (mermaid, graphviz, svgbob, charts)

We ended building this support which will land into core:

This allows us to follow the now forming “industry standard” which is

```mermaid
graph here
```

It will also allow us to support this if we wish, but I worry that the industry has not yet adopted !, I guess GitHub could help push for this, I certainly see merit in calling this out, and it also allow for syntax highlighting of mermaid, something that the current ! less solution does not have.

```!mermaid
graph here
```

We also opted to support attributes like so:

```mermaid height=200,width=150
```

Finally, one thing that is still missing is some sort of parity for inline graphs, which this pattern does not support. BBCODE looks like the only easy way to support it from what I can tell.

I am an inline [mermaid height=100].....[/mermaid]

Mermaid does not have a compelling argument for inline stuff, but other things like mathjax and so on do have that, dealing generally with inline stuff is tricky and nothing in the spec helps with it, nor are there any areas that would be easy to expand to add this support.

Overall, I think the call of adding the ! into the spec so we can differentiate between syntax highlighting and block extensions has lots of merit.

3 Likes

IMO ! does not help with readability or programming. In theory, it could help to avoid collisions between language name & extension name, but i don’t know real world examples of that.

IMO

```quote
text
```

looks more natural than

```!quote
text
```

The last looks like pushing users think like programmers :slight_smile:

If you wish block name to be !mermaid, you can do it right now, without spec change.

You are right. inline markup has no such simple principle to extend, as blocks. But AFAIK, at current moment only math equations are a real pain.

I’d propose:

  • Create a list of demanded inline extensions (except math). If those are rare, inline bbcode would be not too horrible.
  • Land ASAP math block/inline syntax (as separate well known problem)
2 Likes

Regarding

A:

```mermaid

vs B:

```!mermaid

Looks like the consensus is just to stick with A (without the !). Will ensure that GitHub aligns with that as we introduce mermaidjs support.

I was thinking that the !language approach helps differentiate between block rendering/execution vs straight syntax highlighting and would also have the advantage of not adjusting any existing examples of folks inlining mermaid code examples, but folks today don’t do !mermaid for those examples, they do javascript. Similarly even things like inline SVG would be xml in syntax highlighting terms.

Let’s kick off a separate thread for math?

3 Likes

I agree it is kind odd to be forced to do:


```mermaid
graph TD;
    A-->B;
    A-->C;
    B-->D;
    C-->D;
```

```mermaid-hl
graph TD;
    A-->B;
    A-->C;
    B-->D;
    C-->D;
```


That said, I get @vitaly’s argument here that we would be hurting readability for an edge case here.

Teaching people about something like mermaid-hl etc. when they need to apply highlighting to mermaid syntax is probably easier than teaching people about a new ! mark that would be required.

What about attributes? I am not sure we are aligned as an industry, but I think maybe GitHub should support something like this: (there is a forest theme here: https://github.com/mermaid-js/mermaid/tree/master/src/themes)

```mermaid theme=forest
```

We have theoretical support for that scheme at Discourse now, just need to implement the component.

1 Like
```mermaid

Under CommonMark this would display the source code of a Mermaid diagram if no extensions are applied. I do not think it should render the output of that code as this behaviour would be inconsistent from the behaviour of other code blocks with syntax highlighting. If the behaviour is inconsistent between different types of code blocks it doesn’t follow the principle of least surprise.

CommonMark renders to an HTML <code> element. The HTML Standard’s definition is “The code element represents a fragment of computer code.”

If we need to render the output of source code with a CommonMark document, perhaps it would be better to go with a dedicated syntax?

2 Likes

I have no personal preference about attrs format, and dont’ know reasons why something should be prefered/avoided.

I this case, following this logic, we will need new syntax for EVERY new block renderer. That’s overkill.

May be, i could agree, using fenced syntax to wrap things with nested md markup like quotes may be unusual, but i see no problem with code-like blocks, as mermaid.

Fallback of mermaid to code text block when no extension installed looks natural, IMO.

Also, spec (markdown) is for humans, not humans for spec. We should not push users follow blind with abstract rules, if result does not looks “natural”.

Let’s be realistic - doing nothing will cause waste of several years more. I’d be happy if process of math markup stabilization can move forward.

2 Likes

One more possibility - tag name can be more descriptive:

```draw-mermaid
content
```

I’m not sure how nice and good is that. Just sharing idea.

1 Like

Yes it’s too far late to dictate a rule that says a fenced code block without some new signifier must only render as source. And adding modifiers to the first info string token such as !mermaid will break too many things as well.

The simplest most backward compatible solution would be to establish a standard around an optional second token of the info string that, if used, makes explicit whether the code block should be rendered as source code or “executed”. It could work like this:

  1. an = second token is an explicit declaration that the content should be rendered literally, as source code.

    ``` mermaid =
    show the mermaid diagram source code here, 
    perhaps with syntax highlighting
    ```
    
  2. a ( or () second token is an explicit declaration that the content should be “executed” (interpreted, rendered or otherwise processed) if possible.

    ``` mermaid ()
    render the diagram described here
    ```
    
    ``` markdown ()
    render the markdown source here
    ```
    
  3. If neither of the above tokens occurs in the second position, you get todays behavior, thus backward compatibility.

    ``` mermaid theme:dark
    whatever happens today for the above info string
    ```
    

    :triangular_flag_on_post: The above form also serves as the “user friendly” form, meaning that, for the given content type named by the first token, the most natural thing should happen. For Mermaid, what most users expect is that a diagram is rendered.

  4. The remainder of the info string is passed thru to the syntax highlighter or extension determined by the first token, i.e. with the = or ( and ) removed. Calls to existing libraries will continue to work without changes.

    ``` mermaid ( width:300px height:300px )
    invoke the diagram renderer with the
    following args:
       width:300px height:300px
    ```
    
    ``` javascript = numberedLines:true
    configure the syntax highlighter with:
       numberedLines:true
    ```
    

backward compatibility

The only case where backward compatibility might be lost is when all of the following are true:

  • the renderer doesn’t know about the above tokens and does not remove them from the info string before passing it to the extension or syntax highlighter
  • that extension looks beyond the first token
  • and has brittle expectations for the second token (it isn’t designed to skip unknown tokens in the info string) and fails hard

I think this will be rather rare and mostly limited to power users, who will figure it out and update their software or demand that it gets updated.

The other proposals mod the first token. They won’t degrade gracefully.

1 Like

Let me summarize “problem”.

  1. It’s not specific to mermaid, it’s about extending fenced-like markup for blocks, to land new renderers.
  2. Currently, spec says fenced blocks are for code only. But IMO this approach has bigger potential than initially expected.
    • I would not use alternate guards (::: and so on). IMO “```” could be preferable (subject to discuss).
  3. There are two main directions, where to extend fenced blocks:
    a) Self-sufficient isolated content with alternate renderers (diagrams, math and so on)
    b) Blocks with nested markdown content (quotes, asides, notes…)
  4. It’s not mandatory, but would be nice to have recommendation about fenced params format.
  5. IMO, the most influencing parties for CM promotion are Discourse & Gihtub.
    • From my past experience, guys from Discource use markdown-it, and are very responsive. No problems at this side.
    • Communication with Github is not clear. They publish own spec (CM + addons), but i don’t know contact, who can say explicit their plans and intents about future [syntax] extensions. It would be nice if we could move forward in sync.
  6. IMO, extending fenced blocks for isolated content (see [3. a]) is not difficult and does not require too much time to collect & balance opinions. But for “solid” result, feedback from Github is mandatory (see [5]). I’d like to know exactly their opinion, if possible, before start. Of cause, we could do everything with Discource only, but that’s not good for CM spec in long term. I understand importance of unified spec, and would like to avoid any possibilities of future splits.
2 Likes

Could you clarify what breakage do you mean?

IMO, the only possible breakage is collision with language name for highlight, but i don’t know any examples from real world. In worst case, this can be solved by human-friendly tag name & recommendation how to create those.

For example:

  • draw-mermaid
  • `draw-<anything_you_wish>

That similar to !, but more clear for human & still easy to type.

Alternative is to just use mermade and add mermade-code for crazy case, if anyone really wish to show source with highlight.

1 Like

I mean there is already much extant content that expects the following to result in rendered mermaid diagrams:

```mermaid
diagram code
```

All I’m saying is that it’s too late to dictate that the above should always be rendered as source code.

How do you like my updated syntax for execution (mermaid ())?

2 Likes

I clarified the sentence you quoted.

1 Like

Ah, no problem then. IMO, fenced blocks have bigger potential, than initially expected and declared in spec. It should not be a principal problem to clarify.

But as i said, we should unblock syntax extension process from main spec. To be able add new things in parallel, but still with reasonable guarantees of compatibility. And that requires more active participation of Github. At least, with “official” confirmation of acceptance.

Probably, @martinwoodward could give some light about what we could expect.

2 Likes

In the case of Mermaid, there are two primary ways of viewing the block: as source or as the rendered output.

In practice if there are differences with how the block is rendered in different implementations, that rendering is not going to be portable. This could change the meaning of the document between implementations.

In the case of a Mermaid diagram, for implementations that only render the output, it will become impossible for others to extend/change the diagram without access to the source Markdown file (or rewriting the diagram). This may not have been the intent of the original author, for example if they intended to demonstrate how to write the source Mermaid syntax to the reader.

So that’s why I think we need to a seperate, explicit syntax to differentiate between the two conceptually seperate approaches to displaying Mermaid. That could be extending the fenced code block with a flag or a new syntax.

1 Like