# Can AST -\> Markdown -\> AST round-trip always reproduce the original? **URL:** \ **Category:** Application\ **Created:** [September 3, 2021, 3:20pm UTC](https://talk.commonmark.org/t/can-ast-markdown-ast-round-trip-always-reproduce-the-original/3959 "2021-09-03T15:20:28Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![DougReeder](https://cdn.commonmark.org/user_avatar/talk.commonmark.org/dougreeder/32/1026_2.png) [@DougReeder](https://talk.commonmark.org/u/DougReeder)\ **Post date:** [September 3, 2021, 3:20pm UTC](https://talk.commonmark.org/t/can-ast-markdown-ast-round-trip-always-reproduce-the-original/3959/1 "2021-09-03T15:20:28Z")
I’m considering the use of Markdown as a serialization format for my note-taking app. It’s doable to convert the abstract syntax tree (AST) of the editor (Slate) to Markdown. It’s straightforward to take the AST produced by the Commonmark parser and convert it to a Slate AST. The question is - For all ASTs, is there an algorithm that will transform them to Markdown, which, when run through the Commonmark reference implementation, will produce an AST identical to the original AST? ---
**Author:** ![jgm](https://cdn.commonmark.org/user_avatar/talk.commonmark.org/jgm/32/1360_2.png) [@jgm](https://talk.commonmark.org/u/jgm)\ **Post date:** [September 3, 2021, 4:52pm UTC](https://talk.commonmark.org/t/can-ast-markdown-ast-round-trip-always-reproduce-the-original/3959/2 "2021-09-03T16:52:34Z")
Doug Reeder via CommonMark Discussion [noreply@talk.commonmark.org](mailto:noreply@talk.commonmark.org) writes: > The question is - For all ASTs, is there an algorithm that will transform them to Markdown, which, when run through the Commonmark reference implementation, will produce an AST identical to the original AST? For all ASTS…certainly not. (If the AST doesn’t make enough distinctions, then it certainly won’t be possible.) For some ASTs? Yes, if the AST can represent all the distinctions that matter for commonmark, then it should in principle be possible to do a round trip. However, in practice it’s not easy, because of the context-sensitivity of markdown syntax. cmark’s commonmark renderer probably comes pretty close, but I haven’t done extensive tests of round-trippability, and there are a number of issues that I know about: > > > I don't know whether there's a bug here but perhaps it's interesting: > > \`\`\` > $ …loop() { > \> while \["$x" != "$x1" \] ; do > \> echo "$x" ; x1="$x" ; x=$(echo "$x" | cmark -t commonmark) > \> done > \> } > > $ x='x \*\*0\*+\*=\*\*\*' ; loop > x \*\*0\*+\*=\*\*\* > x \*\*0\*+\*=\*\*\\\* > x \*\*0\*+\\\*=\*\\\*\\\* > x \\\*\*0\*+\\\*=\\\*\\\*\\\* > > $ x='x \_\_?\_\_0\_}\_\_' ; loop > x \_\_?\_\_0\_}\_\_ > x \\\_\*?\*\*0\*}\*\* > x \\\_\\\*?\*\*0\*}\*\\\* > x \\\_\\\*?\\\*\*0\*}\\\*\\\* > \`\`\` > > At least it does converge. > > > \`\`\` > $ printf '\_a\*b\*c\_' | cmark -t html > \\a\b\c\\ > $ prin…tf '\_a\*b\*c\_' | cmark -t commonmark > \*a\*b\*c\* > $ printf '\_a\*b\*c\_' | cmark -t commonmark | cmark -t html > \\a\b\c\\ > \`\`\` > > > I see you've carefully avoided accidentally starting a list: > > \`\`\` > $ printf 'a…bcd x)' | cmark -t commonmark --width 4 > abcd > x) > $ printf 'abcd 1)' | cmark -t commonmark --width 4 > abcd 1) > \`\`\` > > But a heading can be accidentally created: > > \`\`\` > $ printf 'abcd =' | cmark -t commonmark --width 4 | cmark -t html > \abcd\ > $ printf 'abcd -' | cmark -t commonmark --width 4 | cmark -t html > \abcd\ > \`\`\` > > I think you can find this bug (and perhaps others) by writing a program that converts to HTML both directly and via CommonMark with width 4 and compares the two results and aborts if they're different, then fuzzing that program with AFL. > > > When \`--wrap=0\` (nowrap) is set, we get proper escaping to prevent an ordered li…st from being created: > \`\`\` > % ./build/src/cmark -t commonmark --width 0 > aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa > 1\\. aaaaaaaaaa > ^D > aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa > 1\\. aaaaaaaaaa > \`\`\` > > But with \`--wrap=72\` we don't: > \`\`\` > % ./build/src/cmark -t commonmark --width 72 > aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa > 1\\. aaaaaaaaaa > ^D > aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa > 1. aaaaaaaaaa > \`\`\` > > See jgm/pandoc#5597 > > > See https://github.com/kivikakk/comrak/issues/111 -- the string \`A\*\*\*B\*C\*\` doesn…'t roundtrip correctly. It's not exactly well-formed, but ideally we should be consistent. ---
**Author:** ![DougReeder](https://cdn.commonmark.org/user_avatar/talk.commonmark.org/dougreeder/32/1026_2.png) [@DougReeder](https://talk.commonmark.org/u/DougReeder)\ **Post date:** [September 3, 2021, 6:40pm UTC](https://talk.commonmark.org/t/can-ast-markdown-ast-round-trip-always-reproduce-the-original/3959/3 "2021-09-03T18:40:37Z")
Thanks, yes, I left out the clause “For all ASTs _that could be generated from Markdown_…”. My own tests suggest that you can _usually_ round trip it, but it would be difficult to have confidence that all the “reasonable” cases are covered. I’ll probably go back to serializing as a enforced subset of HTML. Is there a better serialization format for these ASTs? ---
**Author:** ![jgm](https://cdn.commonmark.org/user_avatar/talk.commonmark.org/jgm/32/1360_2.png) [@jgm](https://talk.commonmark.org/u/jgm)\ **Post date:** [September 3, 2021, 8:08pm UTC](https://talk.commonmark.org/t/can-ast-markdown-ast-round-trip-always-reproduce-the-original/3959/4 "2021-09-03T20:08:17Z")
> I’ll probably go back to serializing as a enforced subset of HTML. Is there a better serialization format for these ASTs? cmark will generate an XML representation of the AST. `cmark -t xml`