And if I apply --normalize it gives the same result as commonmark.js.
It’s not something of big importance, but I am wondering for the following reason. I am testing my implementation against the AST output instead of HTML and at first I was using the commonmark.js output and had a hard time to make the test mentioned above pass (as my implementation was giving same AST as not-normalized cmark gives), so I left that test for better times.
But after some time I’ve decided to switch to cmark for test generation and I was quite surprised, when the test case fixed itself.
Or probably commonmark.js is normalized by default, whereas cmark isn’t.
I am wondering, is it expected that different implementations (or at least reference implementations) match each other’s AST output.
With cmark --normalize --sourcepos -t xml they should match.
I suppose we could just make --normalize the default for XML output.
If we used XML for the spec test, I think we’d use something like
the output of cmark --normalize (without source positions which add
a lot of clutter).
With cmark --normalize --sourcepos -t xml they should match.
Indeed, with --normalize they match, and I didn’t consider absense sourcepositions as ‘doesn’t match’
I suppose we could just make --normalize the default for XML output.
Yes. It seems reasonable when both implementation behave identically by default.
If we used XML for the spec test, I think we’d use something like the output of cmark --normalize (without source positions which add a lot of clutter).
For the spec tests it makes sense to be normalized as they are testing the implementation as a whole (without considering inner parts (e.g. inline parser, block structure parser, normalizer)).
However for the internal tests of particular parser implementation, it seems more reasonable to test non normalized output, as that is what parser outputs before normalization. But these details are not in the concern of a specification.
At the moment, tests for my Haskell implementation of a parser work as follows :
Parse CommonMark using my parser into not normalized Haskell AST.
Parse CommonMark using cmark into not normalized XML.
Parse cmark’s XML and transform it into my Haskell AST.
I also should add that when trying to understand how to implement the parser for particular structure it is much more helpful to refer to not normalized examples.
I also should add that when trying to understand how to implement the parser for particular structure it is much more helpful to refer to not normalized examples.
Well, the non-normalized examples reveal certain quirks of how the particular parser works. But of course, looking at the code is an even better way to study that!
Glad to hear you’re working on a Haskell implementation. I look forward to seeing it.