I see that your style already got quite some traction. 200+ stars etc.
As no one wrote sth yet, let me maybe challenge a few things. Or give some subjective opinions.
Bear in mind: This also always depends on if this markup language is for writers or readers, or supposed to be somewhat an ideal compromise for both (which my Carve markup is supposed to be).
Setext for H1/H2.
Setext heading content is a paragraph line, so it cannot hold text that would itself start a block: 1. Intro, - Overview, > Note, # Hash can never be setext, at any nesting. Same for a heading whose text carries a hard break. Does it fall back to ATX in those cases, or is there input where the output re-parses as a list or a thematic break? Related but cosmetic: a --- underline and a --- thematic break look identical to a human reading a diff, even though the parser knows better.
That said: I personally find the underlining with the same “length” super cumbersome and annoying and a thing from the past.
It really doesn’t make writing pleasant. So in my style (and in most modern markups like MD, Djot etc) this would not be considered useful to have. The `#` ATX headers exist here for a reason.
Four-tilde fences.
The real win here is cross-character immunity, not width: a closing fence has to use the same character as the opener, so a tilde-fenced block can hold backtick fences of any width without counting anything. That is the common case in Markdown documentation. But it only moves the collision rather than removing it, and it moves it straight into your own style guide, where every example is itself a tilde fence and therefore needs five or more tildes around it. STYLE.md says four tildes and does not mention content that contains tildes, so the question is whether the implementation computes the width anyway. The case that bites is not exotic: a closing fence is any line of only fence characters at up to three spaces of indentation, and reStructuredText section underlines are exactly that. Embed an RST snippet, or an ASCII divider, or hongdown’s own output in a hongdown-formatted file, and the block closes early. So: is the fence width derived from the content, or fixed at four? Interop-wise tilde fences are plain CommonMark, but plenty of downstream consumers (editors, highlighters, non-CommonMark renderers) only pattern-match on backticks.
I would recommend 3+ to open, and then always one more per nesting levels (or the other way around). This solves it already cleanly without having to set a fixed “4” length.
Fixed geometry in variable-indent containers.
Two rules with the same shape. Ordered list markers hold a fixed four-column width by shrinking the trailing padding, which is a nice trick at the 9 to 10 boundary, but it cannot survive 100, where the minimum one space pushes content indent to five and re-indents every continuation line in the list. Thematic breaks are 37 spaced dashes at three spaces of indentation: three is the maximum indentation a thematic break may have, so there is no margin left, and the fixed 76-column geometry has to live inside list items, nested lists and blockquotes that each shift it right. Does either rule scale with the wrap width or with nesting depth? The spaced dashes themselves are a good call, incidentally, since an underline with internal spaces can never be read as a setext H2, which matters exactly because H2 is setext here.
That said: The same issue as above: Humans writing such long manual breaks would go mad really fast
I prefer a simple `—` in my markup. Clear and concise.
Idempotency evidence.
The strongest proof is a round-trip property over the spec suite: for every example, parse(md) and parse(format(md)) yield the same AST (or at least the same HTML), plus format(format(md)) == format(md).
I use that for CI and it really helps to find the little bugs in between, especially with some fuzzy testing on top.
Hope this helps a bit to further streamline your specs.