# Djot: A light markup language by @jgm **URL:** \ **Category:** Spec\ **Created:** [July 12, 2022, 1:13am UTC](https://talk.commonmark.org/t/djot-a-light-markup-language-by-jgm/4229 "2022-07-12T01:13:40Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![vas](https://cdn.commonmark.org/user_avatar/talk.commonmark.org/vas/32/1267_2.png) [@vas](https://talk.commonmark.org/u/vas)\ **Post date:** [July 12, 2022, 1:13am UTC](https://talk.commonmark.org/t/djot-a-light-markup-language-by-jgm/4229/1 "2022-07-12T01:13:40Z")
Continuing the discussion from [Beyond Markdown](https://talk.commonmark.org/t/beyond-markdown/2787/61) because I think @jgm’s **jdot** deserves its own forum topic. > **[GitHub - jgm/djot: A light markup language](https://github.com/jgm/djot)** > > A light markup language Link to [a Playground created by @dtinth](https://github.com/jgm/djot/issues/2#issuecomment-1180546329): [https://djot-playground-poc.vercel.app](https://djot-playground-poc.vercel.app) ---
**Author:** ![vas](https://cdn.commonmark.org/user_avatar/talk.commonmark.org/vas/32/1267_2.png) [@vas](https://talk.commonmark.org/u/vas)\ **Post date:** [July 12, 2022, 1:15am UTC](https://talk.commonmark.org/t/djot-a-light-markup-language-by-jgm/4229/2 "2022-07-12T01:15:54Z")
My thoughts… ## Lot’s of things to like: 1. Simpler and richer inline syntax. The [precedence rules](https://htmlpreview.github.io/?https://github.com/jgm/djot/blob/master/doc/syntax.html#precedence). 2. Nixing indented code blocks. Yea! 🎉 3. Footnotes. Yea! 🎉 4. No magic whitespace thresholds. 🎉🎉 5. Leading and trailing pipes in Pipe Tables are required. 6. Official support for custom containers / divs 🚀 ## The things I’m not so sure of, feels to much like writing code: - The elimination of shortcut link syntax. I think this makes things harder for humans just to make them easier for machines. I thought the point of machines was the reverse. I guess am not in favor of [Rationales](https://github.com/jgm/djot#rationale) #1 and #2. - The `{_` and `_}` syntax. But maybe the use cases for this are so corner that it’s ok that it’s not very natural. ## I don’t yet grok the impact of _Rationale #6_ Rationale #6: > Parsers should not be forced to recognize unicode character classes, HTML tags, or entities, or perform unicode case folding. That adds a lot of complexity. I suspect the case-sensitivity will be counterintuitive to most people and will result in a lot of broken links. My understanding is that djot does this purely for performance reasons. This to me is another case of favoring the machine over the human. ## Manual line wrapping: The decision I question most, for Markdown, djot and others > 🚩 To be clear, this criticism is NOT djot specific. But if we are going to talk about a replacement for Markdown, it must include a reevaluation of any assumptions or decisions that might have been a mistake. The unintended consequences to a syntax precipitated by support for manual line wrapping are far ranging and unintuitive, consequences I believe resulting in the well-known non-programmer dislike of Markdown and other “writer-oriented” markup syntaxes. You might think that manually wrapping lines is entirely optional. Yes it is, but paying the costs of that syntax support is not. _Rationale #7:_ > The syntax should be friendly to hard-wrapping: hard-wrapping a paragraph should not lead to different interpretations. Before I get into the costs, I want to question the justification: > why hard-wrap at all? Answer: so that your document is readable just as it is, without conversion to HTML and without special editor modes that soft-wrap long lines Is soft wrapping truly a “special editor mode”? If it is, it seems like another case where we’re forgetting that machines should be serving humans. There are plenty of editors that soft wrap, and many of them soft wrap intelligently, maintaining the indentation of the preceding line. And for any editor that is going to support syntax highlighting, it can just as easily support smart soft wrapping and indentation that reflects the syntax, e.g. dynamically indenting to align list item content past the list marker. ### Some of the many costs (to humans) imposed by manual line wrapping support - You need to use a special character to force a line break, e.g. placing a `\` at the end of each line of verse. - Writing block quotes or list items with manual line wrapping is a real chore. No one should have to enter so many `>` chars or have to manually indent each wrapped line of a list item. That should be the machine’s job, and in fact many editors do automatic indentation of soft-wrapped lines. - Editing existing manually wrapped text becomes a chore. Inserting even a word requires manually rewrapping all of a paragraph. It’s even worse for block quotes, as you have to remove and reinsert `>` as you manually hard wrap. Sure, you can choose not to rewrap, but then why the manual wrapping in the first place? - Supporting manual line wrapping results in syntax complications all over the place. For example, you are forced to choose between CommonMark’s complex rules for starting a list item after a paragraph without a blank line, or always requiring a blank line as djot does. Forcing blank lines seems the simple solution, but it’s very natural for many people to not use them, even feeling “wrong” to be forced to use them when unnecessary: - Complex and unnatural behavior for block continuation: - Alway using blank lines isn’t natural or often desirable, as explained within this example: There other costs. The ones above are off the top of my head. But in essence: 🌶 _Manual line breaks and its unintended consequences to me are Markdown’s greatest Achilles heal. It unnecessarily took us backward in time, back to dumb typewriters, where you have to manual slide the print head at the end of each line, where editing becomes a nightmare. Just because we want to ditch the proprietary or human-unfriendly storage formats of word processors doesn’t mean we have to go back to typewriters._ ---
**Author:** ![vas](https://cdn.commonmark.org/user_avatar/talk.commonmark.org/vas/32/1267_2.png) [@vas](https://talk.commonmark.org/u/vas)\ **Post date:** [July 14, 2022, 9:35pm UTC](https://talk.commonmark.org/t/djot-a-light-markup-language-by-jgm/4229/3 "2022-07-14T21:35:43Z")
I fixed some significant typos and added clarifications above. The most important being: > [@vas](#): > > 🚩 To be clear, this criticism is NOT djot specific. But if we are going to talk about a replacement for Markdown, it must include a reevaluation of any assumptions or decisions that might have been a mistake. My obviously strong and likely controversial thoughts on manual line wrapping reflects a lot time and work I’ve put into a project aimed at making plain text more expressive and flexible while making it more natural at the same time. I’ve examined many, many paths backwards and forwards, not just in my head but in countless use-cases and working code. In doing so, I discovered how much complexity and _unnaturalness_ derive simply from the manual line wrapping fork in the road. This fork occurred so far back that I think most of us, myself included, don’t even realize it was a fork, that we could have made a different choice, and that there is a whole other realm of possibility on the other side of the mountain. I’m not done with my exploration. I’ve only recently decided to backtrack all the way to that primordial fork and will be exhaustively examining the other world. Maybe I’ll discover I was wrong / rediscover the reason all(?) lightweight markup formats made the choice they did. I wasn’t going to write about this yet, but it was djot _doubling down_ on manual line wrapping that spurred me to write. I’m hoping this leads to a fruitful discussion. ---
**Author:** ![jgm](https://cdn.commonmark.org/user_avatar/talk.commonmark.org/jgm/32/1360_2.png) [@jgm](https://talk.commonmark.org/u/jgm)\ **Post date:** [July 15, 2022, 8:59am UTC](https://talk.commonmark.org/t/djot-a-light-markup-language-by-jgm/4229/4 "2022-07-15T08:59:28Z")
Thanks for these interesting comments. The issue of hard-wrapping was one we talked about in the discussions leading up to commonmark. I have to say that I don’t really agree with your readability judgments; it seems to me that the extra whitespace is desirable (and certainly not undesirable) in these cases. A couple additional points: First, if your proposal is that a hard line break should be interpreted as a hard break (i.e. `
`), then there would _still_ be an ambiguity with ```auto She counted (all the way from 1 to 9) and then went in. ``` Should this be parsed as a single paragraph containing a hard line break and the literal `9)`, or a paragraph followed by an ordered list? Here you’d need something like commonmark’s current kludge (making a special exception for the number 1 for starting a list without a line break), or you’d need to forbid hard breaks before potential list markers (an expressive blind spot). Second, genuine hard line breaks inside a paragraph are extremely rare in good typography; it seems a shame to assign this rare thing to the newline character. A somewhat different proposal would make newlines create _paragraph_ breaks. But this isn’t good for readability; it will be too unclear where a paragraph ends and the next one begins. Third, paragraphs consisting of extremely long lines can cause problems unrelated to readability. For example, lines of text in emails are not supposed to exceed 998 characters ([RFC 5322 - Internet Message Format](https://datatracker.ietf.org/doc/html/rfc5322#section-2.1.1)). Coding standards also sometimes impose a maximum line length; if the syntax is used in code comments, it would be affected by this. Diffability is another important consideration: if you have extremely long lines, it is harder to see where changes have occurred inside a paragraph. Djot does not _require_ hard-wrapping, but it allows it, and I think that’s still important. ---
**Author:** ![touchmarine](https://cdn.commonmark.org/user_avatar/talk.commonmark.org/touchmarine/32/2573_2.png) [@touchmarine](https://talk.commonmark.org/u/touchmarine)\ **Post date:** [July 19, 2022, 2:27pm UTC](https://talk.commonmark.org/t/djot-a-light-markup-language-by-jgm/4229/7 "2022-07-19T14:27:15Z")
Sure. I intended to write a short summary that would fit into here but it quickly got long. New topic: [My Experiences Developing a Personal “Beyond Markdown” markdown - Touch Lightweight Markup Language](https://talk.commonmark.org/t/my-experiences-developing-a-personal-beyond-markdown-markdown-touch-lightweight-markup-language/4239) ---
**Author:** ![cben](https://cdn.commonmark.org/user_avatar/talk.commonmark.org/cben/32/407_2.png) [@cben](https://talk.commonmark.org/u/cben)\ **Post date:** [July 25, 2022, 9:26pm UTC](https://talk.commonmark.org/t/djot-a-light-markup-language-by-jgm/4229/8 "2022-07-25T21:26:44Z")
> [@jgm](#): > > Second, genuine hard line breaks inside a paragraph are extremely rare in good typography; it seems a shame to assign this rare thing to the newline character. Depends on use case, but not everything people write is destined for print with “good typography”. Most of it stays online, in casual discussions. Now here is another “fork”: Some people, I have no idea how many, routinely break lines after sentences or short groups of sentences, _with the intent of making the structure of their writing clearer to the reader_. I do. We embrace the structure of plain text to have 2 levels of grouping: long lines that are effectively “sub-paragraphs”, and “real paragraphs” separated by blank lines. (In markdown, I find myself adding double spaces after most line breaks. Rarely do I hard-wrap _for myself_ without wanting the reader to also see the break.) It’s rare in print and definitely violates writing style guides (unless you write poetry). Possibly because paper pages has a cost and a weight? But online I’ve already seen it in some people’s emails in the 90s (can’t speak to before my time 🤷) and AFAICT it’s not going away. > [@jgm](#): > > Diffability is another important consideration: if you have extremely long lines, it is harder to see where changes have occurred inside a paragraph. This consideration goes both ways… - All good diff tools like github/gitlab/meld/delta do 2-level diffing where the whole touched line is highlighted, but then the specific touched words are highlighted stronger within it. So writing each paragraph as one long line is not catastrophic to modern diffing. 😐 - Automated hard-wrapping around a fixed width leads to the worst diffs! When words slide from one line to another, they add tons of noise to the diff that’s hard to filter out. 👎 - OTOH the best advice for minimizing diff noise is indeed “add newline after every sentence”, when working with formats like LaTeX, markdown etc. that’ll still render them as single paragraph. 👍 ---
**Author:** ![cben](https://cdn.commonmark.org/user_avatar/talk.commonmark.org/cben/32/407_2.png) [@cben](https://talk.commonmark.org/u/cben)\ **Post date:** [July 25, 2022, 9:51pm UTC](https://talk.commonmark.org/t/djot-a-light-markup-language-by-jgm/4229/9 "2022-07-25T21:51:59Z")
It’s a good oportunity to mention the little-known `text/plain; format=flowed` [RFC 3676: The Text/Plain Format and DelSp Parameters](https://www.rfc-editor.org/rfc/rfc3676). A brilliant ASCII hack IMHO, and I regret it hasn’t caught on among unix terminal tools — or even among mail readers where it was intended ☹ Hard-wrapping is a lossy action — it’s hard to recognize which newlines are “original” semantic breaks and which were inserted “soft breaks” and safe to remove. The idea of `format=flowed` was that by default line breaks still mean a hard break and you _opt-out_ by leaving a single trailing space before the newline. Pairs of “Space Newline” are generally removable, or at least the Newline is (a 2nd param DelSp controlled whether the space is kept, which fits e.g. English, or dropped, which fits languages like Chinese that don’t use spaces between words). That’s the opposite of markdown, where single source newlines by default are ignored as soft breaks, and 2 trailing spaces _opt-in_ to a hard break. This choice was made for compatibility with existing ambiguous plain text (and with email clients ignorant of format=flawed), where you can’t be sure which breaks were soft. It’s safer to preserve all breaks than munge all lines together. =\> The particular choice of “Space Newline” is debatable, but consider the idea of opt-in vs opt-out. Whether a markup format _defaults_ to ignoring newlines or treating them as semantic line breaks, either way it can _allow_ hard-wrapping without _requiring_ it! ---
**Author:** ![vas](https://cdn.commonmark.org/user_avatar/talk.commonmark.org/vas/32/1267_2.png) [@vas](https://talk.commonmark.org/u/vas)\ **Post date:** [July 26, 2022, 6:03pm UTC](https://talk.commonmark.org/t/djot-a-light-markup-language-by-jgm/4229/10 "2022-07-26T18:03:43Z")
@jgm, I wanted to hear other points of view before I chimed in again, but since it’s been a couple of weeks… I spoke of manual line wrapping being a primordial fork in the road. The real primordial fork is this choice: _Should machines be made to understand humans or should humans be made to make themselves understandable to machines?_ It is this question that drives my analysis. ```plaintext machine human oriented oriented <------------------------------------------------> SGML/HTML Asciidoc Markdown ``` Where does djot fall? For some of its choices, a little to the right of Markdown. But in doubling down on manual line wrapping, to the left, which the examples in my above reply try to demonstrate. > [@jgm](#): > > I have to say that I don’t really agree with your readability judgments; it seems to me that the extra whitespace is desirable (and certainly not undesirable) in these cases. The notion of “tight lists” tells us that extra whitespace _can_ be undesirable. More importantly, requiring it has only one purpose: making humans make themselves understandable to the machine. We would not be allowed to write the following even though it is both unambiguous and natural for us: ```plaintext Before leaving: - eat all the perishables - water the plants - turn out the lights ``` And we would only know that we were not allowed to write it this way because we’d have to learn a set of rules designed to make ourselves understandable to machines. > [@jgm](#): > > First, if your proposal is that a hard line break should be interpreted as a hard break (i.e. `
`), then there would _still_ be an ambiguity with > > ```auto > She counted (all the way from 1 to > 9) and then went in. > > ``` > > Should this be parsed as a single paragraph containing a hard line break and the literal `9)`, or a paragraph followed by an ordered list? Here you’d need something like commonmark’s current kludge… I always thought CommonMark’s "kludge’ was a step in the right direction (and defended it more than once on this forum): Make the machine figure out the human. I’d want to go further, adding a heuristic such as: _Is there an `8)` list item somewhere before this? Is there an `10)` item that follows? If not, this is unlikely to be a list item_. If you eliminate manual line wrapping (treating newlines as soft breaks), then it gets considerably better. First, a large source of ambiguity, what @cben calls “Hard-wrapping is a lossy action”, goes away. There simply would be no line break before `9)`. We can come up with other examples where ambiguity occurs, but they become increasingly contrived. In other words, incredibly rare in the real world. Second, you could modify the heuristic: _If unsure, leave it as-is_. Even if it were a list item, it will still be seen as such by the human reader even without an explicit list rendering (in HTML or whatever), because it was seen as such in the original. On your second point: Without any rules, humans will naturally delineate paragraphs with blank lines where necessary. On your third: Thank you for making me aware of RFC-5322. At some point we have to break the bonds out outmoded limitations (limitations I assume exist because at the time computing resources were so limited choices were made in favor of machines at the expense of humans). Even the RFC says “Receiving implementations would do well to handle an arbitrarily large number of characters in a line for robustness sake.” But more importantly, who writes in plain text markup in an email client that won’t render that Markdown into HTML? Finally, say someone wants/needs to email raw plain text markup. The software would insert newlines where necessary. The occasional \> 998 character paragraph (this one is 780 chars), would get an odd line break in the middle. It’s not a big deal. I’m with @cben on the diffability question. Djot doesn’t require hard-wrapping, but even users who don’t use it pay its costs. ---
**Author:** ![jgm](https://cdn.commonmark.org/user_avatar/talk.commonmark.org/jgm/32/1360_2.png) [@jgm](https://talk.commonmark.org/u/jgm)\ **Post date:** [July 26, 2022, 9:59pm UTC](https://talk.commonmark.org/t/djot-a-light-markup-language-by-jgm/4229/11 "2022-07-26T21:59:27Z")
[I sent a reply by email, but for some reason it didn’t come through. So, apologies if this is a duplicate.] > On Jul 26, 2022, at 11:13 AM, vas via CommonMark Discussion \<[noreply@talk.commonmark.org](mailto:noreply@talk.commonmark.org)\> wrote: > > If you eliminate manual line wrapping (treating newlines as soft breaks), then it gets considerably better. First, a large source of ambiguity, what @cben calls “Hard-wrapping is a lossy action”, goes away. There simply would be no line break before 9). I don’t understand your response to my point. Most likely we are understanding some key terms, like “soft break,” differently. As I was using the term here, a “soft break” is a newline in the markdown source that is interpreted semantically as a space. Djot _does_ “treat newlines as soft breaks” in this sense, so I assume you mean something different. I understood you to be proposing that newlines in paragraphs would produce a hard line break in the rendered output. Assuming that’s right, then you face an ambiguity in ```auto She counted (all the way from 1 to 9) and then went in. ``` Is this a single paragraph with a hard break, ```auto

She counted (all the way from 1 to
9) and then went in.

``` or is it a paragraph followed by a list? ```auto

She counted (all the way from 1 to

  1. and then went in
``` That’s the ambiguity. It’s just the same as the ambiguity we face now with commonmark, and which we resolve with the unprincipled restriction on start numbers. I guess you like that way of resolving it, but I’ve never been happy with it. In any case, it comes up whether or not you allow hard wrapping (i.e., treat newlines in paragraphs as equivalent to spaces). As for your main point: certainly, djot falls a bit to the left of markdown. With markdown and commonmark, the aim was to magically guess what humans intended, as far as possible. My mantra was always “favor what is intuitive to humans, and make the parser more complex if necessary.” That is why the parsing rules are so complex. The problem is, even with all this complexity, there are many cases where we don’t get the results people intuitively intend. So, my choice with djot is to give up trying. Performing the task well would require a high degree of general intelligence. Maybe in the future, some successor of GPT-3 will be used to parse our plain text documents, but for now, I’d rather have a simple set of rules that we can keep in our heads, so the output is predictable. To elaborate: in my example above, any human can tell that the `9)` on the second line is the end of the parenthesized phrase in the preceding line, and not the start of a list, whereas in ```auto A further point is 9) blah blah blah ``` we have a list. But in doing this we’re relying on our grasp of the meaning of what is written; it’s very hard to predict human intentions in such cases with a set of syntactic rules. Consider the minimal pair: ```auto A more interesting number is 6. This is a "perfect" number. ``` vs ```auto A more interesting point is 6. This is a "perfect" number. ``` In the second case a list is probably intended (depending on preceding content), while in the first a list is not intended. We can figure that out because we know the difference between “point” and “number” and we have a grip on what the writer is trying to achieve. Our markdown parsers can’t learn this by being given more and more complex rules. They’d need to have a psychological model of the humans writing the text, and an understandig of the meanings of the words. The idea behind djot is to keep things simple, uniform and predictable while still achieving most of the aims of markdown. John ---
**Author:** ![jgm](https://cdn.commonmark.org/user_avatar/talk.commonmark.org/jgm/32/1360_2.png) [@jgm](https://talk.commonmark.org/u/jgm)\ **Post date:** [July 27, 2022, 5:06pm UTC](https://talk.commonmark.org/t/djot-a-light-markup-language-by-jgm/4229/12 "2022-07-27T17:06:30Z")
On diffability: I think the best approach for diffability is one sentence per line. However, that speaks in favor of djot’s current approach, on which newlines inside paragraphs render as spaces rather than hard breaks. The bottom line, though, is that I don’t want to impose any one style on the user. Some people like one sentence per line. Some people like hard wrapping to a fixed width. Some people like one big line per paragraph. All of these styles work fine with djot. I’d be reluctant to change djot in a way that _requires_ one of these styles and excludes the others. ---
**Author:** ![dmr](https://cdn.commonmark.org/user_avatar/talk.commonmark.org/dmr/32/2588_2.png) [@dmr](https://talk.commonmark.org/u/dmr)\ **Post date:** [August 6, 2022, 10:38pm UTC](https://talk.commonmark.org/t/djot-a-light-markup-language-by-jgm/4229/13 "2022-08-06T22:38:49Z")
The most obvious criticism I have is that > 1. It should be possible to attach arbitrary attributes to any element. Is not a proper design goal. It’s a design _decision_ made to achieve some design goal. A design goal is something like “It should be easy to implement the parser” or “It should be possible to add html blink tags.” Now I’m very dubious on attaching arbitrary attributes as a design choice, but it would be a mistake for me to argue against it since there’s no actual design goal. I don’t know what you’re trying to accomplish. And I need to know to be able to suggest alternatives. Part of what I’m thinking at the moment is that I don’t like the idea of sprinkling bits of markup (in the form of attribute specifications) all over the text, since this directly conflicts with markdown’s goal of being readable as text. An alternative might be to actually standardize the “Front Matter” convention you see in CMS’s that use markdown (e.g. Hugo), where you have ```auto ----- title: My stupid clickbait blog post date: 2022-8-6 tags: - seo - clickbait ----- ``` Then maybe you could define one or more attributes to contain queries (maybe css selectors) to locate block/inline elements and specify attributes to be applied to them. In this way you contain all the ugliness to the front matter. Now this is probably a terrible idea that you shouldn’t use, but without a proper design goal it’s hard to actually demonstrate this. For example, this approach would probably make the parser easier to implement (assuming parsing/evaluating the css selectors is handled by an existing library), and it satisfies the goal of allowing us to apply arbitrary attributes to elements. Or to make this even more pathological, we specify a series of `sed` programs to be run on the finished html, so we don’t have to worry about having a css library. This appears to satisfy the design goals even better. I just threw up in my mouth a little. Let’s switch gears and see if I can come up with some suggestions that are actually worthwhile. One of the changes you made was to cut out redundant syntax elements. Personally, I have a strong preference for the setext headings and would miss them if they were gone, but let’s leave that aside briefly. One could argue that if you’re going to start trimming down the language, there’s no reason to include the inline link definition (`[link text](uri)`) when the reference link definition (`[link text]...[link text]: uri`) is obviously so much better. Assuming for the sake of argument that we agree with me about reference links, we could apply the same reasoning to attributes. Instead of writing ```auto The word *atelier*{weight="600"} is French. ``` (Yuck!) We write e.g. ```auto The word *atelier*{fw} is French. {fw}: weight="600" ``` In both cases, we satisfy the goal of being able to add non-text stuff (links, arbitrary html attributes) to our text without substantially compromising the readability of the text. And by banning the obviously-inferior inline versions, we simplify the parser. Also, we only define `fw` once, so we can re-use the same 4-char reference several times in case we add more french words to our document. The disadvantage of this is that we can’t straightforwardly compose the two definitions, as in ```auto [Link text](url){title="Click me!"} ``` But we could still presumably compose the invocations and write ```auto [Link text]{cmt} {cmt}: title="Click Me" [Link text]: url ``` ---
**Author:** ![dmr](https://cdn.commonmark.org/user_avatar/talk.commonmark.org/dmr/32/2588_2.png) [@dmr](https://talk.commonmark.org/u/dmr)\ **Post date:** [August 6, 2022, 10:55pm UTC](https://talk.commonmark.org/t/djot-a-light-markup-language-by-jgm/4229/14 "2022-08-06T22:55:27Z")
in re the setext headings: I want the way I write the h1 elt to look like a big title, and the way I write the h2 elt to look like a substantially-document-dividing subtitle. I don’t think that the atx headings qualify. Ignoring post-title hashes does at least let me write ```auto # Title ################################## intro ## Subsection 0 ######### text ## Subsection 1 ########## more text ``` Which isn’t bad, but I’m not a huge fan. This maybe isn’t a critical issue, but I still don’t get the same warm fuzzy feelings I get from markdown setext headings. An alternative might be: If the heading contains a trailing `#`, The line immediately after the heading is skipped. Then I write ```auto # Title # ============================= intro ## Sub 0 # ------------------------------------------------ text ## Sub 1 # ------------------------------------------------- text ``` Now this is almost as nice in plain text as markdown (the extra hashes aren’t a big deal), and we don’t actually have to make the parser understand the setext headings, since they’re basically just comments. In fact, I can use `$$$$$$$$$$$$$$$$$$$$$$$$$$$` as my h3 separator if I want. ---
**Author:** ![dmr](https://cdn.commonmark.org/user_avatar/talk.commonmark.org/dmr/32/2588_2.png) [@dmr](https://talk.commonmark.org/u/dmr)\ **Post date:** [August 6, 2022, 11:00pm UTC](https://talk.commonmark.org/t/djot-a-light-markup-language-by-jgm/4229/15 "2022-08-06T23:00:50Z")
Finally, I agree with @jgm on the hard-wrap support thing, but I really don’t feel like saying anything substantial about it. ---
**Author:** ![jgm](https://cdn.commonmark.org/user_avatar/talk.commonmark.org/jgm/32/1360_2.png) [@jgm](https://talk.commonmark.org/u/jgm)\ **Post date:** [August 6, 2022, 11:10pm UTC](https://talk.commonmark.org/t/djot-a-light-markup-language-by-jgm/4229/16 "2022-08-06T23:10:06Z")
> [@dmr](#): > > Now I’m very dubious on attaching arbitrary attributes as a design choice, but it would be a mistake for me to argue against it since there’s no actual design goal. I don’t know what you’re trying to accomplish. And I need to know to be able to suggest alternatives. The primary goal is to provide a flexible way to make the markup extensible. Suppose you are writing a document that needs to index certain terms. What you want is a _filter_: a program that reads the abstract produced by the parser and enhances it by adding an index. (If you’re not familiar with this in practice, go look at some examples of pandoc filters.) But in order for this to work, the filter needs to know which terms are to be put into the index, so this information needs to be represented in the AST. If we can add attributes to bits of text, we can do that using attributes. e.g., `[cat]{.index}`, or `[cat]{.index see="feline"}`. That’s just one application. Instead of marking individual words and phrases, you might want to mark groups of paragraphs. Then you attach the attributes to a fenced div. Or you might want to mark a code block as content that should be turned into a diagram via mermaid, using certain DPI settings or whatever. Again, you can do this with attributes. So, **attributes provide hooks through which external filters can interact with the parsed AST.** That’s the main reason I want them, anyway. They’re also useful for things like defining anchors for cross-references. ---
**Author:** ![dmr](https://cdn.commonmark.org/user_avatar/talk.commonmark.org/dmr/32/2588_2.png) [@dmr](https://talk.commonmark.org/u/dmr)\ **Post date:** [August 6, 2022, 11:40pm UTC](https://talk.commonmark.org/t/djot-a-light-markup-language-by-jgm/4229/17 "2022-08-06T23:40:49Z")
Okay, that makes sense. Looking at your example makes me wonder if I want to augment my attribute-ref proposal to make the attribute specs composable, as in ```auto [cat]{cat} and [dog]{dog} {idx}: .index {cat}: {idx} see="feline" {dog}: {idx} see="canine" ``` Or whether that introduces too much complexity. ---
**Author:** ![yne](https://cdn.commonmark.org/user_avatar/talk.commonmark.org/yne/32/2646_2.png) [@yne](https://talk.commonmark.org/u/yne)\ **Post date:** [October 31, 2022, 3:52pm UTC](https://talk.commonmark.org/t/djot-a-light-markup-language-by-jgm/4229/18 "2022-10-31T15:52:24Z")
I discovered `djot` while writing (and failing twice) a _simple_ commonmark parser, and I must say it’s a relieve to find another project with the same kind of [conclusions](https://github.com/yne/md2dom/wiki/Lesson-learned-while-writing-a-markdown-parser). While I appreciate it strictness (especially on the tables) and _some_ of the new inline syntax (ins/del/sup/sub) that add value for a low complexity cost, It sometimes feel like a want-to-fit-all-profile: - parser : with it simpler/safer syntax - publisher : with it math, smart punctuation … - developer : with it div/span and raw code injection But I’m worried by the following aspects: - Having a first class LaTeX Math support mean that any djot render shall comes with a LaTeX Math renderer library. - Is the ~2000 lines [emoji-to-unicode list](https://github.com/jgm/djot/blob/main/djot/emoji.lua) part of the `djot` standard that all parser shall follow and bundle with them ? - it sometimes add (too) much of choices that bring corner cases: a `i.` can be either `
    ` and `
      ` while it was supposed to eliminate them - having both `~~~` and ``` for code block contradict Rationale 11 - the span/div attribute is a mess to parse with regex, sometimes it’s before it element (block) and sometimes it’s after (inline). Needless to thay that giving programming ability with those attributes or with the `=html` native code injection will prevent this language from being used on any user-given content (forum/chat/readme) - A developer writing it README will get upset by the smart punctuation substituting all it dash and quote, but a PhD or a blogger will probably be happy about it. - I’m not enough of djot master to understand why backtick mess the inline parsing so much: it interrupt when in a middle of an _em_ or _strong_ (I thought the first opener win) so - optional fun fact: the `\` syntax does the opposite of what all programming language does: they create a continuation line I started writing a djot2dom JS render, and it feels simpler to parse, yet all those new corner case and _heavy_ features made me lost my motivation (again). I think I’ll continue my journey in the search for a stricter GFM subset 🙂 ---
      **Author:** ![jgm](https://cdn.commonmark.org/user_avatar/talk.commonmark.org/jgm/32/1360_2.png) [@jgm](https://talk.commonmark.org/u/jgm)\ **Post date:** [October 31, 2022, 6:58pm UTC](https://talk.commonmark.org/t/djot-a-light-markup-language-by-jgm/4229/19 "2022-10-31T18:58:25Z")
      Math support is not a big issue. You can always pass the raw TeX math through verbatim (it is usually still fairly readable). In HTML, you just need this plus one line to pull in MathJax from a CDN. Emojis: well, the standard is vague on what is required to conform for emojis, but I don’t know how important this is. I’m still not sure about having both forms for code blocks; I take the point about Rationale 11. Worries about code injection: the parser will recognize raw HTML and also attributes. How these are rendered is up to the renderer, which could decide to omit all raw HTML and potentially dangerous attributes, or sanitize them somehow. Alternatively, pass the result of a naive render through a standard sanitizing library. Nothing here prevents you from using djot with untrusted input. Yes, unclosed backticks are implicitly closed by the end of the paragraph. This avoids the need for expensive backtracking, and in practice I don’t think it’s a problem. By the way, if you want to write a parser, I think your best approach would be to convert the existing Lua parser to JavaScript. Mostly this should be straightforward, since Lua is quite similar to JavaScript. ---
      **Author:** ![yne](https://cdn.commonmark.org/user_avatar/talk.commonmark.org/yne/32/2646_2.png) [@yne](https://talk.commonmark.org/u/yne)\ **Post date:** [November 1, 2022, 5:03am UTC](https://talk.commonmark.org/t/djot-a-light-markup-language-by-jgm/4229/20 "2022-11-01T05:03:22Z")
      Thanks, I think I got the idea : the spec is more about the parsing than about the rendering. For the code block : I forgot to say that I loved the idea of a variable number of backtick to handle encapsulation. For the parser: my goal is to keep it easy hackable (so in a \<200LoC range) while taking advantage of the browser DOM API to directly build the Elements tree without the need for an IR/AST. So not really the same goal 🙂 ---
      **Author:** ![vas](https://cdn.commonmark.org/user_avatar/talk.commonmark.org/vas/32/1267_2.png) [@vas](https://talk.commonmark.org/u/vas)\ **Post date:** [December 5, 2022, 11:22pm UTC](https://talk.commonmark.org/t/djot-a-light-markup-language-by-jgm/4229/21 "2022-12-05T23:22:09Z")
      My post of djot on Hacker News made it to the front page and has sparked a growing discussion over there: [Djot: A light markup language by the creator of Pandoc and CommonMark | Hacker News](https://news.ycombinator.com/item?id=33867636) ---
      **Author:** ![vas](https://cdn.commonmark.org/user_avatar/talk.commonmark.org/vas/32/1267_2.png) [@vas](https://talk.commonmark.org/u/vas)\ **Post date:** [December 20, 2022, 10:49pm UTC](https://talk.commonmark.org/t/djot-a-light-markup-language-by-jgm/4229/22 "2022-12-20T22:49:28Z")
      > [@jgm](#): > > I don’t understand your response to my point. Most likely we are understanding some key terms, like “soft break,” differently. As I was using the term here, a “soft break” is a newline in the markdown source that is interpreted semantically as a space. Djot _does_ “treat newlines as soft breaks” in this sense, so I assume you mean something different. I just noticed in referencing you answer here in a GitHub comment that I never answered you (Yes, we are using terms differently. No, my proposal would treat the example you give as a a single paragraph with a hard break, _not_ what either Markdown or djot does). I think at the time I figured you didn’t really need my response because your answer made clear your coherent philosophy for djot and why my thoughts on what I call “manual line breaks” didn’t make sense for it. Let me know if you actually want my explanation. [Next page](https://talk.commonmark.org/t/djot-a-light-markup-language-by-jgm/4229.md?page=2)