Continuing the discussion from Djot: A light markup language by @jgm , knowingly acting out xkcd 386 “Duty Calls” :
I think the members of this forum spend too much time talking about power-user features like ease-of-implementation, parsing speed, advanced formatting features, and terse syntax. I don’t want to lock power users out of the ecosystem entirely (after all, I am one), but I don’t think it makes sense to elevate power user needs above the needs of non-power-users. By definition, we can take care of ourselves even when conditions are less than perfect.
The main “non-power-user” problem I see in Markdown is a trickle stream of incoming “bug reports” and help questions from people who think rustdoc corrupted their docs, because they accidentally triggered a feature that they didn’t intend to use. They just want to write . Many people who just want to write are good writers with valuable things to say. Some of the best works ever written use nothing more complex than paragraphs and headers.
It’s too easy to accidentally write refdefs
Can’t find a better place to ask this, so apologiez in advance if there was a better place for discussing this.
I use Markdown in Javadoc comments, because Java 23 implemented support for Markdown in them using a new syntax.
Now, they mention on their docs page, that they support the CommonMark Spec for the overall markdown syntax… However, it’s not mentioned what tool/parser they actively use for converting the text into the final HTML, but my current guess is commonmark.js, as I find the sam…
opened 12:25AM - 18 Nov 24 UTC
closed 02:01PM - 16 Feb 25 UTC
T-rustdoc
A-intra-doc-links
C-external-bug
I tried this code (minimal reproduction `lib.rs`):
```rust
/// - [`SDL_PROP_WIN… DOW_CREATE_COCOA_WINDOW_POINTER`]: the
/// `(__unsafe_unretained)` NSWindow associated with the window, if you want
/// to wrap an existing window.
/// - [`SDL_PROP_WINDOW_CREATE_COCOA_VIEW_POINTER`]: the `(__unsafe_unretained)`
/// NSView associated with the window, defaults to `[window contentView]`
pub fn a() {}
/// - [`SDL_PROP_RENDERER_MAX_TEXTURE_SIZE_NUMBER`]: the maximum texture width
/// and height
/// - [`SDL_PROP_RENDERER_TEXTURE_FORMATS_POINTER`]: a (const [`SDL_PixelFormat`] *)
/// array of pixel formats, terminated with [`SDL_PIXELFORMAT_UNKNOWN`],
/// representing the available texture formats for this renderer.
pub fn b() {}
pub const SDL_PROP_WINDOW_CREATE_COCOA_WINDOW_POINTER: () = ();
pub const SDL_PROP_WINDOW_CREATE_COCOA_VIEW_POINTER: () = ();
pub const SDL_PROP_RENDERER_MAX_TEXTURE_SIZE_NUMBER: () = ();
pub const SDL_PROP_RENDERER_TEXTURE_FORMATS_POINTER: () = ();
pub const SDL_PIXELFORMAT_UNKNOWN: () = ();
#[allow(non_camel_case_types)]
pub type SDL_PixelFormat = ();
```
When I run `cargo doc` on this, there are no warnings or errors, but the output is wrong. The first line of the first list item of `a` is missing, and the first line of the second list item of `b` is also missing.


If I run `cargo clippy` on the file it complains:
```
warning: doc list item without indentation
--> src/lib.rs:3:5
|
3 | /// to wrap an existing window.
| ^^
|
= help: if this is supposed to be its own paragraph, add a blank line
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#doc_lazy_continuation
= note: `#[warn(clippy::doc_lazy_continuation)]` on by default
help: indent this line
|
3 | /// to wrap an existing window.
| +++++++++++++++++++++++++++++++++++++++++++++++++++++++
warning: doc list item without indentation
--> src/lib.rs:12:5
|
12 | /// representing the available texture formats for this renderer.
| ^^
|
= help: if this is supposed to be its own paragraph, add a blank line
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#doc_lazy_continuation
help: indent this line
|
12 | /// representing the available texture formats for this renderer.
| +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
warning: `rustdocbug` (lib) generated 2 warnings
```
But this makes no sense. Those lines have the correct indentation, same as the previous lines. It seems to think there's a new sublist getting created at the end of the first lines, but I don't see why
### Meta
`rustc --version --verbose`:
```
% rustc --version --verbose
rustc 1.82.0 (f6e511eec 2024-10-15)
binary: rustc
commit-hash: f6e511eec7342f59a25f7c0534f1dbea00d01b14
commit-date: 2024-10-15
host: aarch64-apple-darwin
release: 1.82.0
LLVM version: 19.1.1
% rustc +nightly --version --verbose
rustc 1.84.0-nightly (798fb83f7 2024-10-16)
binary: rustc
commit-hash: 798fb83f7d24e31b16acca113496f39ff168c143
commit-date: 2024-10-16
host: aarch64-apple-darwin
release: 1.84.0-nightly
LLVM version: 19.1.1
```
The short version is that you can too-easily write a link reference definition, usually wrapped in a bulleted list:
- [to prevent this]: verbosity (it should not be this easy to write text that doesn't appear in your document at all)
And that line will completely vanish:
This doesn’t just happen in issue trackers; people really do accidentally write this. You can infer how common it is with a search for clippy::doc_nested_refdefs .
For example, compare this docs.rs page with its source code .
It’s too easy to accidentally write HTML
opened 01:42AM - 24 Jun 20 UTC
closed 02:06PM - 26 Jun 20 UTC
A-docs
C-bug
Documentation generated for
```rust
/// trait X<S> { fn f<T>(); }
/// impl<U… > X<U> for U { fn f<V>() {} }
```
and any code after is all strikethrough because of the `<S>`.
Generated doc: [Here](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.List.html#impl-HashStable%3CStableHashingContext%3C%27a%3E%3E)
Associated src code: [Here](https://github.com/rust-lang/rust/blob/04e589ced8a48818f93d6daff94a4f85e3b07271/src/librustc_middle/ty/subst.rs#L339)
cc: @jyn514
This issue has been assigned to @jyn514 via [this comment](https://github.com/rust-lang/rust/issues/73676#issuecomment-649899453).
Many cases of this are Rust-specific, since we write generics in angle brackets, so anyone who doesn’t wrap it in code spans winds up writing Vec<T> and getting Vec. Our lint for this is rustdoc::invalid_html_tags , but you can also hit that lint by accidentally writing a block quote.
For example, compare this docs.rs page with its source code .
It’s too easy to accidentally write block quotes
master ← notriddle:notriddle/bump-pulldown-cmark
Roughly categorizing what I'm seeing here. The specific crates are in [this Gist… ](https://gist.github.com/notriddle/8d1859a17e606b004e91d57fee6d7f15):
## (P)roblems: docs that are broken by this change
### P1: unintended strikethrough
https://github.com/pulldown-cmark/pulldown-cmark/pull/648 allows `abso~fricking~lutely` to render as abso~fricking~lutely. It makes pulldown-cmark agree with cmark-gfm.
This change causes **39** docs to have strikethroughs where it looks like the author intended literal tildes.
### P2: Unintended block quote
https://github.com/pulldown-cmark/pulldown-cmark/pull/675 makes block quotes, introduced by `>`, consistent between paragraph interruption and starting after a blank line. It's pretty unambiguous that this is a bug fix.
Causes **24** crates to appear wrong.
### P3: Footnote reference immediately after link or another footnote
https://github.com/pulldown-cmark/pulldown-cmark/pull/773
The way that pulldown-cmark ignores the apparent footnote reference in `[something][^foot]` is the same way GFM does it, and it's the easiest way to parse this given everything else, but it's weird.
Causes **16** crates to appear wrong.
### P4: block nested inside footnote definition without indenting
https://github.com/pulldown-cmark/pulldown-cmark/pull/654
This is, basically, the expected outcome of GFM-compatible footnote parsing. The syntax now parses the same way GitHub does, which is not the way rustdoc used to.
Causes **6** crates to appear wrong.
### P5: emphasis does not match what author expects
In a text like `__imp_(_)`, commonmark rules say that the result should be __imp_(_), but I'm pretty sure the author intended those to be literal underscores. In the old parser, they were.
Causes **2** crates to appear wrong.
### P6: Single `|` does not continue table
This isn't what GitHub does, and the new parser aligns closer with GitHub, but some authors have used this as a way to put dividers in their table:
```markdown
| header | two |
|--------|-----|
| item | n |
| item | e |
|
| section | l |
| two | o |
```
Causes **6** crates to appear wrong.
### P7: Table is required to have a valid header line
This is not a table according to GitHub, but the old parser would render it as one sometimes.
```
| first | second |
| third | fourth |
| fifth | sixth |
```
The header line also needs to have at least one hyphen in each cell, so this isn't allowed either.
```
| first ||
|--------||
| second ||
```
Causes **4** crates to appear wrong.
### P8: unintended link definition
This is a link definition in the new parser, but not the old one.
```markdown
[Self::method()]:
frobnicates
```
Causes **1** crate to appear wrong.
## (F)ixes: docs that are actually render better with the new parser than the old one
### F1: `[^x]` in the old parser rendered a broken footnote link when no footnote was intended or defined
https://github.com/pulldown-cmark/pulldown-cmark/pull/654
Usually, they're trying to write regex inverted character classes, but they come out as footnotes.
This fix also makes things consistent with GitHub, and the fact that it's not breaking anyone's docs makes me happy.
Fixes **39** crates
### F2: writing `1.` alone on a line in the middle of a paragraph shouldn't start a list
https://github.com/pulldown-cmark/pulldown-cmark/pull/681
It used to do the wrong thing with this:
```
Test paragraph with a count of
1.
```
Fixes **6** crates
### F3: tables interrupt paragraphs
https://github.com/pulldown-cmark/pulldown-cmark/pull/653
This makes things consistent with GitHub, and fixes **NN** docs. Not likely to be a problem, because you kinda have to try really hard to make something look like a table.
Fixes **37** crates
### F4: ASCII art misinterpreted as list
Since asterisks on their own no longer count as lists when they interrupt paragraphs, some things that were never intended as list markers stop being seen as such.
Fixes **1** crate
### F5: footnote has nested, indented children or lazy continuation
This is the flip side of P4, where the author actually *wanted* it to be parsed the way github parses it.
Fixes **2** crates
### F6: Link definition is seen in new parser where old parser did not
The old parser didn't properly recognize link reference definitions when they were right after code blocks.
Fixes **1** crates
### F7: block structure and inline structure are separated better in new parser
Consider this example:
```markdown
> [my link](https://example.com "Example web site
> run by the IETF")
```
The `>` on the second line shouldn't go in there, and now it doesn't.'
Fixes **2** crates
### F8: a table row followed by two spaces isn't a hard break
A very specific bug that only seems to show up when a paragraph is nested inside a list. The new parser fixes it, it only affects **1** crate, and the change makes the generated docs look better.
Fixes **2** crates
### F9: footnote with `<autolink>` was misparsed as link def
The old parser thought this was a link definition:
```
[^0]: <https://example.com>
```
The new parser sees it as a footnote, which is also how github does it.
Fixes **1** crate
### F10: indented or spaced link definition still counts
This document:
```
[first]: https://example.com
[second]: https://example.com
[first]
[second]
```
The old parser ignored the definition of `[second]`, so it only saw one link, but the new one doesn't, so it sees two links. This also agrees with the reference implementation.
The new parser also trims spaces, so `[x ]` and `[x]` are the same thing.
Fixes **2** crates
### F11: intra-word strikethrough
This is the flip side of P1, where the author apparently intended to write intra-word strikeout.
Fixes **1** crate
## (Q)uestionable cases, where the docs were broken before and are still broken now
### Q1: crate author wrote ASCII art, or some language that isn't CommonMark, in their doc comments
This happened on **N** crates, and produces terrible results in both parsers. I'm not bothering to categorize them by which change to pulldown-cmark causes them to render differently.
There are **30** crates that do this
### Q2: links with mismatched parens go from being broken links to not being links at all
https://github.com/pulldown-cmark/pulldown-cmark/pull/738
There are **4** crates that do this
### Q3: incorrect trim in block doc comment
Block comments are supposed to be written like this:
```rust
/**
* first
* second
* third
*/
^^ these two characters are trimmed
```
If you get this wrong, the text that you intended to be a paragraph gets turned into a long unordered list instead, because [trim](https://github.com/rust-lang/rust/blob/3b85d2c7fc6d1698e68b94f7bc1a5c9633f2554d/compiler/rustc_ast/src/util/comments.rs#L53) is computed by checking every line for a common prefix.
It shows up here because incorrectly trimmed block doc comments often have asterisks with no text after them, and https://github.com/pulldown-cmark/pulldown-cmark/pull/681 changes it from a list to plain text.
There are **8** crates that do this
### Q4: numbered list that starts with zero and has two ones
```markdown
0. do stuff
1.
2. do more stuff
3. do more stuff
```
The `0` isn't a valid list item in commonmark, so that first line is a paragraph. The divergence between the new and old parser is on the second line, but I'm not sure what the intended result by the crate author was? I think it's supposed to be a list starting at zero, but it doesn't work in the old version or the new one.
There is **1** crate that does this
### Q5: incorrect footnote definition markup
The parser used to return a dangling footnote reference, but the new one doesn't. Both saw the intended footnote definition as invalid.
There are **18** crates that do this
### Q6: block quotes written at start of line
Try writing block quotes like this:
```markdown
>
First paragraph
>
Second paragraph
```
I think used to work in GitHub-flavored markdown (it works in Pandoc-flavored markdown now), and seems to be what the author intended in the **1** crate that hits this case, but it doesn't work on GitHub now, and it never worked right in rustdoc (though the exact way it fails changed, which is what brought it to my attention).
There is **1** crate that does this
## (S)purious cases where the lint fires and it shouldn't
### S1: loose task list
The lint I wrote fires false positives for task lists. The event stream from the parser is different, thanks to https://github.com/pulldown-cmark/pulldown-cmark/pull/558, but they're detected in the same cases.
There are **7** crates that hit this
### S2: minor change in HTML blocks
The indentation is handled differently, but will usually generate the same rendered result.
There are **3** crates that hit this
### S3: two spaces on the last line of a paragraph
Strictly speaking, this is a different parse result. It will produce slight differences in the spacing.
There is **1** crate that hits this
### S4: spans are different when emphasis wraps code
This causes the lint to claim there's a problem when there isn't.
There is **1** crate that hits this
A weird quirk of CommonMark is that you need a space after - and * to write lists, but you don’t need a space after > to write a block quote.
-not a list
*not a list
>a blockquote
-not a list
*not a list
a blockquote
Accidental block quotes aren’t as bad as accidental HTML or accidental refdefs, since they don’t delete whole chunks of text from your document, but they’re still annoying.
In one example, you wind up writing a block quote where you intended to write the closing bracket on an HTML tag, so you get the rustdoc invalid_html_tags warning. Compare this docs.rs page with its source code .
In another example, this results in breaking out of a paragraph or a table, and you get the clippy::doc_lazy_continuation warning. Compare this docs.rs page with its source code .
It’s too easy to accidentally use lazy paragraph continuations
opened 10:05PM - 16 Jun 24 UTC
closed 09:26AM - 17 Jun 24 UTC
C-bug
I-false-positive
### Summary
A valid markdown doc comment is emitting a warning.
In this case… , I think it should check if the line is starting with `*` or `-` before emitting the lint.
### Lint Name
doc_lazy_continuation
### Reproducer
I tried this code:
```rust
/// stat
///
/// - 1: bla
/// - 6: Intel PECI
/// Not all types are supported by all chips
pub struct B;
```
I saw this happen:
```
error: doc list item missing indentation
--> src/unix/linux/component.rs:59:9
|
59 | /// Not all types are supported by all chips
```
I expected to see this happen:
Nothing
### Version
```text
rust version 1.81.0-nightly (3cf924b93 2024-06-15)
```
### Additional Labels
_No response_
When I added the clippy::doc_lazy_continuation lint, I had to fix all the places where it fired in Clippy itself . It was 6:4 accidental:deliberate lazy continuations.
For another example, compare this docs.rs page with its source code