I’m playing with some ideas for a domain-specific language. The programs written in it are likely to be small but fairly dense so I was hoping to make it easy to document things and to keep documentation in sync with code.
I thought that I might have the source files themselves be markdown so there wouldn’t be a syntax for comments per se. Instead, if my language were called myDSL I might write
# Hello, World!
Now once more with feeling, programmatically
```myDSL
print("Hello, World!")
```
The code block with the language tag would indicate to the lexer that it needs to stop ignoring comments and produce some tokens for the compiler.
Literate programming provides examples of how to extract code from documentation and stitch it into a coherent whole before annoying the compiler. I did a search for “literate markdown” and there is some prior art for this idea: ycomb, fadado/literate and others. Even after looking at those though, I had a few concerns that I’m still fuzzy on.
- We know how to associate delimited comments with declarations; javadoc comments precede the declaration. Finding the text associated with a specific declaration so we can supply IDEs with tooltips might be harder with this approach.
- Often, some files in a project need to be identified as main files where execution starts. These are typically done with a shebang like
#!/bin/rundownwhen compilation isn’t mandatory. Is that compatible with markdown? It seems like it might conflict with the level-1 header. - Most libraries include some symbolic constants defined in code, and referred to from both code and documentation. It’s hard from markdown to refer to specific lines in a code block, or to identify a code block. Maybe there’s a markdown feature we could piggyback on to allow symbolic constants to be defined in the documentation and referred to from code:
could define data available to from both documentation and code. The trend in recent languages seems to be away from strictly typed constants (see Go’s arbitrary precision and untyped constants, and Java’s use of polyexpressions for lambdas and method references). The lack of a nice way to declare a type for symbolic constants declared this way may not be a problem. Any way to associate an anchor with a[my_symbolic_constant]: data:text/plain,"A constant value"```JSON…```block or table might allow code to refer to lookup tables and default configurations defined in documentation. - Many programming languages hava a mechanism to make symbols available to code another source file:
import,include,load,require. Markdown documentation can link to other markdown files via[_](_)syntax. Piggybacking on that syntax would require changing human-visible documentation to conform to the needs of code. Named links might help again
[a_local_dependency]: ../foo/bar.my-dsl.md [a_remote_depenceny]: https://github.com/releases/1.0.0/baz/boo.md ```myDSL use x from [a_local_dependency]; use * from [a_remote_dependency]; ```
- Often the best way to document an API is with examples. Whenever I write usage examples I try to write tests to avoid embarrassing myself. I fail often enough that it’d be nice to extract usage examples and run them with tests a la updoc. So it seems that code in documentation might have at least two flavors: production and test/example code. Is there a convention I could piggyback on to make it obvious to even a reader who has not read the language spec which code snippets are test code and which are production code?
- Sometimes one language will embed another. For example, C can embed inline assembly. If the lexer needs to route snippets of test and production code differently, and possibly extract snippets of code in 2 or more languages, should there be some unambiguous signal to the lexer that this is a code block to extract? Is there one that doesn’t break code pretty-printing when viewed as markdown?
- Good documentation should explain how abstractions can be combined to solve a class of problems. Having all the gory details in the documentation may distract from that. C has a split (module pragmas like inline) between headers that declare what’s available and
.cfiles that implement. Recent languages have almost uniformly moved away from that model for, I assume, good reasons. A linter that discourage large blocks of code in modules that are within a certain link depth from README might help teams structure code to be easily navigable. Alternatively, code blocks that are collapsed by default when viewed in markdown might help.
I’m sure there’re many other issues that I haven’t considered. What do people think? Is embedding code in documentation a terrible idea? Has this been done on a scale that lessons have been learned? What are other issues to consider? Are there other features of markdown that are relevant?
If you’ve read this far, thanks,
mike