Embedded audio and video

[v3ss0n]

Why would someone want to do that? We only doing that in client side, you can check what we did here : https://github.com/cmrd-senya/markdown-it-html5-embed

there is no server-side request there. We did that by looking at fileext and determining mime .

Yes @zzzzBov , there are problems with missing file extensions , and my idea of using lib-magic style header detection is strictly clientside (Uploading file type, Blobs etc) .

[mb21]

yeah, but even if you do it client side, I think it’s suboptimal, e.g. if the files are remote hosted (or currently not available). At least allow for an optional {type=audio} or similar.

[wdhwg001]

Why not just use the alt?

![mp3](path/to/a/mp3/file)

![](path/to/filename.mp3)

![mp4](path/to/a/mp4/file =640x480)

[mb21]

@wdhwg001

![mp3](path/to/file)

How does the parser know the user wasn’t intending to embed an image with alt="mp3"?

[Crissov]

![=mp3](path/to/file)
![mp3=](path/to/file)
![mp3:](path/to/file)

or similar signal character with resource type identifier.

![audio/mpeg;version=1;layer=2](path/to/file)

Note that I didn’t verify the actual syntax for MIME type parameters.

ASCII art could also be used (bad adhoc designs):

![o_o](path/to/video_file)
![.|`](path/to/audio_file)
![o"](path/to/image_file)

… or Unicode emoji:

• U+1F3A5
• U+1F4F7
• U+1F4F9
• U+1F3AC
• U+1F4FC
• U+1F4FA
• U+1F4C0
• U+1F3A6
• U+1F4FB
• U+1F3B5 ♩♪♫♬
• U+1F3B6
• U+1F3A4
• U+1F3BC
• U+1F3A8
  * U+1F39E
  * U+1F4FD
  * U+1F5BC :gallery:
  * U+1F58C

[mofosyne]

For missing extension name, use consistent attributes syntax

A more consistent approach to missing extension name ( missing .mp3) is to just use the currently discussed consistent attributes syntax like:

![ cat meowing ]( example.com/meow ){ mediatype=audio/mpeg }

http://talk.commonmark.org/t/consistent-attribute-syntax/272

---

Why/Purpose of type field?

Purpose of the type field. The purpose of the type field is to indicate to the renderer how to deal with missing content. E.g. If it cannot find meow.mp3, then display a missing audio icon or show ::warning: Audio File Missing:: . If no type is specified, it will try to infer by extention name or mediatype.

e.g.

![ cat meowing ]( meow.mp3 ){ type=audio }

or

!audio[ cat meowing ]( meow.mp3 )

And hey, if the viewer program is smart enough, it could have a button saying "Search online for similar audio files for cat meowing"

[v3ss0n]

I agree with Consistent attrib syntax.

[chrisalley]

While I understand that assuming .mp4 = video/mp4 may occasionally produce unexpected results, the times when this assumption is not accurate appear to be an edge cases. So perhaps consistent attribute syntax could be used to cover just the edge cases. The problem with requiring consistent attribute syntax is that it is an advanced feature and more difficult to remember for general writers. By following the convention, .mp4 = video/mp4, we allow users to simply write ![](video.mp4) and get the results that, in the majority of cases, they would expect. For edge cases they could use consistent attribute syntax to specify the content type, or raw HTML if the consistent attribute syntax extension is not enabled.

As an addendum to the proposed syntax, I suggest using the full source type (e.g. “video/mp4”) for consistency with HTML. Perhaps the order could be significant if multiple video sources are defined. So the following Markdown:

![](video.mp4 video.webm){type=video/mp4,video/webm}

Would render as:

<video>
  <source type="video/mp4" src="video.mp4">
  <source type="video/webm" src="video.webm">
</video>

There was also mention of using the contents of the square brackets for representing the media type. I disagree with this because it goes against the priniciple of uniformity (or at least the philosophy behind it). I suggest that square brackets are also used for alternative text for audio and video. E.g.

![Your browser does not support the video element.](video.mp4 video.webm)

Renders as:

<video>
  <source type="video/mp4" src="video.mp4">
  <source type="video/webm" src="video.webm">
  <p>Your browser does not support the video element.</p>
</video>

[mofosyne]

btw I thought that the [] in ![]() is for alt description for blind users etc…?

[chrisalley]

@mofosyne

btw I thought that the in is for alt description for blind users etc…?

Correct, it corresponds to an image’s alt attribute (alternative text). My reasoning is that we should keep the usage of [] in ![]() relatively consistent for other embedded media and use it for an alternative text if the media itself cannot be rendered. Another option might be to use this to specify subtitles which are a form of alternative text. E.g. this Markdown:

![subtitles.vtt](video.mp4 video.webm)

Generates an HTML track element:

<video>
  <source src="video.mp4" type="video/mp4">
  <source src="video.webm" type="video/webm">
  <track src="subtitles.vtt">
</video>

[chrisalley]

Another reason to use the ![](video.mp4) syntax for embedded resources is that Markua will also use this syntax:

https://leanpub.com/markua/read#leanpub-auto-video

Cross compatibility is a good thing, even if Markua is not aiming for full compatibility with Markdown or CommonMark. The Markua spec linked here includes a whitelist of file extensions similar to what I suggested in earlier posts. .mp4 = video, .ogg = audio, as is the convention.

[mofosyne]

We are not in disagreement. Yes, in most cases you use ![](video.mp4) but in edge cases (e.g. new video format, like webm), then ![](video.webm){type=video/webm} would be useful for when the whitelist doesn’t catch it. And if you care about being recognized as a missing video if nothing is found, then use the full form of !video[](video.webm){type=video/webm} can kick in (Which is the form that WYSIWYG commonmark editor might write as).

[vitaly]

I don’t understand why audio/video should be covered by the spec. This task can be completely automated by external preprocessor. For example, in nodeca:

1. Inline links to youtube/vimeo/… are replaced with video title
2. Links to youtube/vimeo/… on separate paragraph are replaced with video player

Rules can be extended to ANY url pattern, without spec change. So, i don’t know why audio/video files should be special case.

For node.js we did https://github.com/nodeca/embedza to automate things.

Discouse has similar ruby package, “onebox”, but AFAIK it does not supports inline links & flexible fallback (try block format if available, if not - try inline)

[chrisalley]

http://example.com/song.mp3

Does this link to the song or embed an audio player in the page? It is not clear what the author’s intention was. If we use an explicit syntax, ![](http://example.com/song.mp3) in the case of embeds and <http://example.com/song.mp3> in the case of links, both are possible to describe unambigiously.

[cmrd_senya]

@vitaly, how do I distinguish between this cases within the markdown-it plugin? This is the code I wrote some time before. It overrides “md.renderer.rules.link_open” renderer. How can I get if the link I parse is inlined or on a separate paragraph?

[vitaly]

In nodeca we do such processing on separate stage, after markdown. We just load resulting html into AST (with jquery/cheerio), and check if link/image is the only content of paragraph or not.

I see no reasons to do everything at once in markdown parser, until you have very special requirements to speed.

[cmrd_senya]

Well, I already have working code, and it would be simpler to modify what I have and not rewrite everything from scratch, if it’s possible. Especially if it is faster

[cmrd_senya]

So is it possible within the renderer?

[vitaly]

No. Renderer operates with tokens and is not expected to analyze/modify combinations.

You can write plugin for core chain to scan for paragraphs, anylize if content is link_open + text + link_close and do replace. Or manually call parse + scan/replace + render.

[v3ss0n]

Anything new in this ? Any standard way to do now?