Embedded audio and video

Extensions
41

.mp4 does mark that the file contains video - conventionally at least. It could be a used for audio as well, but this seems like an edge case. So I’m not sure that a different start symbol is needed in practice.

The convention is what people will think of when they’re writing Markdown, rather than what is strictly defined.

42

I can see why you’d want to group these together, but picture is going to be difficult to include. Most of the time you shouldn’t use picture, you should use img with srcset. (CSS Tricks has a good write up, and this post is quite blunt about it.) Switching ![]() to produce <picture> tags isn’t the right thing to do for the handful of times it might be right for the author. The principle of least surprise makes me think ![](abc.jpg) should always produce an <img> tag.

1 Like
43

I like this idea! I think it works quite well with some of the other discussion, too. How does this sound:

Syntax for embedded media is like the syntax for links, with one difference. Instead of link text, we have a media description. The rules for this are the same as for link text, except that (a) a media description starts with ![ or !mediatype[ rather than [ (where mediatype is one of image, audio or video) and (b) a media description may contain links. A media description has inline elements as its contents. When an image is rendered to HTML, this is standardly used as the image’s alt attribute.

  • If the mediatype is image then the media is rendered as an image. When rendered to HTML it should produce an <img> element.
  • If the mediatype is video then the media is rendered as embedded video. When rendering to HTML this may produce a <video> element.
  • If the mediatype is audio then the media is rendered as embedded audio. When rendering to HTML this may produce a <audio> element.
  • If the mediatype is not specified, or is unrecognised by the renderer, then the renderer may attempt to determine the type of media by file extension or other means. If it does not, it should default to producing an image.

So that means that existing markup like ![](image.jpg) will continue to work, as well as items like ![](image.php). Markup like ![](music.mp3) can be rendered as audio tags, as can !audio[](music.php).

I like that the syntax is extensible, too: if there are types of embedded media that we haven’t thought of (or haven’t been invented yet!) then it can be incorporated into the scheme without rewriting the spec.

3 Likes
44

We at the Diaspora* community are discussing the syntax to use for audio/video embed. As a major markdown user, it is important for us to be in conformity with the Markdown specifications, practices and ecosystem. So we are interested in having an approved official extension to the standard for media embedding.

5 Likes
45

As Founder of http://phwa.be , and Original Author of!()[mediafile.ext] Embedded audio and video for markdown-it’s audio + video implmentation . I would like to get an approved official extension for embeddable media types.

At http://phwa.be we are going to support many other embeddable documents ( any renderable document tru html5 ) . Tru the same syntax !()[doc.webm/mp4/ogv/ogg/pdf/**etc**]

1 Like
46

Yeah, or if we had generic attributes we could do:

![my title](music.php){type=audio}

There was also some discussion in the generic directives thread as to whether something like this should be rendered as a block or inline element. The tentative conclusion was that :audio[my title]{src=music.php} should be rendered inline (even if the markdown processor doesn’t know the audio directive/extension it would generate a <span> which postprocessor or js could pick up), and with two colons it should be rendered as a block (a <div> if extension unknown):

::audio[my title]{src=music.php}
47

Do you really want to make a http call each time to check the document type so you know whether you should render an <audio> element or embed the containing document? I think @zzzzBov really has a point that there might not always be a file extension present.

1 Like
48

Looks like it’s time for me to recommend additional specialized post-processors again.

I’m not saying that we can’t or shouldn’t have a format for embedding audio and video, but consider this a suggestion for possible alternatives:

<embed type="audio" src="path/to/audio.ext">
<embed type="video" src="path/to/video.ext">
<embed type="table" src="path/to/data.csv">

or if you need multiple sources:

<embed type="audio" src="path/to/audio.abc, path/to/audio.xyz">

The post processor could grab these embed elements and convert them into the appropriate markup. Authoring becomes easier, the specs become simpler, and the features are more atomic and reusable outside of the context of markdown.

49

Why would someone want to do that? We only doing that in client side, you can check what we did here : https://github.com/cmrd-senya/markdown-it-html5-embed

there is no server-side request there. We did that by looking at fileext and determining mime .

Yes @zzzzBov , there are problems with missing file extensions , and my idea of using lib-magic style header detection is strictly clientside (Uploading file type, Blobs etc) .

50

yeah, but even if you do it client side, I think it’s suboptimal, e.g. if the files are remote hosted (or currently not available). At least allow for an optional {type=audio} or similar.

51

Why not just use the alt?

![mp3](path/to/a/mp3/file)

![](path/to/filename.mp3)

![mp4](path/to/a/mp4/file =640x480)

52

How does the parser know the user wasn’t intending to embed an image with alt="mp3"?

53 
![=mp3](path/to/file)
![mp3=](path/to/file)
![mp3:](path/to/file)

or similar signal character with resource type identifier.

![audio/mpeg;version=1;layer=2](path/to/file)

Note that I didn’t verify the actual syntax for MIME type parameters.

ASCII art could also be used (bad adhoc designs):

![o_o](path/to/video_file)
![.|`](path/to/audio_file)
![o"](path/to/image_file)

… or Unicode emoji:

  • :movie_camera: U+1F3A5 :movie_camera:
  • :camera: U+1F4F7 :camera:
  • :video_camera: U+1F4F9 :video_camera:
  • :clapper: U+1F3AC :clapper:
  • :vhs: U+1F4FC :vhs:
  • :tv: U+1F4FA :tv:
  • :dvd: U+1F4C0 :dvd:
  • :cinema: U+1F3A6 :cinema:
  • :radio: U+1F4FB :radio:
  • :musical_note: U+1F3B5 :musical_note: ♩♪♫♬
  • :notes: U+1F3B6 :notes:
  • :microphone: U+1F3A4 :microphone:
  • :musical_score: U+1F3BC :musical_score:
  • :art: U+1F3A8 :art:
    *:film_strip: U+1F39E :film_frames:
    *:film_projector: U+1F4FD :film_projector:
    *:framed_picture: U+1F5BC :gallery:
    *:paintbrush: U+1F58C :paintbrush:
1 Like
54

For missing extension name, use consistent attributes syntax

A more consistent approach to missing extension name ( missing .mp3) is to just use the currently discussed consistent attributes syntax like:

![ cat meowing ]( example.com/meow ){ mediatype=audio/mpeg }

http://talk.commonmark.org/t/consistent-attribute-syntax/272

Why/Purpose of type field?

Purpose of the type field. The purpose of the type field is to indicate to the renderer how to deal with missing content. E.g. If it cannot find meow.mp3, then display a missing audio icon or show ::warning: Audio File Missing:: . If no type is specified, it will try to infer by extention name or mediatype.

e.g.

![ cat meowing ]( meow.mp3 ){ type=audio }

or

!audio[ cat meowing ]( meow.mp3 )

And hey, if the viewer program is smart enough, it could have a button saying "Search online for similar audio files for cat meowing"

3 Likes
56

I agree with Consistent attrib syntax.

3 Likes
57

While I understand that assuming .mp4 = video/mp4 may occasionally produce unexpected results, the times when this assumption is not accurate appear to be an edge cases. So perhaps consistent attribute syntax could be used to cover just the edge cases. The problem with requiring consistent attribute syntax is that it is an advanced feature and more difficult to remember for general writers. By following the convention, .mp4 = video/mp4, we allow users to simply write ![](video.mp4) and get the results that, in the majority of cases, they would expect. For edge cases they could use consistent attribute syntax to specify the content type, or raw HTML if the consistent attribute syntax extension is not enabled.

As an addendum to the proposed syntax, I suggest using the full source type (e.g. “video/mp4”) for consistency with HTML. Perhaps the order could be significant if multiple video sources are defined. So the following Markdown:

![](video.mp4 video.webm){type=video/mp4,video/webm}

Would render as:

<video>
  <source type="video/mp4" src="video.mp4">
  <source type="video/webm" src="video.webm">
</video>

There was also mention of using the contents of the square brackets for representing the media type. I disagree with this because it goes against the priniciple of uniformity (or at least the philosophy behind it). I suggest that square brackets are also used for alternative text for audio and video. E.g.

![Your browser does not support the video element.](video.mp4 video.webm)

Renders as:

<video>
  <source type="video/mp4" src="video.mp4">
  <source type="video/webm" src="video.webm">
  <p>Your browser does not support the video element.</p>
</video>
2 Likes
58

btw I thought that the [] in ![]() is for alt description for blind users etc…?

59

Correct, it corresponds to an image’s alt attribute (alternative text). My reasoning is that we should keep the usage of [] in ![]() relatively consistent for other embedded media and use it for an alternative text if the media itself cannot be rendered. Another option might be to use this to specify subtitles which are a form of alternative text. E.g. this Markdown:

![subtitles.vtt](video.mp4 video.webm)

Generates an HTML track element:

<video>
  <source src="video.mp4" type="video/mp4">
  <source src="video.webm" type="video/webm">
  <track src="subtitles.vtt">
</video>
3 Likes
60

Another reason to use the ![](video.mp4) syntax for embedded resources is that Markua will also use this syntax:

https://leanpub.com/markua/read#leanpub-auto-video

Cross compatibility is a good thing, even if Markua is not aiming for full compatibility with Markdown or CommonMark. The Markua spec linked here includes a whitelist of file extensions similar to what I suggested in earlier posts. .mp4 = video, .ogg = audio, as is the convention.

1 Like
61

We are not in disagreement. Yes, in most cases you use ![](video.mp4) but in edge cases (e.g. new video format, like webm), then ![](video.webm){type=video/webm} would be useful for when the whitelist doesn’t catch it. And if you care about being recognized as a missing video if nothing is found, then use the full form of !video[](video.webm){type=video/webm} can kick in (Which is the form that WYSIWYG commonmark editor might write as).