Embedded audio and video

Thanks , sorry , i was being busy with some deadline .
I will fork and add some changes.

How about naming it markdown-it-multimedia-html5 ?
Also can you credit me in authorā€™s list? I will be contributing from time to time.

checking audio or video can be done via reading the header right?

Thatā€™s ok. Iā€™m looking forward for it :smile:
Check for the recent changes (v0.1.0)

I donā€™t have authors list yet, but Iā€™ll add one if you want. Though authors are usually contributors listed in the commit history. I also added link to this discussion as a source origin in the source code.

I donā€™t really want to change the name, because it is already registered in npm and bower, though it is not a big deal.

Yes, basing on MIME type, not on the extension. It would be better, but I thought to leave it for a while in a simple way.

@v3ss0n, I can give you write permissions for the repository, if you want, so we donā€™t get bothered with forking and merging.

Write permission is fine :slight_smile: . But i will work on branches .

What are you going to implement?

Itā€™s worth noting that Maruka is using the .mp4 file extension together with the Markdown image syntax for video.

https://leanpub.com/markua/read#leanpub-auto-resource-type-image-code-math-text-audio-and-video

Markua Processors rely on file extension to determine the type of media and must not attempt to parse files to determine their type. Because of this, the choice of acceptable file extensions for the various media types is a subset of the total available, so that audio and video files can be distinguished solely by their file extension instead of by examining the file or by requiring authors to type some special metadata syntax.

and

.mp4
The file is treated as an MP4 video.

and when describing the audio file extensions:

.m4a, .aac
The file is treated as an MP4 AAC (Advanced Audio Coding) audio file. Note that .mp4 is not supported as a file extension for MP4 AAC audio, since that is the file extension used for MP4 video.

This is essentially the same as what I suggested for CommonMark.

While Maruka isnā€™t exactly Markdown (itā€™s not aiming for backward compatibility - it removes some syntax from Markdown and adds new syntax), I think it would be wise to aim for some level of syntax compatibility. Markdown and Maruka are close enough to allow copy/pasting text between the two (with some modification).

Maruka? More markdown specs? Will this ever end lol.

I am itching to do libmagic-like header detecting in javascript. Should i attempt that route? Iā€™ve done header parsing for detecting file type in Python , i think it should work fine for javascript fileobject , except we do not have mmap in javascript.

with that we can easily detect mime via file contents.

The CMS that I primarily work with serves all media (images, video, audio, pdfs, you name it) from requests that use the .asax extension. This is similar to a .php file serving any sort of data. If this extension is going to be useful, there needs to be a way to mark what type of media youā€™re serving.

My initial reaction would be to mark the type of media via the document fragment (I blogged about leveraging this for styling images with markdown), but it could conflict with existing document fragments used in SVG.

As this is being proposed as an extension, maybe it makes sense to use different symbols for the different embeds.

Audio could use @[](), and video could use ^[](). That way itā€™s up to the author to specify what theyā€™re embedding.

1 Like

.mp4 does mark that the file contains video - conventionally at least. It could be a used for audio as well, but this seems like an edge case. So Iā€™m not sure that a different start symbol is needed in practice.

The convention is what people will think of when theyā€™re writing Markdown, rather than what is strictly defined.

I can see why youā€™d want to group these together, but picture is going to be difficult to include. Most of the time you shouldnā€™t use picture, you should use img with srcset. (CSS Tricks has a good write up, and this post is quite blunt about it.) Switching ![]() to produce <picture> tags isnā€™t the right thing to do for the handful of times it might be right for the author. The principle of least surprise makes me think ![](abc.jpg) should always produce an <img> tag.

1 Like

I like this idea! I think it works quite well with some of the other discussion, too. How does this sound:

Syntax for embedded media is like the syntax for links, with one difference. Instead of link text, we have a media description. The rules for this are the same as for link text, except that (a) a media description starts with ![ or !mediatype[ rather than [ (where mediatype is one of image, audio or video) and (b) a media description may contain links. A media description has inline elements as its contents. When an image is rendered to HTML, this is standardly used as the imageā€™s alt attribute.

  • If the mediatype is image then the media is rendered as an image. When rendered to HTML it should produce an <img> element.
  • If the mediatype is video then the media is rendered as embedded video. When rendering to HTML this may produce a <video> element.
  • If the mediatype is audio then the media is rendered as embedded audio. When rendering to HTML this may produce a <audio> element.
  • If the mediatype is not specified, or is unrecognised by the renderer, then the renderer may attempt to determine the type of media by file extension or other means. If it does not, it should default to producing an image.

So that means that existing markup like ![](image.jpg) will continue to work, as well as items like ![](image.php). Markup like ![](music.mp3) can be rendered as audio tags, as can !audio[](music.php).

I like that the syntax is extensible, too: if there are types of embedded media that we havenā€™t thought of (or havenā€™t been invented yet!) then it can be incorporated into the scheme without rewriting the spec.

3 Likes

We at the Diaspora* community are discussing the syntax to use for audio/video embed. As a major markdown user, it is important for us to be in conformity with the Markdown specifications, practices and ecosystem. So we are interested in having an approved official extension to the standard for media embedding.

5 Likes

As Founder of http://phwa.be , and Original Author of!()[mediafile.ext] Embedded audio and video for markdown-itā€™s audio + video implmentation . I would like to get an approved official extension for embeddable media types.

At http://phwa.be we are going to support many other embeddable documents ( any renderable document tru html5 ) . Tru the same syntax !()[doc.webm/mp4/ogv/ogg/pdf/**etc**]

1 Like

Yeah, or if we had generic attributes we could do:

![my title](music.php){type=audio}

There was also some discussion in the generic directives thread as to whether something like this should be rendered as a block or inline element. The tentative conclusion was that :audio[my title]{src=music.php} should be rendered inline (even if the markdown processor doesnā€™t know the audio directive/extension it would generate a <span> which postprocessor or js could pick up), and with two colons it should be rendered as a block (a <div> if extension unknown):

::audio[my title]{src=music.php}

Do you really want to make a http call each time to check the document type so you know whether you should render an <audio> element or embed the containing document? I think @zzzzBov really has a point that there might not always be a file extension present.

1 Like

Looks like itā€™s time for me to recommend additional specialized post-processors again.

Iā€™m not saying that we canā€™t or shouldnā€™t have a format for embedding audio and video, but consider this a suggestion for possible alternatives:

<embed type="audio" src="path/to/audio.ext">
<embed type="video" src="path/to/video.ext">
<embed type="table" src="path/to/data.csv">

or if you need multiple sources:

<embed type="audio" src="path/to/audio.abc, path/to/audio.xyz">

The post processor could grab these embed elements and convert them into the appropriate markup. Authoring becomes easier, the specs become simpler, and the features are more atomic and reusable outside of the context of markdown.

Why would someone want to do that? We only doing that in client side, you can check what we did here : https://github.com/cmrd-senya/markdown-it-html5-embed

there is no server-side request there. We did that by looking at fileext and determining mime .

Yes @zzzzBov , there are problems with missing file extensions , and my idea of using lib-magic style header detection is strictly clientside (Uploading file type, Blobs etc) .

yeah, but even if you do it client side, I think itā€™s suboptimal, e.g. if the files are remote hosted (or currently not available). At least allow for an optional {type=audio} or similar.