EDIT: I have created: https://github.com/cirosantilli/commonmark-implementation-compare which is a local Vagrantfile based comparison.
Another possibility would be a service like: http://johnmacfarlane.net/babelmark2 , but if we manage to maintain it a Vagrantfile is ideal as it makes things more explicit.
I would love to see something like:
for Markdown implementations with the tests in this spec. Should also have an error percent count.
We could use something like https://github.com/karlcow/markdown-testsuite/blob/master/Vagrantfile to implement it, and then serve the results online.
This would give a great incentive for other implementations to comply.