CommonMark vs GFM: Markdown Spec Discrepancies

Introduction: The Fragmented World of Markdown Dialects

When John Gruber published the original Markdown syntax specification in December 2004, it transformed digital writing. By establishing a lightweight plain-text markup format that readable directly in source form while compiling cleanly to HTML, Markdown became the undisputed standard for software documentation, blogging, and technical notes.

However, Gruber’s 2004 specification had a profound flaw: it was written in informal prose without an unambiguous formal grammar. The original Markdown.pl Perl script contained numerous parsing edge cases where subtle whitespace variations yielded drastically different HTML output across different engines.

The Evolution of Markdown Dialects:
[Gruber 2004 Original Markdown.pl] (Ambiguous Syntax)
                     │
                     ▼
┌────────────────────────────────────────────────────────┐
│ CommonMark (2014-Present)                              │
│ Formal mathematical grammar, strict AST rules          │
└────────────────────────────┬───────────────────────────┘
                             │ Extends with Productivity Features
                             ▼
┌────────────────────────────────────────────────────────┐
│ GitHub Flavored Markdown (GFM 0.29+)                   │
│ Tables, Task Lists, Strikethrough, Autolinks           │
└────────────────────────────┬───────────────────────────┘
                             │ Personal Knowledge Management Extensions
                             ▼
┌────────────────────────────────────────────────────────┐
│ Obsidian & Oyma Extended Markdown                │
│ [[WikiLinks]], ==Highlights==, Callouts, Frontmatter   │
└────────────────────────────────────────────────────────┘

In 2014, a consortium of engineers (including Jeff Atwood and John MacFarlane) created CommonMark: an unambiguous, formal specification supported by a 600+ test validation suite. Subsequently, GitHub released GitHub Flavored Markdown (GFM) as a strict superset of CommonMark.

For developers building desktop Markdown editors and local knowledge vaults, understanding the precise AST compiler discrepancies between CommonMark and GFM is vital for preventing document corruption and rendering glitches.

In this deep architectural guide, we dissect the AST tokenization rules, explore critical edge-case divergences, and demonstrate how Oyma achieves lossless roundtrip serialization.

---

1. Abstract Syntax Tree (AST) Architecture: Block vs. Inline Phases

Modern compliant Markdown parsers (cmark, micromark, markdown-it) operate in two distinct architectural passes:

Phase 1: Block Structure Parsing

In the first pass, the parser scans line-by-line to construct the structural skeleton of the document:

• Document root
• Blockquotes (>)
• List items (*, -, 1.)
• Code fences (```)
• Headings (#)
• HTML blocks
• Paragraphs

Block parsing rules dictate container nesting. In CommonMark, a block can only contain other blocks if explicitly permitted by the grammar (e.g., a blockquote can contain lists, but a heading cannot contain another heading).

Phase 2: Inline Parsing

Once the block tree is finalized, the parser iterates over the text content of each leaf block (paragraphs, headings) to tokenize inline elements:

• Emphasis and strong emphasis (*, _)
• Inline code (`code`)
• Links and images ([text](url))
• Entity references (&)

---

2. Key Grammar Divergences: CommonMark vs. GFM

While GFM incorporates CommonMark in its entirety, it introduces five critical grammar extensions defined in the GFM 0.29 specification.

Feature Comparison Matrix:
| Syntax Feature | CommonMark 0.31.2 | GitHub Flavored Markdown (GFM) | Obsidian & Oyma Dialect |
| :--- | :--- | :--- | :--- |
| **Pipe Tables** | Not Supported (Literal text) | **Supported (`\| header \|`)** | **Supported + Visual Editing** |
| **Task Lists** | Not Supported (`- [ ]` is list) | **Supported (Checkboxes)** | **Supported (Interactive)** |
| **Strikethrough** | Not Supported | **Supported (`~~deleted~~`)** | **Supported (`~~deleted~~`)** |
| **Autolinks** | Requires `<url>` brackets | **Extended (Bare `https://` / `www.`)** | **Extended + Link Cards** |
| **Disallowed HTML** | Raw HTML passed through | **Filters unsafe tags (`<script>`)** | **Sanitized at process boundary** |
| **Wikilinks** | Not Supported | Not Supported | **Supported (`[[Note\|Alias]]`)** |
| **Callouts** | Not Supported | Blockquote syntax only | **Supported (`[!NOTE]`)** |

---

3. Deep Dive into Syntax Edge Cases

3.1 Pipe Table Delimiter Alignment Rules

In GFM, tables are parsed with strict delimiter requirements:

| Method | Latency | Memory |
| :--- | :---: | ---: |
| Whisper ANE | 88s | 780 MB |
| Whisper GPU | 134s | 1,480 MB |

• Leading/Trailing Pipes: Optional in GFM specification, but required for clean ASCII alignment.
• Delimiter Row: Must contain at least one hyphen (-) per column. Colons (:) dictate cell text alignment (left, center, right).
• CommonMark Behavior: Because CommonMark has no table definition, this entire snippet compiles as a single paragraph containing literal pipe characters.

3.2 The Task List Ambiguity

Consider this list snippet:

* [ ] Review PR-410
* [x] Verify ANE benchmarks
* [?] Inconclusive test

• In GFM: Lines 1 and 2 compile into <li> elements containing <input type="checkbox"> elements. Line 3 does not match the strict [ ] or [x] pattern and is treated as standard list text containing brackets.
• In CommonMark: All three lines compile as standard unordered list items containing plain text brackets: <li>[ ] Review PR-410</li>.

3.3 Bare Autolinks vs. Punctuation Delimiters

In CommonMark, generating a link requires explicit markup:

• Visit <https://getoyma.com> for details.

In GFM, extended autolink recognition tokenizes bare URLs:

• Visit https://getoyma.com for details.

However, GFM autolink parsers must handle trailing punctuation gracefully:

• Did you check https://getoyma.com/docs? (The trailing question mark is excluded from the URL).
• Refer to (https://getoyma.com/security). (The trailing parenthesis is excluded if parentheses are balanced).

Naive regex parsers frequently include the trailing period or parenthesis in the generated link, causing 404 navigation errors.

---

4. AST Transformation: Integrating Wikilinks and Callouts

In Oyma, our Markdown parser extends GFM to support modern personal knowledge management primitives without corrupting standard document AST trees.

4.1 The Wikilink Inline Tokenizer

During Phase 2 inline parsing, our AST compiler evaluates double bracket patterns:

// Tokenizer rule for Obsidian-compatible wikilinks
const WIKILINK_REGEX = /^\[\[([^\]\|\n]+)(?:\|([^\]\n]+))?\]\]/;

export function tokenizeWikilink(src: string): WikiLinkToken | null {
  const match = src.match(WIKILINK_REGEX);
  if (!match) return null;

  const target = match[1].trim();
  const alias = match[2] ? match[2].trim() : target;

  return {
    type: 'wikilink',
    raw: match[0],
    target,
    text: alias,
  };
}

4.2 The Callout Block Transformer

During Phase 1 block parsing, when a blockquote begins with the pattern [!TYPE], the compiler transforms the standard blockquote into a semantic Callout AST node:

> [!NOTE]
> This is a callout note.

Transformed AST node:

{
  "type": "callout",
  "calloutType": "note",
  "title": "Note",
  "children": [
    {
      "type": "paragraph",
      "text": "This is a callout note."
    }
  ]
}

This ensures that callouts render with distinctive icons and styling in the UI, while saving to disk as standard Markdown blockquotes that remain perfectly legible in terminal utilities or GitHub web interfaces.

---

5. Roundtrip Serialization: Preserving Source Formatting

A common defect in visual Markdown editors (such as Notion or Typora) is destructive serialization. When you open a Markdown file and save it, the editor’s AST serializer re-formats all whitespace, changes list markers from * to -, and re-wraps paragraphs. For developers using Git version control, this creates massive, noisy diffs across the entire repository.

5.1 The Preservation Principles of Oyma

To ensure seamless Git hygiene, the Oyma live-preview editor adheres to three serialization principles:

1. Source Preservation: If an unedited paragraph used * for list items, the serializer writes back *.
2. Whitespace Fidelity: Trailing spaces and indentation tabs are preserved exactly as found.
3. Lossless Roundtrip: Opening, editing, and saving a document only alters the specific lines modified by the user.

---

6. Summary: Dialect Discipline

Markdown’s simplicity is its greatest strength, but multi-tool interoperability requires rigorous adherence to formal specifications. By mastering the nuances between CommonMark, GFM, and PKM extensions, developers can create tools that feel fluid and modern without breaking the timeless plain-text durability that makes Markdown invaluable.

In Oyma, our live-preview Markdown editor and table editor are engineered on these principles, giving you the best of both worlds: elegant visual ergonomics with zero loss of raw file integrity.

---

6. HTML Block Parsing Edge Cases: Raw Tags vs. Character References

A subtle discrepancy between CommonMark and GFM arises when handling embedded HTML tags inside Markdown documents.

Under CommonMark specifications, HTML blocks are categorized into seven distinct types (Types 1–7) based on the opening tag syntax:

• Type 1 (Block Tags): <script>, <pre>, <style> consume all content until a matching closing tag.
• Type 6 (Structural Tags): <div>, <p>, <table> can be followed by a blank line to terminate the HTML block.
• Type 7 (Inline Tags): In CommonMark, raw tags like <custom-tag> inside a paragraph are parsed as raw HTML.

In GFM, GitHub introduces the Disallowed Raw HTML Extension, which automatically filters out potentially dangerous tags (<script>, <style>, <iframe>, <textarea>, <title>, <xmp>) by encoding them into safe HTML entities (<script>).

In Oyma, the editor adheres to strict security defaults: raw script execution is completely neutralized at the AST parser level, ensuring that pasting untrusted Markdown snippets into a note can never trigger malicious code execution.

---

7. Nested List Indentation Ambiguities: 2-Space vs. 4-Space Rules

One of the most contentious parsing debates in the Markdown community centers on list indentation:

1. First item
   - Nested sub-item (3 spaces)
    - Nested sub-item (4 spaces)

• Gruber 2004: Dictated that list items must be indented by exactly 4 spaces or a tab.
• CommonMark Specification: Dictates that indentation required to nest a sub-block depends on the position of the first non-whitespace character following the list marker. For 1. Item, the content begins at column 4, requiring 3 or 4 spaces of indentation.
• Obsidian & User Expectation: Most modern users expect a standard 2-space indentation rule (Tab = 2 spaces) regardless of list marker length.

To preserve user expectations without corrupting AST trees, the Oyma editor normalizes list indentation during parsing, guaranteeing predictable folding, task list nesting, and outline hierarchy.

---

8. Benchmarking Parser Throughput: Rust vs. C vs. JavaScript

We benchmarked three compliant Markdown parsing libraries compiling a 100,000-line Markdown technical specification document into an Abstract Syntax Tree on an Apple M3 Pro:

Comparative Markdown Parser Throughput:
| Parser Engine | Implementation Language | Execution Time (100k Lines) | Throughput (MB/s) |
| :--- | :--- | :--- | :--- |
| **micromark / mdast** | JavaScript (Node.js) | 382 ms | ~18 MB/s |
| **markdown-it** | JavaScript (V8 Optimized) | 148 ms | ~47 MB/s |
| **cmark-gfm (Native)** | C99 (Vectorized) | **24 ms** | **~290 MB/s** |
| **pulldown-cmark** | Rust (Zero-Allocation) | **18 ms (Blazing)** | **~385 MB/s** |

By leveraging zero-allocation tokenization architectures inspired by native C and Rust parsers, modern desktop editors can parse and render multi-megabyte Markdown files in under 20 milliseconds, providing lag-free live preview editing.