Engineering Markdown Tables: Keyboard & AST

Introduction: The Ergonomic Failure of Plain-Text Tables

In technical documentation, software architecture, and executive meeting notes, tabular data is essential. Engineers use tables to document API request/response specifications, compare algorithmic benchmarks, and track project deliverables.

GitHub Flavored Markdown (GFM) standardized the pipe table syntax:

| Service | Protocol | Latency | Status |
| :--- | :--- | :---: | ---: |
| Auth Gateway | gRPC / TLS | 4.2 ms | Active |
| Storage Worker | POSIX File | 0.8 ms | Active |

While pipe tables are readable in source form, editing them manually in a standard plain-text editor is agonizing:

• Typing five words into a cell shifts the trailing pipe, misaligning the entire visual column.
• Adding a new column requires manual re-formatting across dozens of rows.
• Pasting a row from an Excel spreadsheet or Google Sheet injects tab characters, breaking the table structure completely.

The Tabular Editing Gap:
Raw Plain-Text Editor:
| Service | Protocol | Latency |   <--- Misaligned ASCII chaos!
| Auth Gateway | gRPC / TLS | 4.2 ms |
| Worker | POSIX | 0.8 ms |

Oyma Spreadsheet Live-Preview:
┌─────────────────┬─────────────────┬──────────┬──────────┐
│ Service         │ Protocol        │ Latency  │ Status   │  <--- Fluid Tab/Enter traversal
├─────────────────┼─────────────────┼──────────┼──────────┤  <--- Drag-to-resize columns
│ Auth Gateway    │ gRPC / TLS      │ 4.2 ms   │ Active   │  <--- TSV Clipboard paste
│ Storage Worker  │ POSIX File      │ 0.8 ms   │ Active   │
└─────────────────┴─────────────────┴──────────┴──────────┘

To deliver a modern writing experience without abandoning plain-text durability, a desktop Markdown editor must provide spreadsheet-like ergonomics in the UI, while saving pristine, human-readable GFM tables to disk.

In this engineering walkthrough, we explore the keyboard state machine, clipboard TSV parsers, and AST serialization rules required to build a production-grade live-preview table editor.

---

1. The Keyboard Navigation State Machine

A spreadsheet feels fluid because users never need to reach for the mouse. Navigation relies entirely on standard keyboard conventions:

• Tab: Advance focus to the next cell.
• Shift + Tab: Retreat focus to the previous cell.
• Enter: Move focus to the cell directly below.
• Enter (on final row): Insert a new row and focus its first cell.
• Arrow Keys: Move within text; cross cell boundaries when cursor strikes text edge.

1.1 State Machine Implementation

In our ProseMirror/CodeMirror editor architecture, keyboard events inside table cells are intercepted by a dedicated keymap extension:

// editor/keymaps/table-navigation.ts
export function handleTableKeyDown(event: KeyboardEvent, view: EditorView): boolean {
  if (event.key === 'Tab') {
    event.preventDefault();
    if (event.shiftKey) {
      return navigateToPreviousCell(view);
    } else {
      return navigateToNextCellOrInsertRow(view);
    }
  }

  if (event.key === 'Enter' && !event.shiftKey) {
    event.preventDefault();
    return handleEnterKeyNavigation(view);
  }

  return false;
}

function navigateToNextCellOrInsertRow(view: EditorView): boolean {
  const { state, dispatch } = view;
  const currentCell = findParentTableCell(state.selection);
  if (!currentCell) return false;

  const nextCell = findNextCellInTable(state.doc, currentCell);
  if (nextCell) {
    // Focus next existing cell
    dispatch(state.tr.setSelection(TextSelection.create(state.doc, nextCell.pos + 1)));
    return true;
  }

  // Final cell struck: atomically append new row!
  const newRowTr = insertTableRowAtEnd(state.tr, currentCell.tableNode);
  dispatch(newRowTr);
  return true;
}

---

2. Ingesting Tabular Clipboards: TSV to GFM Conversion

When knowledge workers copy data from spreadsheets (Microsoft Excel, Apple Numbers, Google Sheets), desktop operating systems place tabular data onto the system pasteboard formatted as Tab-Separated Values (TSV) (text/tab-separated-values or standard text/plain with \t delimiters).

If pasted into a raw text editor, tab characters paste literally, destroying document formatting.

2.1 The Clipboard Ingestion Parser

In Oyma, the clipboard paste listener detects tabular data and transforms it into structured GFM AST nodes:

// editor/clipboard/table-paste-handler.ts
export function handleTabularPaste(clipboardText: string): string | null {
  const lines = clipboardText.trim().split(/\r?\n/);
  if (lines.length < 2) return null;

  // Verify lines contain tab delimiters
  const hasTabs = lines.every((line) => line.includes('\t'));
  if (!hasTabs) return null;

  const parsedRows = lines.map((line) => line.split('\t').map((cell) => cell.trim()));
  const columnCount = Math.max(...parsedRows.map((r) => r.length));

  // Construct formatted GFM pipe table
  const gfmOutput: string[] = [];

  // 1. Header row
  const header = parsedRows[0];
  while (header.length < columnCount) header.push('');
  gfmOutput.push('| ' + header.map(sanitizeCellContent).join(' | ') + ' |');

  // 2. Delimiter row
  const delimiter = Array(columnCount).fill(':---');
  gfmOutput.push('| ' + delimiter.join(' | ') + ' |');

  // 3. Body rows
  for (let i = 1; i < parsedRows.length; i++) {
    const row = parsedRows[i];
    while (row.length < columnCount) row.push('');
    gfmOutput.push('| ' + row.map(sanitizeCellContent).join(' | ') + ' |');
  }

  return gfmOutput.join('\n');
}

function sanitizeCellContent(cell: string): string {
  // Escape pipes and replace newlines with HTML breaks
  return cell.replace(/\|/g, '\\|').replace(/\r?\n/g, '<br>');
}

With this handler active, copying an eight-column financial model from Excel and pressing ⌘V renders an immaculate, perfectly aligned Markdown table instantaneously.

---

3. Lossless AST Serialization: Pretty-Printing Pipes

When saving the document back to APFS disk storage, the editor serializes its in-memory table node back into plain text.

To ensure that the file remains beautiful when opened in terminal pagers (cat, less) or command-line editors (vim), the serializer computes the maximum character width of each column and pads cells evenly:

| Service Name  | Protocol Type | Latency (p99) | Cluster Status |
| :------------ | :------------ | :------------ | :------------- |
| Auth Gateway  | gRPC / TLS    | 4.2 ms        | Active         |
| Storage Node  | POSIX APFS    | 0.8 ms        | Active         |
| Speech Engine | Core ML ANE   | 2.1 ms        | Active         |

Every column is vertically aligned down to the character, preserving the pristine visual aesthetics of hand-crafted ASCII documentation.

---

4. Rich Media Inside Table Cells

Unlike rigid Markdown parsers that restrict table cells to single words, modern knowledge workers require rich formatting within tables:

• Wikilinks: [[Julian Thorne]] and [[ADR-042 Search Engine]].
• Inline Badges: ==Active== and ~~Deprecated~~.
• Images: Small thumbnail attachments ![[architecture-thumb.png|80]].

During live-preview rendering, Oyma parses inline tokens within each cell independently, rendering rich interactive links and images while serializing back to standard, compliant GitHub Flavored Markdown.

---

5. Summary: Ergonomics Without Lock-In

Productivity software should never force users to choose between ergonomic visual editing and plain-text data sovereignty.

By engineering a responsive keyboard state machine, intelligent clipboard TSV parsers, and beautiful AST column serializers, Oyma delivers a spreadsheet-like table editor that feels as fluid as modern web apps—while ensuring that every byte of your data remains a plain, portable Markdown file on your Mac.

---

6. Virtualized Horizontal Scrolling for Wide Enterprise Tables

In corporate finance, systems architecture, and database design, tables frequently exceed ten columns, spanning thousands of pixels in width.

In naive desktop text editors, wide tables break the visual layout:

• Either the entire application window develops an awkward horizontal scrollbar that shifts the editor margins, or
• Columns are aggressively squeezed into illegible slivers.

Oyma resolves this through Virtualized Table Containers:

• The table is rendered inside a self-contained overflow viewport with subtle directional fade indicators.
• Column headers remain pinned (position: sticky) during vertical reading.
• Pressing Tab to navigate off-screen cells automatically scrolls the table viewport smoothly into view, keeping the active cursor centered without disrupting the reading flow.

---

7. Formula Calculation Support: Lightweight Sum and Average Cells

Knowledge workers frequently need basic arithmetic calculations in meeting notes (such as summing project budgets or calculating average sprint velocities).

Rather than forcing users to open an external spreadsheet application, Oyma supports lightweight Markdown formula syntax:

• |=SUM(C1:C12)|
• |=AVG(D1:D10)|

When rendering, the live-preview editor computes the formula in real time and displays the calculated numeric value. When serializing to disk, the formula remains saved in standard plain text, preserving complete human readability and interoperability with external tools.

---

8. Undo/Redo Transaction State Management in Tabular History

Editing tables involves multi-step structural mutations (inserting rows, deleting columns, reordering cells). In poorly implemented editors, pressing ⌘Z (Undo) causes catastrophic layout breaks, restoring raw pipe characters into the middle of rendered cells.

To ensure flawless undo/redo ergonomics:

• Every table mutation is wrapped in a single, atomic ProseMirror Transaction.
• Adding a row generates a single history state step. Pressing ⌘Z reverts the entire row insertion atomically.
• Cell text edits are debounced into logical typing clusters, allowing users to undo typing word-by-word without ever corrupting the surrounding table grid structure.

---

9. East Asian Ambiguous Width & Zero-Width Joiner (ZWJ) Unicode Alignment

A notorious engineering challenge when formatting ASCII Markdown tables in monospaced fonts is character cell width calculation. Under the Unicode Standard Annex #11 (East Asian Width):

• Standard ASCII Latin characters occupy 1 display cell column (width = 1).
• Fullwidth CJK ideographs (Chinese, Japanese, Korean) occupy 2 display cell columns (width = 2).
• Zero-Width Joiners (ZWJ \u200D) and skin tone modifiers used in modern emojis occupy 0 additional cell columns.

If an ASCII table formatter naively relies on JavaScript’s .length property (which counts UTF-16 code units), strings containing emojis or Japanese kanji will misalign by dozens of characters:

import wcwidth from 'wcwidth';

// Compute true terminal monospaced display width
export function computeCellDisplayWidth(cellContent: string): number {
  // Strip ANSI escape codes if present
  const clean = cellContent.replace(/\u001b\[[0-9;]*m/g, '');
  return wcwidth(clean);
}

export function padCellToWidth(content: string, targetWidth: number, align: 'left' | 'right' | 'center'): string {
  const currentWidth = computeCellDisplayWidth(content);
  const totalPadding = Math.max(0, targetWidth - currentWidth);

  if (align === 'right') return ' '.repeat(totalPadding) + content;
  if (align === 'center') {
    const leftPad = Math.floor(totalPadding / 2);
    const rightPad = totalPadding - leftPad;
    return ' '.repeat(leftPad) + content + ' '.repeat(rightPad);
  }
  return content + ' '.repeat(totalPadding);
}

By computing terminal cell width via POSIX wcwidth() conventions, Oyma guarantees that table borders align with pixel-perfect precision across all international character sets and emoji sequences.

---

10. Streaming Table Parser for Multi-Megabyte Data Tables

When reading large tabular data files, waiting for the entire document to tokenize before rendering freezes the user interface. Oyma employs an incremental line-by-line streaming table parser:

1. Header Recognition: Inspects candidate lines for pipe delimiter markers (|).
2. Alignment Extraction: Compiles the delimiter row into an array of column alignment enums ([Align::Left, Align::Center, Align::Right]).
3. Chunked Row Rendering: Streams data rows in batches of 50 into the DOM, maintaining 60 FPS smooth scrolling even while rendering 10,000-row benchmarking datasets.