This is the full developer documentation for redlines
# Quickstart
> Install redlines and compare two texts from the CLI or from Python.
`redlines` compares two strings and produces structured output showing their differences. Changes are represented with strike-throughs and highlights, in the manner of Microsoft Word’s track changes, and the output carries change information, positions and statistics for programmatic use.
## Install
[Section titled “Install”](#install)
```bash
pip install redlines
```
Python 3.10 to 3.14 are supported.
### Optional extras
[Section titled “Optional extras”](#optional-extras)
| Extra | Install | What it adds |
| ------------- | ----------------------------------- | ----------------------------------------------------------------------------------------- |
| `pdf` | `pip install redlines[pdf]` | Comparing PDF files |
| `nupunkt` | `pip install redlines[nupunkt]` | Sentence boundary detection that handles abbreviations, citations and URLs (Python 3.11+) |
| `levenshtein` | `pip install redlines[levenshtein]` | Levenshtein distance in the statistics |
## Compare from the command line
[Section titled “Compare from the command line”](#compare-from-the-command-line)
JSON is the default output, so a bare invocation is enough:
```bash
redlines "The quick brown fox jumps over the lazy dog." "The quick brown fox walks past the lazy dog."
```
Files work the same way, and `--pretty` makes the JSON readable:
```bash
redlines --pretty old_version.txt new_version.txt
```
## Compare from Python
[Section titled “Compare from Python”](#compare-from-python)
```python
from redlines import Redlines
test = Redlines(
"The quick brown fox jumps over the lazy dog.",
"The quick brown fox walks past the lazy dog.",
markdown_style="none",
)
print(test.output_markdown)
# The quick brown fox jumps over walks past the lazy dog.
```
Other output formats are available on the same object: `output_json` for structured changes and statistics, `output_rich` for terminal display, and `compare(markdown_style=...)` for the six markdown styles.
## Where to go next
[Section titled “Where to go next”](#where-to-go-next)
* The [agent integration guide](/redlines/guides/agent-guide/) covers invocation from agents and automation, the JSON structure, error handling and integration patterns.
* The [API reference](/redlines/api/) is generated from the docstrings and documents every class and method.
* The [decision records](/redlines/project/adr/) explain why the library is shaped the way it is, and where it is going.
## Reading this site as an agent
[Section titled “Reading this site as an agent”](#reading-this-site-as-an-agent)
Every page is available as plain markdown by appending `.md` to its path — [`/guides/agent-guide.md`](/redlines/guides/agent-guide.md), for instance. [`/llms.txt`](/redlines/llms.txt) indexes the site, [`/llms-small.txt`](/redlines/llms-small.txt) is the usage documentation in one file, and [`/llms-full.txt`](/redlines/llms-full.txt) adds the planning documents and decision records.
# Agent integration guide
> Invocation, output formats, JSON structure, error handling and integration patterns for calling redlines from agents and automation.
This is the 0.6 guide
It documents the current release and is accurate for it. For 1.0 it is being replaced by a short contract page plus task pages whose code comes from `examples/`, with the machine-readable parts — schemas, `llms.txt`, the MCP tool descriptions — carrying what prose used to. See [ADR-0027](/redlines/project/adr/0027-agent-docs-machine-surface/).
## Quick Start
[Section titled “Quick Start”](#quick-start)
### 🤖 Agent-Friendly CLI (New!)
[Section titled “🤖 Agent-Friendly CLI (New!)”](#-agent-friendly-cli-new)
For maximum simplicity, you can now invoke redlines without specifying a command. It automatically outputs JSON (the most agent-friendly format):
```bash
# Simplest invocation - just provide two strings/files
redlines "source text" "test text"
# Pretty-print for readability
redlines --pretty "source text" "test text"
# Works with files too
redlines old_version.txt new_version.txt
```
**Why this is better for agents:**
* No need to choose between `text`, `json`, `markdown`, `stats` commands upfront
* JSON output is structured and parseable
* Consistent, predictable behavior
* Fewer tokens needed in prompts
**Traditional commands still work** if you need specific output formats (see [Output Formats](#output-formats)).
### Installation
[Section titled “Installation”](#installation)
```bash
# Basic installation
pip install redlines
# With PDF file comparison support
pip install redlines[pdf]
# With advanced sentence tokenization (Python 3.11+)
pip install redlines[nupunkt]
# With Levenshtein distance metrics
pip install redlines[levenshtein]
```
### Your First Comparison (30 seconds)
[Section titled “Your First Comparison (30 seconds)”](#your-first-comparison-30-seconds)
```python
from redlines import Redlines
# Compare two strings
diff = Redlines(
"The quick brown fox jumps over the lazy dog.",
"The quick brown fox walks past the lazy dog."
)
# Get markdown output
print(diff.output_markdown)
# Output: The quick brown fox jumps over walks past the lazy dog.
```
### CLI Quick Start
[Section titled “CLI Quick Start”](#cli-quick-start)
```bash
# Compare strings (command-less, outputs JSON by default)
redlines "Hello world" "Hi world"
# Pretty-print JSON output
redlines --pretty "Hello world" "Hi world"
# Compare files (auto-detected, command-less)
redlines old_version.txt new_version.txt
# Or use explicit commands for specific output formats
redlines text "Hello world" "Hi world"
redlines json old_version.txt new_version.txt --pretty
redlines markdown file1.txt file2.txt --markdown-style ghfm
# Check if files differ (for CI/CD)
if redlines file1.txt file2.txt > /dev/null 2>&1; then
echo "Files have changes"
else
echo "Files are identical"
fi
```
***
## Common Patterns
[Section titled “Common Patterns”](#common-patterns)
### Pattern 1: Compare Two Files
[Section titled “Pattern 1: Compare Two Files”](#pattern-1-compare-two-files)
```python
from pathlib import Path
from redlines import Redlines
# Read files
source = Path("old_version.txt").read_text()
test = Path("new_version.txt").read_text()
# Compare
diff = Redlines(source, test)
# Get results
print(f"Total changes: {diff.stats().total_changes}")
print(diff.output_markdown)
```
### Pattern 1b: Compare PDF Files
[Section titled “Pattern 1b: Compare PDF Files”](#pattern-1b-compare-pdf-files)
```python
from redlines import Redlines
from redlines.pdf import PDFFile
# Load PDF files (requires: pip install redlines[pdf])
source = PDFFile("contract_v1.pdf")
test = PDFFile("contract_v2.pdf")
# Compare
diff = Redlines(source, test)
# Get results
print(f"Total changes: {diff.stats().total_changes}")
print(diff.output_markdown)
# Access page information
print(f"Source has {source.page_count} pages")
for page in source.pages:
print(f"Page {page.page_number}: {len(page.text)} chars")
```
```bash
# CLI: PDF files are auto-detected
redlines contract_v1.pdf contract_v2.pdf --pretty
```
### Pattern 2: Get Machine-Readable JSON
[Section titled “Pattern 2: Get Machine-Readable JSON”](#pattern-2-get-machine-readable-json)
```python
import json
from redlines import Redlines
diff = Redlines(source, test)
# Get JSON output
json_output = diff.output_json(pretty=True)
data = json.loads(json_output)
# Process changes
for change in data["changes"]:
print(f"{change['type']}: {change.get('source_text', '')} → {change.get('test_text', '')}")
```
### Pattern 3: Filter Specific Operations
[Section titled “Pattern 3: Filter Specific Operations”](#pattern-3-filter-specific-operations)
```python
from redlines import Redlines
diff = Redlines(source, test)
# Get only insertions
insertions = diff.get_changes(operation="insert")
for change in insertions:
print(f"Added: {change.test_text}")
# Get only deletions
deletions = diff.get_changes(operation="delete")
for change in deletions:
print(f"Removed: {change.source_text}")
# Get only replacements
replacements = diff.get_changes(operation="replace")
for change in replacements:
print(f"Changed: {change.source_text} → {change.test_text}")
```
### Pattern 4: Collect Statistics
[Section titled “Pattern 4: Collect Statistics”](#pattern-4-collect-statistics)
```python
from redlines import Redlines
diff = Redlines(source, test)
stats = diff.stats()
print(f"Total changes: {stats.total_changes}")
print(f"Insertions: {stats.insertions}")
print(f"Deletions: {stats.deletions}")
print(f"Replacements: {stats.replacements}")
print(f"Change ratio: {stats.change_ratio:.1%}")
print(f"Characters added: {stats.chars_added}")
print(f"Characters deleted: {stats.chars_deleted}")
print(f"Net change: {stats.chars_net_change}")
```
### Pattern 5: Batch Process Multiple Files
[Section titled “Pattern 5: Batch Process Multiple Files”](#pattern-5-batch-process-multiple-files)
```python
from pathlib import Path
from redlines import Redlines
def compare_directory(dir1: Path, dir2: Path):
"""Compare all matching files in two directories."""
results = []
for file1 in dir1.glob("*.txt"):
file2 = dir2 / file1.name
if not file2.exists():
continue
diff = Redlines(
file1.read_text(),
file2.read_text()
)
stats = diff.stats()
if stats.total_changes > 0:
results.append({
"file": file1.name,
"changes": stats.total_changes,
"change_ratio": stats.change_ratio
})
return results
# Usage
results = compare_directory(Path("old/"), Path("new/"))
for result in results:
print(f"{result['file']}: {result['changes']} changes ({result['change_ratio']:.1%})")
```
### Pattern 6: Generate HTML Report
[Section titled “Pattern 6: Generate HTML Report”](#pattern-6-generate-html-report)
```python
from redlines import Redlines
diff = Redlines(source, test, markdown_style="none")
html_template = f"""
Diff Report
{diff.output_markdown}
"""
Path("report.html").write_text(html_template)
```
***
## Output Formats
[Section titled “Output Formats”](#output-formats)
### Markdown Styles
[Section titled “Markdown Styles”](#markdown-styles)
```python
from redlines import Redlines
from redlines.enums import MarkdownStyle
# Available styles:
styles = {
"red_green": MarkdownStyle.RED_GREEN, # Red strikethrough + green bold (default)
"none": MarkdownStyle.NONE, # Plain / HTML tags
"red": MarkdownStyle.RED, # All changes in red
"ghfm": MarkdownStyle.GHFM, # GitHub Flavored Markdown
"bbcode": MarkdownStyle.BBCODE, # BBCode format
"streamlit": MarkdownStyle.STREAMLIT, # Streamlit-compatible
}
# Use a style
diff = Redlines(source, test, markdown_style=MarkdownStyle.GHFM)
print(diff.output_markdown)
```
### Rich Terminal Output
[Section titled “Rich Terminal Output”](#rich-terminal-output)
```python
from redlines import Redlines
from rich import print as rprint
diff = Redlines(source, test)
# Get Rich-formatted output for terminal
rprint(diff.output_rich)
```
### JSON Output
[Section titled “JSON Output”](#json-output)
```python
import json
from redlines import Redlines
diff = Redlines(source, test)
# Pretty-printed JSON
json_str = diff.output_json(pretty=True)
# Compact JSON
json_str = diff.output_json(pretty=False)
# Parse and use
data = json.loads(json_str)
```
***
## JSON Schema Reference
[Section titled “JSON Schema Reference”](#json-schema-reference)
### Complete JSON Structure
[Section titled “Complete JSON Structure”](#complete-json-structure)
```json
{
"source": "original text",
"test": "modified text",
"source_tokens": ["token1", "token2", " ¶ "],
"test_tokens": ["token1", "token3", " ¶ "],
"changes": [
{
"type": "replace",
"source_text": "token2",
"test_text": "token3",
"source_position": [1, 2],
"test_position": [1, 2],
"source_char_position": [7, 13],
"test_char_position": [7, 13]
}
],
"stats": {
"total_changes": 1,
"deletions": 0,
"insertions": 0,
"replacements": 1,
"longest_change_length": 6,
"shortest_change_length": 6,
"average_change_length": 6.0,
"change_ratio": 0.15,
"chars_added": 6,
"chars_deleted": 6,
"chars_net_change": 0,
"levenshtein_distance": 3
}
}
```
### Field Descriptions
[Section titled “Field Descriptions”](#field-descriptions)
#### Root Fields
[Section titled “Root Fields”](#root-fields)
* **`source`** (string): Original text
* **`test`** (string): Modified text
* **`source_tokens`** (array of strings): Tokenized source text (`¶` marks paragraph boundaries)
* **`test_tokens`** (array of strings): Tokenized test text
#### Change Object
[Section titled “Change Object”](#change-object)
* **`type`** (string): Operation type - `"insert"`, `"delete"`, or `"replace"`
* **`source_text`** (string | null): Original text (null for insertions)
* **`test_text`** (string | null): Modified text (null for deletions)
* **`source_position`** (array of 2 ints | null): Token position `[start, end]` in source (null for insertions)
* **`test_position`** (array of 2 ints | null): Token position `[start, end]` in test (null for deletions)
* **`source_char_position`** (array of 2 ints | null): Character position `[start, end]` in source
* **`test_char_position`** (array of 2 ints | null): Character position `[start, end]` in test
#### Stats Object
[Section titled “Stats Object”](#stats-object)
* **`total_changes`** (int): Total number of change operations
* **`deletions`** (int): Count of deletion operations
* **`insertions`** (int): Count of insertion operations
* **`replacements`** (int): Count of replacement operations
* **`longest_change_length`** (int): Length of longest change in characters
* **`shortest_change_length`** (int): Length of shortest change in characters
* **`average_change_length`** (float): Mean change length in characters
* **`change_ratio`** (float): Percentage of text modified (0.0 to 1.0)
* **`chars_added`** (int): Total characters added (insertions + replacement additions)
* **`chars_deleted`** (int): Total characters deleted (deletions + replacement deletions)
* **`chars_net_change`** (int): Net character change (added - deleted)
* **`levenshtein_distance`** (int | null): Edit distance (null if Levenshtein library not installed)
***
## Decision Matrix
[Section titled “Decision Matrix”](#decision-matrix)
| Use Case | Recommended Format | CLI Command | Reason |
| ---------------------- | -------------------- | ------------------------------------------------------------- | -------------------------------------- |
| **AI agents** | JSON | `redlines file1 file2` | Simplest invocation, structured output |
| **CI/CD check** | JSON | `redlines file1 file2` or `redlines json file1 file2 --quiet` | Parseable, scriptable, exit codes |
| **Human review** | Markdown (GHFM) | `redlines markdown file1 file2 --markdown-style ghfm` | GitHub-compatible rendering |
| **Terminal display** | Rich text | `redlines text file1 file2` | Colored, formatted output |
| **Metrics collection** | Stats | `redlines stats file1 file2 --quiet` | Log-friendly plain text |
| **Automation/scripts** | JSON + exit codes | `redlines file1 file2` | Exit code 0=changes, 1=no changes |
| **HTML report** | Markdown (none) | Programmatic API | Clean HTML tags for styling |
| **Documentation** | Markdown (streamlit) | Programmatic API | Streamlit-compatible markup |
### Exit Code Usage
[Section titled “Exit Code Usage”](#exit-code-usage)
```bash
# Check if files differ (useful in CI/CD) - command-less invocation
if redlines file1.txt file2.txt > /dev/null 2>&1; then
echo "Exit code 0: Changes detected"
else
exitcode=$?
if [ $exitcode -eq 1 ]; then
echo "Exit code 1: No changes (files identical)"
else
echo "Exit code 2: Error occurred"
fi
fi
# Or use the stats command for more verbose output
if redlines stats file1.txt file2.txt --quiet; then
echo "Exit code 0: Changes detected"
else
echo "Exit code 1: No changes or error occurred"
fi
```
***
## Programmatic API
[Section titled “Programmatic API”](#programmatic-api)
### Core Classes
[Section titled “Core Classes”](#core-classes)
#### `Redlines` Class
[Section titled “Redlines Class”](#redlines-class)
```python
from redlines import Redlines
# Create instance
diff = Redlines(
source="original text",
test="modified text",
processor=None, # Optional: Custom processor (default: WholeDocumentProcessor)
markdown_style="red_green" # Optional: Markdown style
)
# Or compare later
diff = Redlines("original text")
result = diff.compare("modified text")
```
#### Key Properties and Methods
[Section titled “Key Properties and Methods”](#key-properties-and-methods)
```python
# Get changes (excludes "equal" operations)
changes: list[Redline] = diff.changes
redlines: list[Redline] = diff.redlines # Alias for changes
# Filter changes by operation
insertions = diff.get_changes(operation="insert")
deletions = diff.get_changes(operation="delete")
replacements = diff.get_changes(operation="replace")
all_changes = diff.get_changes() # Same as .changes
# Get statistics
stats: Stats = diff.stats()
# Get opcodes (like difflib)
opcodes: list[tuple] = diff.opcodes # [(operation, i1, i2, j1, j2), ...]
# Output formats
markdown: str = diff.output_markdown
rich_text: Text = diff.output_rich
json_str: str = diff.output_json(pretty=False)
```
### `Redline` Dataclass
[Section titled “Redline Dataclass”](#redline-dataclass)
```python
from redlines.processor import Redline
# Structure of a Redline object
@dataclass
class Redline:
operation: Literal["delete", "insert", "replace"]
source_text: str | None
test_text: str | None
source_position: tuple[int, int] | None # (start, end) token indices
test_position: tuple[int, int] | None
# Example usage
for change in diff.changes:
if change.operation == "replace":
print(f"Line {change.source_position}: {change.source_text} → {change.test_text}")
```
### `Stats` Dataclass
[Section titled “Stats Dataclass”](#stats-dataclass)
```python
from redlines.processor import Stats
# Structure of a Stats object
@dataclass
class Stats:
total_changes: int
deletions: int
insertions: int
replacements: int
longest_change_length: int
shortest_change_length: int
average_change_length: float
change_ratio: float # 0.0 to 1.0
chars_added: int
chars_deleted: int
chars_net_change: int
levenshtein_distance: int | None # None if library not installed
# Example usage
stats = diff.stats()
print(f"Modified {stats.change_ratio:.1%} of the document")
print(f"Net change: {stats.chars_net_change:+d} characters")
```
***
## Error Handling
[Section titled “Error Handling”](#error-handling)
### Common Errors and Solutions
[Section titled “Common Errors and Solutions”](#common-errors-and-solutions)
#### 1. File Not Found (CLI)
[Section titled “1. File Not Found (CLI)”](#1-file-not-found-cli)
```bash
$ redlines json nonexistent.txt other.txt
# Error: Failed to read file 'nonexistent.txt': [Errno 2] No such file or directory
# Solution: Check file paths
if [ -f "file.txt" ]; then
redlines json file.txt other.txt
else
echo "File not found"
fi
```
#### 2. Encoding Errors (CLI)
[Section titled “2. Encoding Errors (CLI)”](#2-encoding-errors-cli)
```bash
# Error: Failed to read file 'file.txt': File encoding is not UTF-8
# Solution: Convert file to UTF-8
iconv -f ISO-8859-1 -t UTF-8 file.txt > file_utf8.txt
redlines json file_utf8.txt other.txt
```
#### 3. Invalid Operation Filter
[Section titled “3. Invalid Operation Filter”](#3-invalid-operation-filter)
```python
from redlines import Redlines
diff = Redlines(source, test)
try:
changes = diff.get_changes(operation="invalid")
except ValueError as e:
print(f"Error: {e}") # Error: Invalid operation: invalid
# Valid operations: "insert", "delete", "replace", or None
```
#### 4. Missing Optional Dependencies
[Section titled “4. Missing Optional Dependencies”](#4-missing-optional-dependencies)
```python
from redlines.processor import LEVENSHTEIN_AVAILABLE
if not LEVENSHTEIN_AVAILABLE:
print("Levenshtein library not installed - distance will be None")
# Install with: pip install redlines[levenshtein]
# Stats will still work, but levenshtein_distance will be None
stats = diff.stats()
if stats.levenshtein_distance is not None:
print(f"Edit distance: {stats.levenshtein_distance}")
```
#### 5. Empty or Identical Files
[Section titled “5. Empty or Identical Files”](#5-empty-or-identical-files)
```python
from redlines import Redlines
# Empty files
diff = Redlines("", "")
stats = diff.stats()
assert stats.total_changes == 0
assert stats.change_ratio == 0.0
# Identical files
diff = Redlines("Hello world", "Hello world")
stats = diff.stats()
assert stats.total_changes == 0
# CLI exit code will be 1 (no changes)
```
### Defensive Programming Pattern
[Section titled “Defensive Programming Pattern”](#defensive-programming-pattern)
```python
from pathlib import Path
from redlines import Redlines
import json
def safe_compare_files(file1: str, file2: str) -> dict:
"""Safely compare two files with comprehensive error handling."""
try:
# Validate files exist
path1, path2 = Path(file1), Path(file2)
if not path1.exists():
return {"error": f"File not found: {file1}"}
if not path2.exists():
return {"error": f"File not found: {file2}"}
# Read files
try:
source = path1.read_text(encoding="utf-8")
test = path2.read_text(encoding="utf-8")
except UnicodeDecodeError as e:
return {"error": f"Encoding error: {e}"}
# Compare
diff = Redlines(source, test)
stats = diff.stats()
return {
"success": True,
"file1": file1,
"file2": file2,
"total_changes": stats.total_changes,
"change_ratio": stats.change_ratio,
"has_changes": stats.total_changes > 0
}
except Exception as e:
return {"error": f"Unexpected error: {e}"}
# Usage
result = safe_compare_files("old.txt", "new.txt")
if "error" in result:
print(f"Error: {result['error']}")
else:
print(f"Changes: {result['total_changes']}")
```
***
## Performance Guidelines
[Section titled “Performance Guidelines”](#performance-guidelines)
### Processor Comparison
[Section titled “Processor Comparison”](#processor-comparison)
| Processor | Speed | Use Case | Python Version |
| ------------------------------------ | ------------------ | --------------------------------------------- | -------------- |
| **WholeDocumentProcessor** (default) | Fast (5-6x faster) | Simple documents, speed critical | 3.10+ |
| **NupunktProcessor** | Slower | Legal/technical docs, sentence-level accuracy | 3.11+ |
### Speed Benchmarks
[Section titled “Speed Benchmarks”](#speed-benchmarks)
| File Size | WholeDocument | Nupunkt | Difference |
| --------- | ------------- | --------- | ---------- |
| 400 chars | 0.19 ms | 0.40 ms | 2.1x |
| 4 KB | 0.39 ms | 2.34 ms | 6.0x |
| 40 KB | 3.85 ms | 25.14 ms | 6.5x |
| 400 KB | 37.92 ms | 241.68 ms | 6.4x |
**Throughput:**
* WholeDocumentProcessor: \~10 million chars/second
* NupunktProcessor: \~1.6 million chars/second
### When to Use Each Processor
[Section titled “When to Use Each Processor”](#when-to-use-each-processor)
#### Use WholeDocumentProcessor (Default) When:
[Section titled “Use WholeDocumentProcessor (Default) When:”](#use-wholedocumentprocessor-default-when)
* Processing large volumes of documents
* Speed is critical
* Documents are simple without complex punctuation
* Paragraph-level granularity is sufficient
#### Use NupunktProcessor When:
[Section titled “Use NupunktProcessor When:”](#use-nupunktprocessor-when)
```python
from redlines import Redlines
from redlines.processor import NupunktProcessor
processor = NupunktProcessor()
diff = Redlines(source, test, processor=processor)
```
* Working with legal or technical documents
* Need sentence-level granularity
* Documents contain many abbreviations (Dr., Mr., etc.)
* URLs, email addresses, or decimal numbers in text
* Performance overhead (2-6x) is acceptable
Sentence mode preserves the input’s paragraph boundaries (fixed in 0.6.2): sentences are anchored within their paragraph, so output is not reflowed one sentence per paragraph.
### Performance Tips
[Section titled “Performance Tips”](#performance-tips)
1. **Reuse Redlines instance for multiple comparisons:**
```python
diff = Redlines(source)
result1 = diff.compare(test1)
result2 = diff.compare(test2) # Faster than creating new instance
```
2. **Use CLI for one-off comparisons:**
```bash
# CLI is optimized for single comparisons
redlines json file1.txt file2.txt
```
3. **Batch processing pattern:**
```python
# Process multiple files efficiently
from concurrent.futures import ThreadPoolExecutor
def compare_file_pair(files):
file1, file2 = files
return Redlines(
Path(file1).read_text(),
Path(file2).read_text()
).stats()
with ThreadPoolExecutor(max_workers=4) as executor:
results = executor.map(compare_file_pair, file_pairs)
```
***
## Integration Examples
[Section titled “Integration Examples”](#integration-examples)
### Example 1: Pre-commit Hook
[Section titled “Example 1: Pre-commit Hook”](#example-1-pre-commit-hook)
.git/hooks/pre-commit
```bash
#!/bin/bash
# Compare staged files with HEAD
for file in $(git diff --cached --name-only --diff-filter=M); do
if [ -f "$file" ]; then
# Get old version
git show HEAD:"$file" > /tmp/old_version
# Compare with new version
if ! redlines stats /tmp/old_version "$file" --quiet > /dev/null; then
echo "Error comparing $file"
exit 1
fi
echo "✓ $file: Changes validated"
rm /tmp/old_version
fi
done
exit 0
```
### Example 2: CI/CD Check
[Section titled “Example 2: CI/CD Check”](#example-2-cicd-check)
.github/workflows/check-docs.yml
```yaml
name: Check Documentation Changes
on: [pull_request]
jobs:
check-docs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install redlines
run: pip install redlines
- name: Check README changes
run: |
git show main:README.md > old_readme.md
if redlines stats old_readme.md README.md --quiet; then
echo "README has changes"
redlines markdown old_readme.md README.md --markdown-style ghfm >> $GITHUB_STEP_SUMMARY
else
echo "README unchanged"
fi
```
### Example 3: Batch Report Generator
[Section titled “Example 3: Batch Report Generator”](#example-3-batch-report-generator)
```python
#!/usr/bin/env python3
"""Generate HTML report comparing two directories."""
from pathlib import Path
from redlines import Redlines
import json
def generate_report(dir1: Path, dir2: Path, output: Path):
"""Generate HTML diff report for all files in two directories."""
results = []
for file1 in sorted(dir1.glob("**/*.txt")):
file2 = dir2 / file1.relative_to(dir1)
if not file2.exists():
continue
diff = Redlines(
file1.read_text(),
file2.read_text(),
markdown_style="none"
)
stats = diff.stats()
if stats.total_changes > 0:
results.append({
"file": str(file1.relative_to(dir1)),
"stats": {
"changes": stats.total_changes,
"ratio": f"{stats.change_ratio:.1%}",
"added": stats.chars_added,
"deleted": stats.chars_deleted,
},
"diff": diff.output_markdown
})
# Generate HTML
html = f"""
Diff Report
"""
html += """
"""
output.write_text(html)
print(f"Report generated: {output}")
if __name__ == "__main__":
import sys
if len(sys.argv) != 4:
print("Usage: generate_report.py ")
sys.exit(1)
generate_report(Path(sys.argv[1]), Path(sys.argv[2]), Path(sys.argv[3]))
```
### Example 4: Test Suite Integration
[Section titled “Example 4: Test Suite Integration”](#example-4-test-suite-integration)
```python
import pytest
from redlines import Redlines
def test_function_preserves_behavior():
"""Test that refactored code produces same output."""
# Original implementation output
original_output = "Hello World"
# New implementation output
new_output = my_refactored_function()
# Compare
diff = Redlines(original_output, new_output)
stats = diff.stats()
# Assert no changes
assert stats.total_changes == 0, f"Output changed: {diff.output_markdown}"
def test_documentation_completeness():
"""Ensure all API changes are documented."""
old_docs = Path("docs/api_v1.md").read_text()
new_docs = Path("docs/api_v2.md").read_text()
diff = Redlines(old_docs, new_docs)
stats = diff.stats()
# Ensure significant documentation updates
assert stats.change_ratio > 0.05, "API changed but docs not updated enough"
```
### Example 5: Document Change Tracker
[Section titled “Example 5: Document Change Tracker”](#example-5-document-change-tracker)
```python
#!/usr/bin/env python3
"""Track document changes over time."""
import json
from pathlib import Path
from datetime import datetime
from redlines import Redlines
class DocumentTracker:
def __init__(self, storage_dir: Path):
self.storage_dir = storage_dir
self.storage_dir.mkdir(exist_ok=True)
self.history_file = storage_dir / "history.json"
self.history = self._load_history()
def _load_history(self):
if self.history_file.exists():
return json.loads(self.history_file.read_text())
return {"documents": {}}
def _save_history(self):
self.history_file.write_text(json.dumps(self.history, indent=2))
def track_change(self, doc_name: str, content: str):
"""Track a change to a document."""
doc_history = self.history["documents"].setdefault(doc_name, {
"versions": [],
"total_changes": 0
})
# Get previous version
versions = doc_history["versions"]
if versions:
prev_version = (self.storage_dir / versions[-1]["file"]).read_text()
# Compare
diff = Redlines(prev_version, content)
stats = diff.stats()
if stats.total_changes == 0:
print(f"{doc_name}: No changes")
return
# Record change
version_file = f"{doc_name}_{len(versions)}.txt"
(self.storage_dir / version_file).write_text(content)
versions.append({
"file": version_file,
"timestamp": datetime.now().isoformat(),
"changes": stats.total_changes,
"change_ratio": stats.change_ratio,
"chars_net_change": stats.chars_net_change
})
doc_history["total_changes"] += stats.total_changes
print(f"{doc_name}: {stats.total_changes} changes ({stats.change_ratio:.1%})")
else:
# First version
version_file = f"{doc_name}_0.txt"
(self.storage_dir / version_file).write_text(content)
versions.append({
"file": version_file,
"timestamp": datetime.now().isoformat(),
"changes": 0,
"change_ratio": 0.0,
"chars_net_change": 0
})
print(f"{doc_name}: Initial version tracked")
self._save_history()
def get_diff(self, doc_name: str, version1: int, version2: int) -> str:
"""Get diff between two versions."""
versions = self.history["documents"][doc_name]["versions"]
content1 = (self.storage_dir / versions[version1]["file"]).read_text()
content2 = (self.storage_dir / versions[version2]["file"]).read_text()
diff = Redlines(content1, content2)
return diff.output_markdown
# Usage
tracker = DocumentTracker(Path(".document_history"))
tracker.track_change("README.md", Path("README.md").read_text())
```
***
## Additional Resources
[Section titled “Additional Resources”](#additional-resources)
* **Full Documentation:** [this site](/redlines/)
* **GitHub Repository:**
* **Example Scripts:** [examples/](https://github.com/houfu/redlines/tree/main/examples) directory
* **Demo Project:** [redlines-textual](https://github.com/houfu/redlines-textual)
***
## Quick Reference Card
[Section titled “Quick Reference Card”](#quick-reference-card)
```bash
# CLI - Command-less (recommended for agents)
redlines "source" "test" # JSON output
redlines --pretty "source" "test" # Pretty JSON
redlines file1.txt file2.txt # Works with files
# CLI - Traditional commands (for specific output formats)
redlines text SOURCE TEST # Rich terminal display
redlines json SOURCE TEST --pretty # JSON with formatting
redlines markdown SOURCE TEST -m ghfm # Markdown output
redlines stats SOURCE TEST --quiet # Statistics only
```
```python
# Python API
from redlines import Redlines
# Compare
diff = Redlines(source, test)
# Get changes
all_changes = diff.changes
insertions = diff.get_changes(operation="insert")
# Get stats
stats = diff.stats()
# Output
markdown = diff.output_markdown
json_str = diff.output_json(pretty=True)
rich_text = diff.output_rich
```
***
**Last Updated:** 2025-10-22 **Version:** 0.6.0+ **Python:** 3.10+
# redlines
> Compare two texts and see what changed — as markdown, rich terminal output, or JSON an agent can parse.
```bash
pip install redlines
redlines "the quick brown fox" "the quick red fox"
```
Quickstart
Install, compare two texts from the CLI or from Python, and find your way around the output formats.
[Start here](/redlines/start/quickstart/)
Agent integration
Invocation, the JSON structure, exit codes, error handling and integration patterns for calling redlines from agents and automation.
[Agent guide](/redlines/guides/agent-guide/) · [llms.txt](/redlines/llms.txt)
API reference
Every class and method, generated from the docstrings that ship with the package.
[API reference](/redlines/api/)
Why it works this way
The decisions behind redlines 1.0 — what was rejected, and the conditions under which each should be revisited.
[Decision records](/redlines/project/adr/) · [Roadmap](/redlines/project/roadmap/)
The 1.0 documentation is still being written
The pages here cover the current release. The schemas, profile guide and benchmark report arrive with 1.0 — see the [roadmap](/redlines/project/roadmap/).
# Architecture decision records
> Why each design decision was made, what was rejected, and when to revisit it.
This directory records the decisions behind redlines 1.0 — the reasoning, the alternatives that were considered and rejected, and the conditions under which each decision should be revisited.
## Why these exist
[Section titled “Why these exist”](#why-these-exist)
[`docs/PRD.md`](/redlines/project/prd/) says what the product is. [`ROADMAP.md`](/redlines/project/roadmap/) says which release each feature is in. Neither is a good place to keep *why* a choice was made, because both get rewritten as the product changes, and rewriting erases the reasoning. An ADR is written once, is never edited except to change its status, and is superseded rather than updated. In a year, when someone (including us) asks “why doesn’t redlines write DOCX?”, the answer should be a file, not a memory.
## Conventions
[Section titled “Conventions”](#conventions)
* One decision per file, numbered in the order they were taken: `NNNN-short-title.md`.
* **Status** is one of: Proposed (recommended, not yet agreed), Accepted, Superseded by ADR-NNNN, or Deprecated.
* An accepted ADR is not edited when we change our minds. A new ADR is written that supersedes it, and the old one’s status is changed to point at the new one. The trail of superseded decisions is the point.
* Each ADR ends with **Revisit when**: the concrete signal that should make us reopen it. A decision with no revisit condition is usually a decision that was never really made.
## Index
[Section titled “Index”](#index)
| ADR | Title | Status |
| --------------------------------------------------------------------- | ---------------------------------------------------------------------------- | -------- |
| [0001](/redlines/project/adr/0001-format-neutral-structural-engine/) | Build a format-neutral structural comparison engine | Accepted |
| [0002](/redlines/project/adr/0002-primary-persona-agent-developers/) | Optimise for LLM and agent pipeline developers | Accepted |
| [0003](/redlines/project/adr/0003-compatibility-facade/) | Ship 1.0 with a compatibility facade, not a clean break | Accepted |
| [0004](/redlines/project/adr/0004-stdlib-core-optional-extras/) | Keep a stdlib core with optional extras | Accepted |
| [0005](/redlines/project/adr/0005-minimal-core-open-semantic-layer/) | Minimal structural core with an open semantic layer | Accepted |
| [0006](/redlines/project/adr/0006-structure-profiles/) | Drive readers with declarative structure profiles | Accepted |
| [0007](/redlines/project/adr/0007-no-ocr-no-llm-in-library/) | No OCR and no LLM calls inside the library | Accepted |
| [0008](/redlines/project/adr/0008-multi-pass-block-alignment/) | Align blocks in explainable passes | Accepted |
| [0009](/redlines/project/adr/0009-moves-before-splits/) | Detect moves and renumbering in 1.0; splits and merges later | Accepted |
| [0010](/redlines/project/adr/0010-keep-difflib-for-leaf-diffs/) | Keep difflib as the leaf differ | Accepted |
| [0011](/redlines/project/adr/0011-json-canonical-annotated-renderer/) | JSON as the canonical change format, with an annotated renderer | Accepted |
| [0012](/redlines/project/adr/0012-html-like-addresses/) | Address blocks with HTML-like paths and document labels | Accepted |
| [0013](/redlines/project/adr/0013-the-1-0-slice-text-and-markdown/) | Limit 1.0 to plain text and markdown | Accepted |
| [0014](/redlines/project/adr/0014-no-ooxml-writing/) | Never write OOXML; delegate tracked changes to appliers | Accepted |
| [0015](/redlines/project/adr/0015-verify-mode-in-1-0/) | Ship verify mode in 1.0 | Accepted |
| [0016](/redlines/project/adr/0016-summary-renderer-in-core/) | Implement the summary renderer in core, deliver it with MCP | Accepted |
| [0017](/redlines/project/adr/0017-separate-mcp-package/) | Publish the MCP server as a separate package | Accepted |
| [0018](/redlines/project/adr/0018-mcp-tools-prompts-resources/) | Use MCP tools, prompts and resources so models can author profiles | Accepted |
| [0019](/redlines/project/adr/0019-client-side-demo-site/) | Run the demo site entirely in the browser | Accepted |
| [0020](/redlines/project/adr/0020-mcp-before-site/) | Ship the MCP server before the site | Accepted |
| [0021](/redlines/project/adr/0021-alignment-benchmark/) | Make alignment quality measurable with our own benchmark | Accepted |
| [0022](/redlines/project/adr/0022-keep-the-name/) | Keep the name redlines | Accepted |
| [0023](/redlines/project/adr/0023-python-support-and-typing/) | Keep Python 3.10+ and strict typing | Accepted |
| [0024](/redlines/project/adr/0024-no-formatting-change-detection/) | No inline formatting change detection in 1.0 | Accepted |
| [0025](/redlines/project/adr/0025-cli-as-thin-skin/) | Treat the CLI and the MCP server as two skins over one function table | Accepted |
| [0026](/redlines/project/adr/0026-docs-site-on-astro-starlight/) | Publish the documentation with Astro Starlight, in the same site as the demo | Accepted |
| [0027](/redlines/project/adr/0027-agent-docs-machine-surface/) | Serve agents with a machine surface and a contract page, not a guide | Accepted |
## Evidence base
[Section titled “Evidence base”](#evidence-base)
Most of these decisions rest on a survey of the 2026 redlining landscape: [competitive-landscape-2026-08.md](https://github.com/houfu/redlines/blob/main/docs/competitive-landscape-2026-08.md). Where an ADR cites a competitor’s behaviour, that file has the source.
# ADR-0001: Build a format-neutral structural comparison engine
**Status:** Accepted **Date:** 2026-08-26 **Deciders:** houfu
## Context
[Section titled “Context”](#context)
redlines was written in 2023 on a design that flattens a document into one token stream, marks paragraph boundaries with a `¶` token, runs one `difflib.SequenceMatcher`, and renders the opcodes. That design succeeded on its own terms — roughly 3.7M PyPI downloads, adoption by the DeepLearning.AI prompt engineering course — and has not changed materially since.
By 2026 the field around it has moved. Two camps have formed. *Comparers* take two DOCX files and emit a DOCX with native Word tracked changes: python-redlines/Docxodus, SuperDoc’s Document Engine, stemma, safe-docx, folio, jubarte, and commercially Litera and Draftable. *Appliers* take edits an LLM has decided on and project them into a DOCX as `w:ins`/`w:del`: adeu, superdoc-redlines, docx-mcp, Anthropic’s docx skill.
Three observations about that field matter. Every one of them is bound to OOXML — the input is always a .docx. All of them treat the diff algorithm as a commodity, using diff-match-patch or Myers over words. And none of them exposes a change model above the level of “these words changed in paragraph 37”; SuperDoc’s own documentation calls its diff payload “opaque and intended for replay, not semantic inspection”, and python-redlines returns a byte blob and a one-line revision count.
Meanwhile the flat spine has cliffs that show what it costs to encode structure as a character in a token stream: `autojunk=True` silently reports an entire 1,050-token repetitive schedule as replaced when two words change; adjacent edits separated by punctuation count as two changes; sentence mode discards paragraph structure entirely.
## Decision
[Section titled “Decision”](#decision)
redlines 1.0 becomes a **format-neutral structural comparison engine**. Documents are parsed into trees of blocks; blocks are aligned before the text inside them is diffed; the result is a change tree with addresses. DOCX input and tracked-changes output are borrowed from other projects rather than built (see ADR-0013 and ADR-0014).
## Alternatives considered
[Section titled “Alternatives considered”](#alternatives-considered)
**A DOCX-first structural redliner** — own the whole OOXML pipeline, parser and writer, and compete head-on with Docxodus, stemma and safe-docx. Rejected: those are funded or full-time efforts with a two-year head start on OOXML fidelity, and fidelity is measured on a pixel benchmark where a text-neutral engine cannot win. It would also mean competing on the axis where redlines has no advantage while abandoning the one where it has a large one.
**An incremental fix to the flat spine** — patch autojunk, add semantic cleanup, add a DOCX reader as a `Document` subclass, enrich the JSON. Rejected: it fixes the symptoms without addressing the cause, and leaves redlines a slightly better version of a 2023 design in a field that has moved. It would not let anyone ask about clause 7.2, distinguish a move from a delete-plus-insert, or verify the scope of an edit.
## Consequences
[Section titled “Consequences”](#consequences)
Positive: it occupies an unoccupied slot. Nothing open-source does format-neutral hierarchical document comparison with an inspectable change model. It keeps the property that made redlines popular — anything that can be turned into text can be compared — and adds the structure that makes the output meaningful. It positions redlines as the comparison engine *in the middle* of the new stack, fed by readers and consumed by appliers, rather than as a rival at either end.
Negative: it is a large rewrite of the core, with a compatibility burden (ADR-0003). It requires solving block alignment, which is genuinely hard and which nobody in the open-source field has solved well. And it means accepting a poor score on the one public benchmark that exists (`neurotic_docx_bench` measured redlines 0.6.1 at 45.9 mean, bottom of the table), because that benchmark measures visual fidelity to Word’s markup — which is not what this engine is for.
## Revisit when
[Section titled “Revisit when”](#revisit-when)
If block alignment quality (ADR-0021) cannot reach the targets on real documents after honest effort, the structural thesis is wrong and the incremental path becomes the better one. If a well-funded project ships format-neutral structural diffing with an open change model before 1.0, reconsider whether to build or to contribute.
## Related
[Section titled “Related”](#related)
ADR-0002, ADR-0005, ADR-0013, ADR-0014, ADR-0021. Evidence: `docs/competitive-landscape-2026-08.md`.
# ADR-0002: Optimise for LLM and agent pipeline developers
**Status:** Accepted **Date:** 2026-08-26 **Deciders:** houfu
## Context
[Section titled “Context”](#context)
Two plausible audiences exist for a structural redliner. Legal engineers comparing two versions of a contract, who want clause-level diffs and Word-compatible output. And developers building LLM and agent pipelines, who need to know what a model changed, where, and whether that was all it was supposed to change.
The download curve says which audience redlines already has. Roughly 3.7M lifetime PyPI downloads against \~158 GitHub stars is the signature of course-and-notebook usage, not of a tool people build products on; the README itself cites the DeepLearning.AI course lesson that drove it. Only two dependent packages are registered on libraries.io, so almost all use is direct.
The legal-comparison audience is well served by incumbents with a decade of OOXML fidelity work behind them, and by a 2026 cohort of Rust and TypeScript tools built specifically for it.
Separately, an incumbent has validated the agent-facing format: Draftable shipped an “AI-Ready Redline Export” in its February–March 2026 release, a deterministic plain-text change summary designed to be pasted into ChatGPT, Claude, Harvey or Legora. That is redlines’ natural output, sold as a feature.
## Decision
[Section titled “Decision”](#decision)
The primary user is the **LLM/agent pipeline developer**. Feature sequencing, output formats, documentation and the MCP surface are designed for that person first. The document engineer is a secondary user who drives the block model and alignment requirements but does not drive OOXML fidelity work.
## Alternatives considered
[Section titled “Alternatives considered”](#alternatives-considered)
**Legal engineers first.** Rejected: it points the roadmap at DOCX fidelity, where the competition is strongest and redlines is weakest, and it abandons the existing user base.
**Both equally.** Rejected as a non-decision. Serving both equally in practice means DOCX work competes with agent work for the same hours, and DOCX work always looks more urgent because it is more concrete.
## Consequences
[Section titled “Consequences”](#consequences)
Positive: it makes several later decisions easy. JSON and a deterministic summary are first-class; verify mode (ADR-0015) becomes a headline feature; the MCP server (ADR-0017, ADR-0018) is a deliverable rather than an afterthought; DOCX can be deferred without guilt.
Negative: the demo site cannot accept a Word file at 1.0 (ADR-0013), which is the first thing a lawyer will try. Some legal users will bounce. We accept that in exchange for a coherent 1.0.
## Revisit when
[Section titled “Revisit when”](#revisit-when)
If the MCP server and agent-facing outputs fail to find users within two quarters of release while inbound requests are dominated by “can it read my Word documents”, the persona choice was wrong and the roadmap should be re-ordered around DOCX.
## Related
[Section titled “Related”](#related)
ADR-0013, ADR-0015, ADR-0017, ADR-0018.
# ADR-0003: Ship 1.0 with a compatibility facade, not a clean break
**Status:** Accepted **Date:** 2026-08-26 **Deciders:** houfu
## Context
[Section titled “Context”](#context)
The 0.6 public API is small and very widely used: `Redlines(source, test).output_markdown`, `compare()`, `output_json()`, `changes`, `get_changes()`, `stats()`, `opcodes`, the processor classes, `Document` and `PlainTextFile`. The single most-copied line in the ecosystem is the course example, which asserts an exact output string.
The structural engine (ADR-0001) needs a different shape: readers, profiles, a comparison object, a change tree. Bolting that onto the existing class would distort both.
## Decision
[Section titled “Decision”](#decision)
redlines 1.0 introduces a new top-level API for the structural engine, and keeps the entire 0.6 surface working unchanged, reimplemented over the new core as a facade that builds a one-block-per-paragraph tree. Deprecation warnings where something is superseded; no removals in 1.0.
## Alternatives considered
[Section titled “Alternatives considered”](#alternatives-considered)
**Strictly additive** — no new top-level API, everything hangs off the existing `Redlines` class and processor pattern. Rejected: the class is built around a source-and-test pair and a flat opcode list; expressing readers, profiles, block trees and verify through it would produce a worse API for the new work and a confusing one for the old.
**Clean break** — 1.0 is a new API, 0.6.x stays on PyPI as the legacy line. Rejected: notebooks and course material do not get updated. A clean break turns 3.7M downloads of goodwill into a support burden and a fork of the documentation.
## Consequences
[Section titled “Consequences”](#consequences)
Positive: existing users upgrade without noticing. The 0.6 test suite becomes the compatibility contract, and passing it unmodified is a hard release criterion. It also forces the new core to be general enough to express the degenerate flat case, which is a useful design constraint.
Negative: the facade is real work (roughly a week) and real risk — re-implementing `output_markdown` over a tree can change whitespace or paragraph handling in edge cases the current tests do not cover. Mitigation: generate golden files from 0.6 across the README, course and issue examples *before* writing the facade. The package also carries two conceptual models for at least one major version, which costs documentation clarity.
## Revisit when
[Section titled “Revisit when”](#revisit-when)
At 2.0, when deprecated surfaces may be removed. If the golden-file work reveals that byte-identical output is impossible for some style, that specific case should get its own note rather than a blanket relaxation of the criterion.
## Related
[Section titled “Related”](#related)
ADR-0001, ADR-0010.
# ADR-0004: Keep a stdlib core with optional extras
**Status:** Accepted **Date:** 2026-08-26 **Deciders:** houfu
## Context
[Section titled “Context”](#context)
Part of redlines’ identity is that `pip install redlines` is instant and works anywhere — a notebook, a Lambda, a CI job, a WASM runtime. Its current dependencies are click, click-default-group, rich-click and rich; nupunkt and Levenshtein are already optional extras.
The competition sits at the other end. python-redlines extracts a self-contained .NET executable from a platform wheel to a cache directory and invokes it by subprocess. superdoc-redlines needs Node 18, jsdom and an AGPL-licensed editor. adeu needs Python 3.12+.
Meanwhile the structural engine has real reasons to want dependencies: rapidfuzz for alignment similarity, python-docx for DOCX, pypdf for PDF, a markdown parser, pydantic for schemas.
## Decision
[Section titled “Decision”](#decision)
The core stays on the standard library plus click and rich. Everything else is an optional extra that degrades gracefully when absent: `[fuzzy]` (rapidfuzz), `[nupunkt]`, `[levenshtein]`, and later `[docx]`, `[pdf]`. Nothing in the core may import an extra unconditionally.
## Alternatives considered
[Section titled “Alternatives considered”](#alternatives-considered)
**Stdlib only, always** — no third-party dependencies even as extras. Rejected: alignment quality plausibly improves with rapidfuzz, and refusing a *optional* dependency is dogma rather than design. The zero-install property is about the default install, not about purity.
**Take dependencies freely** — rapidfuzz, python-docx, a markdown library and pydantic in core. Rejected: it is the simplest engineering and the worst positioning. It would cost the property that most distinguishes redlines from every competitor, and it would break the browser deployment (ADR-0019).
## Consequences
[Section titled “Consequences”](#consequences)
Positive: the default install stays small and portable; the browser build is possible at all; users install only what their formats need.
Negative: every capability that touches an extra needs a graceful-absence path and a test for it, which is ongoing tax. Alignment behaves differently with and without rapidfuzz — meaning results differ between a tuned local run and the browser, where rapidfuzz has no build. That divergence must be measured (ADR-0021) and, if large, either closed or disclosed.
## Revisit when
[Section titled “Revisit when”](#revisit-when)
If the difflib-ratio fallback proves materially worse than rapidfuzz on the benchmark, consider vendoring a small pure-Python similarity implementation into core so behaviour is uniform everywhere.
## Related
[Section titled “Related”](#related)
ADR-0008, ADR-0019, ADR-0021.
# ADR-0005: Minimal structural core with an open semantic layer
**Status:** Accepted **Date:** 2026-08-26 **Deciders:** houfu
## Context
[Section titled “Context”](#context)
A structural engine needs a document model. The obvious failure mode is to build a superset that DOCX, markdown and Akoma Ntoso all map into; such models grow OOXML-shaped, because OOXML is the most detailed of the three, and then every reader has to fill in fields it does not have.
The opposite failure is a model so thin that the output is no better than a flat diff with paragraph numbers. “Paragraph 14 was modified” is not meaningfully more useful to an LLM than a token span.
The insight that resolves this, raised during review: what makes the output valuable to a model is not *structure* but *meaning*. “The definition of Confidential Information was modified” and “a cross-reference to clause 7.2 was inserted” are statements a model can act on. Structure is how you find them; semantics is what you report.
## Decision
[Section titled “Decision”](#decision)
The block model has two layers.
A **minimal structural core**, with a closed vocabulary: `kind` (document, section, heading, paragraph, list\_item, table, row, cell, unknown), `text`, `label`, `level`, `path`, `children`. Format-specific detail lives in a free-form `attrs` and never in the core schema.
An **open semantic layer**, entirely optional: a `role` on blocks (title, recital, definition, clause, sub\_clause, schedule, signature, note, quote, code, boilerplate) and `spans` inside them (emphasis, defined\_term, cross\_reference, party, date, amount, citation). The vocabulary is recommended, not enforced. Roles and spans are assigned by a pluggable pass driven by a structure profile (ADR-0006).
Alignment and diffing operate on text only. Roles may break ties between otherwise equal fuzzy candidates, and nothing more. Change nodes carry the role of the block they affect and the span types touched, so summaries can speak semantically.
## Alternatives considered
[Section titled “Alternatives considered”](#alternatives-considered)
**Structural-only core.** Rejected on review: it would make redlines a better diff but not a different one, and semantics is the thing no competitor has.
**A format superset.** Rejected for the reasons above.
**Semantics driving alignment.** Rejected for 1.0: matching on roles as well as text makes alignment failures much harder to explain, and explainability is the point of ADR-0008. The tie-break is the one exception, and it is bounded.
## Consequences
[Section titled “Consequences”](#consequences)
Positive: readers stay cheap to write, because a reader that knows nothing semantic still produces a valid tree. The JSON schema stays stable while the semantic vocabulary evolves. And the output can say what changed in the document’s own terms.
Negative: an open vocabulary means two profiles can use different role names for the same thing, so downstream consumers cannot rely on a fixed set. We accept that; a recommended list plus documentation is the mitigation, and enforcing a closed vocabulary would defeat the purpose.
## Revisit when
[Section titled “Revisit when”](#revisit-when)
If consumers (our own renderers included) start needing guarantees about which roles exist, promote a small subset — probably `definition`, `clause`, `schedule` — to a documented guaranteed set while leaving the rest open.
## Related
[Section titled “Related”](#related)
ADR-0006, ADR-0007, ADR-0008, ADR-0011.
# ADR-0006: Drive readers with declarative structure profiles
**Status:** Accepted **Date:** 2026-08-26 **Deciders:** houfu
## Context
[Section titled “Context”](#context)
Turning plain text into a block tree means detecting labels, inferring hierarchy from them, attaching continuations, and recognising headings; then a semantic pass assigns roles and spans (ADR-0005). All of that is pattern matching, and the patterns differ by document family.
A Singapore statute numbers sections `4.—(1)`. A US contract uses `Section 4.1`. An EU regulation uses `Article 4(1)(a)`. An internal policy uses `4.1.2` with bold headings. A markdown draft from an LLM uses `##` and `1.` list items. Alpha and roman labels are ambiguous in isolation — `(i)` after `(h)` is alphabetic, after `7.2` it is roman — and are only resolvable from the surrounding numbering context, which is itself family-specific. Headings that reset numbering (Schedule, Annex, Part) are what make labels unambiguous again after a boundary, and which headings do that varies too.
Hard-coding one family’s patterns means being wrong for every other family, and being wrong silently.
The observation that settled this, raised during review: at some level it is the *user* who declares what structure they care about. The library’s job is to apply that declaration faithfully, not to guess.
## Decision
[Section titled “Decision”](#decision)
Readers are driven by a declarative **structure profile**: label patterns and their nesting precedence; which headings reset numbering; heading recognition rules; role assignment rules; span extractors.
Built-in profiles ship for `generic` (paragraphs only), `contract` and `markdown`, with `legislation` following. A profile is selected explicitly or, later, auto-selected by scoring a sample of the document, with the winner and confidence reported. Profiles can be loaded from a file or passed as a mapping, from Python, the CLI (`--profile`), the MCP tools and the site, so a profile written once is reusable everywhere.
The profile format is a design requirement, not just a documentation concern: it must be flat, plainly named and schema-published, such that a model given the schema and one worked example can write a valid profile for a new family in a single turn.
Every block records how it was recognised (`matched_by`) and a confidence; the tree reports how many blocks fell through to plain paragraphs. When nothing matches, the reader degrades to one block per paragraph and alignment still works.
## Alternatives considered
[Section titled “Alternatives considered”](#alternatives-considered)
**Hard-coded legal heuristics.** Rejected: right for one family, silently wrong for the rest, and every new family needs a release.
**Roles only from syntax that already carries them** (markdown headings, DOCX styles). Rejected: it gives up on plain text entirely, and plain text is what LLMs and PDFs produce.
**A model-backed structure pass.** Rejected, and forbidden by ADR-0007. It would make structure non-deterministic and uninspectable — the two properties that make this design defensible.
## Consequences
[Section titled “Consequences”](#consequences)
Positive: semantic understanding becomes *declared* — inspectable, portable, versionable, improvable by users without a release. It gives the MCP server something unique to do (ADR-0018): a model can author a profile in a loop without any model call inside the library. And it is a defensible position against both the OOXML-first tools, which have no notion of a clause, and any future LLM-heavy competitor, whose structure cannot be inspected.
Negative: the profile format is now on the critical path and is a genuine design task — get it wrong and every reader is awkward. Auto-selection is extra machinery. Users with unusual documents must write a profile, which is a real barrier even if a model can help. And a wrong profile produces a confidently wrong tree, which is why `matched_by`, confidence and fallback counts are mandatory rather than nice to have.
## Revisit when
[Section titled “Revisit when”](#revisit-when)
If in practice everyone uses the built-in profiles unmodified, the format is over-engineered and could be simplified. If profiles proliferate and diverge, consider a shared community registry.
## Related
[Section titled “Related”](#related)
ADR-0005, ADR-0007, ADR-0013, ADR-0018.
# ADR-0007: No OCR and no LLM calls inside the library
**Status:** Accepted **Date:** 2026-08-26 **Deciders:** houfu
## Context
[Section titled “Context”](#context)
Once the model carries semantic roles (ADR-0005) and structure is inferred by heuristics (ADR-0006), there is a standing temptation to improve both with a model call: ask an LLM which blocks are definitions, or run OCR so scanned PDFs can be compared. Both would raise measured accuracy on hard inputs.
The concern raised during review was precisely this: that supporting PDF and DOCX would drag OCR and LLM requirements into a library whose top requirements are semantic.
## Decision
[Section titled “Decision”](#decision)
The library never calls an OCR engine and never calls an LLM. The semantic pass is deterministic heuristics over text, always. The PDF reader, when it arrives, extracts embedded text only and flags the resulting structure as inferred; a scanned PDF with no text layer is reported as unreadable, not OCR’d.
Both the semantic pass and the reader interface are pluggable, so anyone who wants a model-backed pass can write one *outside* the library and register it.
## Alternatives considered
[Section titled “Alternatives considered”](#alternatives-considered)
**An optional LLM-backed semantic pass shipped in the library, off by default.** Rejected: “off by default” erodes. Once it exists it becomes the recommended path, then the tested path, and the deterministic path rots. It would also put an API key, a network call and a bill inside a library people run in notebooks and CI.
**Optional OCR via an extra.** Rejected for the same reason plus a practical one: OCR quality dominates every downstream result, so the library’s measured behaviour would become a measure of the OCR engine.
## Consequences
[Section titled “Consequences”](#consequences)
Positive: results are reproducible, testable and free. The same inputs give the same output today and in a year, which is what makes verify mode (ADR-0015) trustworthy and golden-file tests possible. It keeps the browser deployment viable (no network). And it draws a clean line for contributors about what belongs inside.
Negative: accuracy on messy inputs is capped by what heuristics can do, and we will lose comparisons against tools that do use models. Scanned documents are simply out of scope. We accept both; the benchmark (ADR-0021) exists so the ceiling is known rather than felt.
## Revisit when
[Section titled “Revisit when”](#revisit-when)
Not lightly. If a model-backed pass becomes essential, the right shape is a separate companion package that produces profiles or annotations the library consumes — never a call from inside.
## Related
[Section titled “Related”](#related)
ADR-0005, ADR-0006, ADR-0013, ADR-0015, ADR-0019.
# ADR-0008: Align blocks in explainable passes
**Status:** Accepted **Date:** 2026-08-26 **Deciders:** houfu
## Context
[Section titled “Context”](#context)
Block alignment is the heart of the structural engine and the thing that makes or breaks ADR-0001. Given two block trees, decide which block in the source corresponds to which block in the test, before diffing any text.
Prior art: Open-XML-PowerTools’ WmlComparer hashes block content and runs LCS. Docxodus adds move detection by post-hoc word-level Jaccard similarity (threshold 0.8, minimum three words) and, in its newer `docxdiff` engine, paragraph split/merge detection. adeu aligns paragraphs and falls back to whole-block replacement below a 0.35 similarity ratio. The academic line is tree edit distance — Zhang-Shasha, Chawathe, GumTree.
Documents also carry a signal none of those exploit systematically: the labels themselves. Clause `7.2` in one version is overwhelmingly likely to correspond to clause `7.2` in the other, even when its text changed substantially.
## Decision
[Section titled “Decision”](#decision)
Alignment runs in ordered passes: exact content match; then label match; then fuzzy similarity above a configurable threshold; then positional fill-in for what remains. Similarity uses difflib’s ratio in core and rapidfuzz when installed (ADR-0004). Every matched pair records **which pass matched it**, and that record is exposed in the output.
Unmatched blocks become inserts and deletes. A deleted block that fuzzy-matches an inserted block elsewhere is a move (ADR-0009). Matched content with differing labels is a renumbering.
This is deliberately more machinery than a single global LCS. The agreed disposition is to build it as designed and pare it back if performance or odd results demand — with the pass record as the evidence for what to cut.
## Alternatives considered
[Section titled “Alternatives considered”](#alternatives-considered)
**Single global LCS over block hashes.** Simpler and proven in WmlComparer, but it cannot use labels, handles moves only as delete-plus-insert, and gives no explanation of why two blocks were considered the same.
**Embedding similarity.** Rejected: it would introduce a model dependency (ADR-0007), make results non-deterministic across versions, and be unexplainable.
**Tree edit distance.** Rejected for 1.0: the general algorithms are expensive and their edit scripts do not map cleanly onto the change vocabulary users want (move, renumber, split). Worth revisiting if the pass approach plateaus.
## Consequences
[Section titled “Consequences”](#consequences)
Positive: deterministic, explainable and tunable. “Matched by label 7.2” is an answer a user can argue with. Passes can be individually disabled or reordered per profile.
Negative: more parameters to get wrong; thresholds that suit contracts may misfire on prose or on tables of near-identical rows. Worst case is quadratic in block count, so early exit on exact matches matters. Behaviour differs subtly with and without rapidfuzz.
## Revisit when
[Section titled “Revisit when”](#revisit-when)
There is an explicit review gate after the benchmark (ADR-0021) exists: if a pass contributes few matches or many wrong ones, cut it. Revisit tree edit distance if split/merge detection (ADR-0009) proves unmanageable in the pass framework.
## Related
[Section titled “Related”](#related)
ADR-0004, ADR-0005, ADR-0009, ADR-0021.
# ADR-0009: Detect moves and renumbering in 1.0; splits and merges later
**Status:** Accepted **Date:** 2026-08-26 **Deciders:** houfu
## Context
[Section titled “Context”](#context)
Beyond insert, delete and modify, four structural change kinds matter in real documents: a block **moved** elsewhere; blocks **renumbered** because something was inserted above them; one block **split** into two; two blocks **merged** into one.
These are what a flat diff cannot express. A moved clause appears as a large deletion and an unrelated large insertion. A renumbering makes every label look edited. Both are the changes users most often complain about in Word’s own Compare output.
They differ sharply in cost. Moves and renumbering fall almost directly out of the alignment passes in ADR-0008: a move is an unmatched delete that fuzzy-matches an unmatched insert; a renumbering is matched content with a different label. Splits and merges need concatenation matching — testing whether one block’s content corresponds to two consecutive blocks’ content combined — which is a different and more expensive search.
## Decision
[Section titled “Decision”](#decision)
Moves and renumbering ship in 1.0. Splits and merges are 1.1.
Move detection is a **release gate**, not a feature: 1.0 does not ship with move recall below 0.9 on the synthetic-mutation corpus, or with any move false positive on the hand-labelled set that a reviewer would call wrong.
## Alternatives considered
[Section titled “Alternatives considered”](#alternatives-considered)
**All four in 1.0.** Rejected: split/merge is the larger piece of alignment work and would delay everything behind it, including both end deliverables.
**None in 1.0** — ship structural comparison with insert/delete/modify only. Rejected: moves are the single most visible thing that distinguishes a structural redliner from a flat one. Shipping without them would make 1.0 hard to tell apart from a good flat diff in a demo.
## Consequences
[Section titled “Consequences”](#consequences)
Positive: a demo can show a moved clause reported as a move, which is the clearest possible illustration of the thesis in one screen. The gate also forces the benchmark to exist and to be honest before release.
Negative: a false-positive move is worse than a missed one — telling a lawyer a clause moved when it did not damages trust more than silence. Hence the asymmetric gate (recall threshold plus zero tolerated bad false positives). Splits and merges absent in 1.0 means a paragraph broken in two still shows as a delete and two inserts, which will be noticed.
## Revisit when
[Section titled “Revisit when”](#revisit-when)
If the false-positive constraint proves impossible to meet at useful recall, consider shipping moves behind a flag, off by default, with the benchmark numbers published — but not silently lowering the bar.
## Related
[Section titled “Related”](#related)
ADR-0008, ADR-0021.
# ADR-0010: Keep difflib as the leaf differ
**Status:** Accepted **Date:** 2026-08-26 **Deciders:** houfu
## Context
[Section titled “Context”](#context)
Once blocks are aligned (ADR-0008), the text inside each aligned pair still needs diffing. Today that is `difflib.SequenceMatcher` over regex word tokens, and it is the part of redlines that has worked well since 2023.
Two defects are real and measured. First, `SequenceMatcher` is constructed with the default `autojunk=True`, which for any sequence of 200+ items treats every token occurring more than 1% of the time as “popular” and refuses to start a match on it. On varied prose this is harmless — a 565-token contract with three scattered edits gives identical results with autojunk on and off. On repetitive text it is catastrophic and silent: a 1,050-token block of one clause repeated thirty times, with a single two-word change, is reported as `('replace', 11, 1050, 11, 1050)` — the whole document replaced. Repetitive blocks are common in real documents (schedules, price lists, “Intentionally omitted” runs).
Second, there is no semantic cleanup: “thirty (30) days” to “sixty (60) days” becomes two separate replaces split by an equal `(` token, so the JSON and the statistics both report two changes where a human sees one.
The alternative on offer is diff-match-patch, which every competitor uses. It is also archived upstream (Google archived the repository on 5 August 2024); the PyPI package tracks a community fork.
## Decision
[Section titled “Decision”](#decision)
Keep difflib. Disable `autojunk` (and expose it as an option). Add a cleanup pass that merges adjacent operations separated only by punctuation or whitespace. Keep the nupunkt sentence tokenisation as a leaf-level option, with paragraph structure always preserved — the current sentence mode reflows the document one sentence per paragraph, which is a defect of encoding structure in a `¶` token and disappears with the block model.
diff-match-patch stays out. The processor interface remains, so it can be added later as an alternative leaf differ without disturbing anything.
## Alternatives considered
[Section titled “Alternatives considered”](#alternatives-considered)
**Replace difflib with diff-match-patch now.** Rejected: it adds a dependency (against ADR-0004) on an upstream-archived library, to fix problems that either have one-line fixes or disappear once diffs run inside short aligned blocks. The interesting algorithmic work in this project is block alignment, which diff-match-patch does not address at all.
## Consequences
[Section titled “Consequences”](#consequences)
Positive: no new dependency; the browser build stays trivial; the code path that most users already rely on is unchanged in character. Aligned blocks are short, so `autojunk=False` costs little in practice and the quadratic worst case is bounded by block size rather than document size.
Negative: our cleanup will be less sophisticated than diff-match-patch’s `cleanupSemantic`, and word-boundary quality may lag competitors on pathological inputs. Turning autojunk off has a real cost on any remaining whole-document comparisons, which the 0.6 compatibility facade still performs — so the hygiene release should measure it.
## Revisit when
[Section titled “Revisit when”](#revisit-when)
If leaf-diff quality shows up as a complaint after block alignment lands, add diff-match-patch as an optional processor rather than swapping the default.
## Related
[Section titled “Related”](#related)
ADR-0004, ADR-0008.
# ADR-0011: JSON as the canonical change format, with an annotated renderer
**Status:** Accepted **Date:** 2026-08-26 **Deciders:** houfu
## Context
[Section titled “Context”](#context)
The change tree is the product (ADR-0001), so its wire format is the most important interface decision in the project. Three consumers matter: Python callers, MCP clients (which are LLMs), and the browser.
The question raised during review was whether XML would be a more stable long-lived interface than JSON.
The survey shows a clear split in what the field does, and it is not JSON-versus-XML for the same data. Every agent-facing tool speaks JSON: MCP tool results are JSON by protocol; Draftable’s change-details API is JSON; Docxodus exposes `GetEditScriptJson`; adeu’s edit batches and diff hunks are JSON. The XML-based representations in the field are all *documents with changes inline* — OOXML’s `w:ins`/`w:del`, Akoma Ntoso’s amendment markup, and the plain-text cousin CriticMarkup that adeu projects revisions into. So the real distinction is **change list as data** versus **annotated document**.
On stability: XML offers namespaces and XSD; JSON Schema plus an explicit version field gives equivalent validation and versioning. The practical difference is tooling. JSON round-trips to Python dicts with the stdlib, is native in the browser, and is what every MCP client already parses. XML would mean ElementTree or lxml on the producing side and hand-written mapping on the consuming side, for no gain in the delivery path.
On what actually helps a model: Anthropic’s guidance favouring XML tags is about *delimiting sections in a prompt*, not about wire formats; structured outputs and tool calls are JSON. Where markup genuinely helps is in place — a model reading `within {--thirty (30)--}{++sixty (60)++} days` reasons about the clause better than one reading a change object with two offsets.
## Decision
[Section titled “Decision”](#decision)
JSON is the canonical serialisation, with a published JSON Schema, a top-level `schema_version`, and a stated compatibility policy: additive changes bump the minor, breaking changes bump the major and the previous version stays producible.
Alongside it, an **annotated-document renderer** is a first-class output: the source document with changes marked in place, using CriticMarkup for text and markdown and a tag variant (``, ``, ``) for HTML. This is what the MCP summary and annotate tools lean on when a model needs surrounding context.
The v1 flat JSON stays unchanged and remains what `output_json()` produces, per ADR-0003.
## Alternatives considered
[Section titled “Alternatives considered”](#alternatives-considered)
**XML as the canonical format.** Rejected: nothing in the delivery path consumes it natively, and its real advantage is expressible as a renderer.
**JSON only, no annotated view.** Rejected: it would leave the genuinely useful idea behind XML-based standards on the table, and the MCP server would be weaker for it.
## Consequences
[Section titled “Consequences”](#consequences)
Positive: one canonical format that every consumer already parses, plus a second representation tuned for reading. An XML renderer over the same tree remains about a day’s work if an enterprise integration ever needs one, and it would not disturb the canonical format.
Negative: two representations to keep consistent, and the annotated renderer needs a defined escaping story for documents that already contain CriticMarkup-like syntax. Schema versioning is a commitment: once published, breaking it is expensive.
## Revisit when
[Section titled “Revisit when”](#revisit-when)
If a real consumer requires XML, add the renderer. If the schema needs a breaking change, that is a new ADR, not an edit to this one.
## Related
[Section titled “Related”](#related)
ADR-0003, ADR-0012, ADR-0016, ADR-0018.
# ADR-0012: Address blocks with HTML-like paths and document labels
**Status:** Accepted **Date:** 2026-08-26 **Deciders:** houfu
## Context
[Section titled “Context”](#context)
Every change needs an address: something verify (ADR-0015) can be scoped to, an MCP client can point at, a user can recognise, and a renderer can display. The 0.6 engine has only global token offsets, which mean nothing to a reader and shift whenever anything earlier in the document changes.
Three candidate schemes: positional paths, document labels (“7.2”, “(a)”, “Schedule 2”), and character offsets.
## Decision
[Section titled “Decision”](#decision)
Carry all three, in a syntax borrowed from HTML rather than invented: a DOM-like path (an XPath-style `/body/section[7]/clause[2]`, or a CSS-style equivalent — the exact spelling is a design task), the document’s own label where one exists, a heading breadcrumb where available, and character offsets *within* the block. Global offsets survive only in the v1 output for compatibility.
Priority note: this is a required feature but a lower-priority one than the semantic layer. Get roles right first.
## Alternatives considered
[Section titled “Alternatives considered”](#alternatives-considered)
**Global offsets only.** Rejected: unreadable and unstable.
**Labels only.** Rejected: many blocks have no label (preamble, recitals, unlabelled paragraphs, table cells), and labels are not unique across schedules.
**A bespoke path syntax.** Rejected on the reasoning that familiarity beats novelty: if it looks like something people already read in HTML and XML tooling, both humans and models need no explanation.
## Consequences
[Section titled “Consequences”](#consequences)
Positive: addresses are stable under edits elsewhere in the document, human-recognisable where labels exist, and machine-usable everywhere. Scoping verify by label (“clause 7”) or by path both work.
Negative: three coexisting addressing schemes is more surface than one, and paths do shift when a block is inserted above — which is exactly why labels are carried alongside. An address is a position, not an identity; stable block identity across versions is the alignment’s job, not the address’s, and conflating the two would be a design error.
## Revisit when
[Section titled “Revisit when”](#revisit-when)
If consumers need identity rather than position — for example to store review comments against blocks across versions — that is a separate concept (a block id) and a separate ADR.
## Related
[Section titled “Related”](#related)
ADR-0005, ADR-0011, ADR-0015.
# ADR-0013: Limit 1.0 to plain text and markdown
**Status:** Accepted **Date:** 2026-08-26 **Deciders:** houfu
## Context
[Section titled “Context”](#context)
An earlier draft of the plan had 1.0 reading txt, markdown, DOCX, PDF and HTML, so the demo site could promise “upload anything”.
The concern raised on review: the top requirements are semantic (roles, clause structure, cross-references), and PDF and DOCX threaten to drag OCR and LLM requirements in behind them, bulking up a release whose point is to demonstrate a thesis.
That concern is well founded for PDF and partly founded for DOCX. Semantic roles come from heuristics over clean text (ADR-0006). Plain text and markdown supply clean text directly. PDF does not: extracted text loses heading structure and list nesting, page furniture interleaves with content, and scanned documents have no text at all — which is where OCR pressure comes from. DOCX needs neither OCR nor a model, and its styles would actually *help* the semantic pass, but it does add a dependency, a reader to maintain, and a set of format edge cases.
## Decision
[Section titled “Decision”](#decision)
1.0 reads plain text and markdown only. HTML, DOCX and PDF move to 1.1, in that order of cost. The demo site’s promise changes from “upload anything” to “drop text or markdown”, stated plainly, with a specific “coming in 1.1” message for other types.
The markdown reader is the small stdlib-regex one — ATX headings, nested lists, numbered clause patterns, pipe tables, fenced code — not a markdown-it dependency. Markdown is what LLMs emit, so it is the primary persona’s most common input and cannot be deferred.
To keep the release demonstrable without uploads, a single sample pair is defined and becomes the site’s default state: a short services agreement in markdown and an amended version containing exactly one of each detectable change — a modified definition, a moved clause, a renumbering, an updated cross-reference, a deleted sub-clause, an inserted table row, a whitespace-only non-change, and an edit inside a repetitive schedule that the flat 0.6 engine gets wrong. Its expected change tree is a golden file: the first test written and the last allowed to fail.
## Alternatives considered
[Section titled “Alternatives considered”](#alternatives-considered)
**All five formats in 1.0.** Rejected: scope, and the risk that the weakest reader (PDF) becomes how people judge the engine.
**Defer markdown too, treating it as plain text.** Considered and rejected: markdown is nearly free given the shared label-detection and continuation logic, and deferring it would leave the primary persona’s own inputs unparsed.
**Keep DOCX in 1.0 and defer only PDF.** This was the standing recommendation before the review, on the grounds that DOCX is where most real version pairs live. Overruled in favour of a smaller slice; the reader is first in the 1.1 queue after HTML.
## Consequences
[Section titled “Consequences”](#consequences)
Positive: 1.0 stays a demonstrable slice with no OCR question, no model question, and no heavy dependency. The Pyodide build (ADR-0019) needs no extras at all.
Negative: a lawyer arriving at the site with a .docx is turned away, which is a real first-impression cost. Mitigations: the sample pair is the default state so capability is visible before any upload, the deferral message is specific, and DOCX is early in 1.1.
## Revisit when
[Section titled “Revisit when”](#revisit-when)
If inbound demand is dominated by DOCX before 1.0 ships, pull the reader forward — it is a contained piece of work, and this ADR’s reasoning does not argue that DOCX is hard, only that it is not needed to prove the thesis.
## Related
[Section titled “Related”](#related)
ADR-0002, ADR-0006, ADR-0007, ADR-0014, ADR-0019.
# ADR-0014: Never write OOXML; delegate tracked changes to appliers
**Status:** Accepted **Date:** 2026-08-26 **Deciders:** houfu
## Context
[Section titled “Context”](#context)
The obvious “complete” version of a redlining library produces a Word document with native tracked changes. That is what python-redlines, Docxodus, stemma, safe-docx, folio, jubarte, SuperDoc, adeu and superdoc-redlines all do, and it is what the one public benchmark measures.
It is also a deep well. Writing `w:ins`/`w:del` that Word accepts, rejects and filters correctly means handling numbering, styles, tables, footnotes, headers, content controls and fields. python-redlines documents a bug where move markup produces a file Word refuses to open. The original Open-XML-PowerTools engine is described as crashing on documents with minor format issues. Every serious implementation in this space is a multi-year effort by a team.
Meanwhile at least four open-source OOXML patchers already exist and are actively maintained, and two of them (adeu, superdoc-redlines) are explicitly designed to accept an edit batch from an external decision-maker.
## Decision
[Section titled “Decision”](#decision)
redlines does not write OOXML — not in 1.0, not on the current roadmap. When DOCX output is wanted, the change tree is exported as an edit batch for an existing applier (adeu primary, superdoc-redlines secondary), which does the writing. That export is 1.1, deferred alongside DOCX reading.
One design constraint applies from 1.0: every inline change must be recoverable as (block address, old text, new text) with enough surrounding context to anchor a text search. That keeps the door open without building anything now.
## Alternatives considered
[Section titled “Alternatives considered”](#alternatives-considered)
**A native `w:ins`/`w:del` writer.** Rejected: it is where the competition is strongest and redlines has no edge, and it would consume the whole roadmap.
**No DOCX output at all, ever.** Rejected as unnecessarily absolute; delegation costs little and turns a rival into a back end.
## Consequences
[Section titled “Consequences”](#consequences)
Positive: it keeps the project’s scope honest and makes redlines a complement rather than a competitor to adeu and Docxodus — a joint story (“redlines computes the structural diff, adeu writes the tracked changes”) is coherent and worth proposing to those maintainers once the JSON schema is drafted, since the schema is the integration point.
Negative: our DOCX story depends on someone else’s addressing model. adeu targets edits by `target_text` search, which its own issues (#28, #29) admit is ambiguous on repeated text. Mitigation when the export is built: emit strict match mode with enough context, fall back to block-level replacement, and report ambiguities rather than guessing. There is also a positioning cost — “does it produce Word redlines?” will be answered “through another tool”, which some evaluators will score as a no.
## Revisit when
[Section titled “Revisit when”](#revisit-when)
If every applier stalls or disappears, or if a paying use case requires a single-dependency path to a tracked-changes DOCX, reconsider — but the first response should be to contribute to an applier, not to start a fifth one.
## Related
[Section titled “Related”](#related)
ADR-0001, ADR-0013.
# ADR-0015: Ship verify mode in 1.0
**Status:** Accepted **Date:** 2026-08-26 **Deciders:** houfu
## Context
[Section titled “Context”](#context)
Agent pipelines that edit documents have a question no tool answers: *did the model change only what it was told to change?* Today the answer is a human reading a diff.
The pattern is visible in the field. Anthropic’s docx skill ships a `validators/redlining.py` that strips one author’s revisions from both documents and text-compares them to check that the tracked changes faithfully represent the intended edit — a diff used as a validator. Microsoft’s Legal Agent in Word is described as using a “deterministic resolution layer” rather than trusting regenerated text. The need is recognised; nobody has generalised it.
A structural engine can answer much more than “is the text the same”: it can say whether anything outside the permitted scope changed, whether blocks moved, whether numbering shifted, and whether the change density outside the target area is non-zero. With the semantic layer (ADR-0005), scope can be expressed in the document’s own terms — “only the definitions section and clause 7”.
## Decision
[Section titled “Decision”](#decision)
Verify ships in 1.0 as a headline feature, not as a follow-on. Inputs: the original document, the edited document, and an allowed scope expressed as block addresses, labels or roles. Output: pass or fail, the list of out-of-scope changes, and the structural side effects (moves, renumbering).
It is deterministic. The library does not accept a natural-language instruction and derive the scope from it; deriving scope from an instruction is the caller’s job — and on the MCP surface, the model’s.
Whitespace-only and label-only changes are configurable exemptions.
Text-anchor scoping (“only the paragraph containing this phrase”) is deferred to 1.1, because it imports the ambiguity problem adeu is fighting.
## Alternatives considered
[Section titled “Alternatives considered”](#alternatives-considered)
**Ship compare first, verify later.** Rejected: verify is a thin layer over the change tree, and it is the single clearest reason for the primary persona to adopt the library. Deferring it would make 1.0 a better diff rather than a new capability.
**Accept an instruction and use an LLM to derive scope.** Rejected under ADR-0007. It would also make verification results non-reproducible, which defeats the purpose of a validator.
## Consequences
[Section titled “Consequences”](#consequences)
Positive: a feature no competitor has, aimed squarely at the primary persona, at low marginal cost. It also gives the MCP server (ADR-0018) something to do beyond producing diffs, and it makes the deterministic-semantics rule pay for itself: a validator that gives different answers on different runs is worthless.
Negative: it invites the question “how do I know my scope was right?”, which is a real usability problem — a wrong scope produces a confident pass. Mitigation: `read_blocks` and `preview_structure` exist so a caller can inspect addresses before scoping, and verify reports what it considered in scope.
## Revisit when
[Section titled “Revisit when”](#revisit-when)
If users consistently want instruction-derived scope, build it as an example or a companion, not inside the library.
## Related
[Section titled “Related”](#related)
ADR-0005, ADR-0007, ADR-0012, ADR-0018.
# ADR-0016: Implement the summary renderer in core, deliver it with MCP
**Status:** Accepted **Date:** 2026-08-26 **Deciders:** houfu
## Context
[Section titled “Context”](#context)
A deterministic, plain-text summary of changes — one line per block change with its address, role and inline detail, plus section totals — is the output most likely to be copied into a prompt or a review email.
It is also commercially validated. Draftable shipped exactly this in February–March 2026 as “AI-Ready Redline Export”, a portable text format for pasting into ChatGPT, Claude, Harvey or Legora.
Two questions: when to build it, and where the code lives. The review’s steer was that it belongs with the MCP deliverable, since that is where a model consumes it.
## Decision
[Section titled “Decision”](#decision)
The summary renderer is **built during the MCP milestone**, but **implemented in the core library**, not in the MCP package. The CLI (`redlines summary`) and the site use the same implementation.
Output is stable-ordered and deterministic, so it can be golden-tested and diffed between runs.
## Alternatives considered
[Section titled “Alternatives considered”](#alternatives-considered)
**Build it with the other renderers, before MCP.** Rejected on timing: its shape should be driven by what a model actually needs, which is clearest while building the tools that feed it.
**Implement it inside the MCP package.** Rejected: it would mean CLI and site users either lose the feature or get a second, divergent implementation. There is a standing principle that no comparison or rendering logic lives outside the core, so that behaviour seen on the site is reproducible from Python with the same inputs.
## Consequences
[Section titled “Consequences”](#consequences)
Positive: one summary format everywhere; the MCP server stays a thin skin; the format is designed against a real consumer.
Negative: a small sequencing awkwardness — a core feature is built during a milestone named after a different package — which should be noted in the roadmap so it is not mistaken for scope creep in the MCP work.
## Revisit when
[Section titled “Revisit when”](#revisit-when)
If the summary format needs to differ materially between an MCP client and a human reader, that is two renderers with two names, not one renderer with a mode flag.
## Related
[Section titled “Related”](#related)
ADR-0011, ADR-0017, ADR-0018, ADR-0025.
# ADR-0017: Publish the MCP server as a separate package
**Status:** Accepted **Date:** 2026-08-26 **Deciders:** houfu
## Context
[Section titled “Context”](#context)
An MCP server is one of the two end deliverables. It could be an extra on the main package (`redlines[mcp]`, adding a `redlines serve` command) or its own distribution (`redlines-mcp`) depending on the core.
The recommendation on the table was the extra, on discoverability grounds. The decision went the other way.
## Decision
[Section titled “Decision”](#decision)
`redlines-mcp` is a separate package depending on a compatible range of `redlines`. The CLI stays in the core.
## Rationale
[Section titled “Rationale”](#rationale)
Three reasons. It keeps fastmcp and its transitive dependencies entirely out of the core repository, which matters more than usual here because of ADR-0004 and because the core must remain importable in a browser runtime (ADR-0019) — an MCP dependency in the tree, even an optional one, is a maintenance and audit surface for a library whose selling point is that it installs anywhere. It lets the server iterate on its own cadence, which matters while the MCP protocol itself is still moving. And it keeps two audiences’ issue trackers apart.
## Alternatives considered
[Section titled “Alternatives considered”](#alternatives-considered)
**`redlines[mcp]` extra.** The standing recommendation. Better discoverability (one package, one README), one version to reason about, no synchronisation problem. Overruled for the reasons above.
**No MCP server.** Not seriously considered given ADR-0002.
## Consequences
[Section titled “Consequences”](#consequences)
Positive: a clean core; independent releases; the server can depend on fast-moving MCP tooling without imposing it.
Negative: two packages can drift, and a core release can break the server. Mitigations: pin a compatible core range; run the MCP golden tests against the core’s main branch inside the core’s CI; cut releases of the two together. Discoverability splits, so both READMEs must cross-link prominently, and the registry listings matter more.
## Revisit when
[Section titled “Revisit when”](#revisit-when)
If keeping the two in sync becomes a recurring source of breakage, merging into an extra is a reversible decision — the code boundary would be unchanged.
## Related
[Section titled “Related”](#related)
ADR-0002, ADR-0004, ADR-0018, ADR-0019, ADR-0025.
# ADR-0018: Use MCP tools, prompts and resources so models can author profiles
**Status:** Accepted **Date:** 2026-08-26 **Deciders:** houfu
## Context
[Section titled “Context”](#context)
The first sketch of the MCP server was a set of tools — compare, summary, verify, read\_blocks — which would have made it the CLI over a socket, adding reach but nothing new.
The observation that changed it, raised during review: the MCP server is the only surface where a **model can write a structure profile**. The CLI cannot hold that conversation and the site has no model in it at all.
That matters because profiles (ADR-0006) are how semantic understanding enters the system, and writing one for an unfamiliar document family is exactly the kind of pattern-spotting work a model is good at and a user finds tedious.
MCP has three primitives, not one. **Tools** are model-invoked. **Prompts** are user-invoked templates the server fills with its own context. **Resources** are documents a client can read directly. A tools-only server uses a third of the protocol.
## Decision
[Section titled “Decision”](#decision)
The server uses all three.
**Tools:** `compare`, `summary`, `annotate`, `verify`, `read_blocks`, plus `preview_structure` (apply a profile to one document and return the block tree with `matched_by`, confidence and fallback count) and `validate_profile` (schema and pattern errors with line references). Each accepts a file path or inline content, plus an optional profile by path or inline.
**Prompts:** `draft_profile` (hands the model the profile schema, a built-in profile as a worked example, and a sample of the user’s document), `refine_profile` (the current profile plus `preview_structure` output plus what looks wrong), and later `explain_changes`.
**Resources:** the profile schema, every built-in profile, the change-tree schema, the skill text.
Together these close a loop with no human in the middle and no model call inside the library: draft a profile, apply it with `preview_structure`, read the confidence and fallback counts, see that schedules did not reset numbering or that `(i)` was mis-nested, refine, repeat, then `compare`. That loop is documented as the canonical workflow in the skill text, with a worked transcript against the sample pair, and a golden test replays it with a fixed draft.
Transport is stdio in 0.1 — what Claude Code, Claude Desktop and Cursor use. Streamable HTTP follows in 1.1 for hosted agents.
## Alternatives considered
[Section titled “Alternatives considered”](#alternatives-considered)
**Tools only.** Rejected: it wastes the protocol’s most distinctive capability and reduces the server to a transport.
**One omnibus tool with a mode flag.** Rejected: models call several narrow, well-described tools more reliably than one wide one.
## Consequences
[Section titled “Consequences”](#consequences)
Positive: the server becomes the place where semantic understanding is *created*, not just consumed — a genuinely differentiated reason to install it. It also makes the no-LLM-in-core rule (ADR-0007) work in our favour: the model uses the library rather than the library using a model.
Negative: it imposes a hard constraint on the profile format — a model given the schema and one example must be able to produce a valid profile in a single turn, which rules out anything deeply nested or clever. It is also more surface to document, test and keep current as the MCP specification evolves. And profiles authored in a chat must be savable as files, or the loop produces work that evaporates.
## Revisit when
[Section titled “Revisit when”](#revisit-when)
If clients in practice ignore prompts and resources, shrink to tools plus a resource for the schema. If the profile format cannot be made legible enough for single-turn authoring, this loop is not viable and the profile design needs to change, not the server.
## Related
[Section titled “Related”](#related)
ADR-0006, ADR-0007, ADR-0016, ADR-0017.
# ADR-0019: Run the demo site entirely in the browser
**Status:** Accepted **Date:** 2026-08-26 **Deciders:** houfu
## Context
[Section titled “Context”](#context)
The second end deliverable is a website where someone drops two documents and sees the changes — a usable tool, not a screenshot. Three ways to build it: client-side Python compiled to WebAssembly (Pyodide) on static hosting; a server-hosted Python app; or Streamlit, as with PLUS Explorer.
The documents in question are contracts. “Your documents never leave your machine” is the claim every competitor demo makes — python-redlines’ demo runs Docxodus as WASM client-side, jubarte’s site says the same — and it is the first thing a legal user checks.
## Decision
[Section titled “Decision”](#decision)
The site is static and runs the published redlines wheel in Pyodide inside a web worker. No backend, no upload endpoint, no analytics on document content, and a plain statement to that effect on the page. Hosted on GitHub Pages, source in the main repository under `site/` so the wheel and the site version together.
A corollary becomes a standing constraint: **the core and every 1.0 reader must import under Pyodide**, checked by a CI job that builds the wheel and imports it in a browser runtime. Extras that are unavailable degrade gracefully.
Checked on 26 August 2026 against Pyodide 0.28: lxml, rich, click and pydantic are built in; python-docx and pypdf are pure-Python wheels installable with micropip; **rapidfuzz and python-Levenshtein are C extensions with no Pyodide build**. Since 1.0 reads only text and markdown (ADR-0013), 1.0 needs no extras at all in the browser.
The site shows: two inputs (drop, upload or paste), the sample pair loaded by default, a block-change list with roles and addresses expandable to inline redlines, the annotated document, the summary, and the JSON with a copy button, plus a `dropped` notice per file and a profile selector.
## Alternatives considered
[Section titled “Alternatives considered”](#alternatives-considered)
**Server-hosted app.** Simplest engineering, any dependency, could add model-backed features. Rejected: uploaded contracts would touch our server — a real objection for the intended audience — and it costs money to run unattended.
**Streamlit.** Fast to build and familiar. Rejected: server-side uploads again, limited control over the diff UI, and it reads as a demo rather than a tool.
## Consequences
[Section titled “Consequences”](#consequences)
Positive: zero hosting cost, a privacy claim that is true by construction, and a forcing function that keeps the library light. It also proves the dependency discipline in public.
Negative: a 10–15 MB initial runtime load; no model-backed features on the site ever (which is consistent with ADR-0007 but means the site cannot answer questions about a diff); and **alignment in the browser runs on the difflib-ratio fallback rather than rapidfuzz**, so if tuned and untuned quality diverge noticeably, the site will under-sell the engine. That divergence must be measured (ADR-0021) and either closed or disclosed.
There is also a scope risk: a site that works well invites requests for accounts, history and DOCX download. Anything needing a server is out of scope by construction, and that is the answer.
## Revisit when
[Section titled “Revisit when”](#revisit-when)
If the Pyodide load time or the difflib/rapidfuzz gap makes the site unrepresentative, consider a pure-Python similarity implementation in core (see ADR-0004) rather than a server.
## Related
[Section titled “Related”](#related)
ADR-0004, ADR-0007, ADR-0013, ADR-0020, ADR-0021.
# ADR-0020: Ship the MCP server before the site
**Status:** Accepted **Date:** 2026-08-26 **Deciders:** houfu
## Context
[Section titled “Context”](#context)
Both end deliverables sit on the same core. The order in which they are built is a real choice, because whichever comes first shapes the contracts the other inherits.
## Decision
[Section titled “Decision”](#decision)
The MCP server ships first, within days of the 1.0 core. The site follows, once the renderers are stable.
## Rationale
[Section titled “Rationale”](#rationale)
The server is thin over the core, so it reaches the primary persona (ADR-0002) at the lowest cost. More importantly, building it first *forces the contracts to be right*: the change-tree JSON, the summary format, verify’s inputs and outputs, and the profile format all get exercised by a demanding consumer before any UI depends on them. A schema frozen against a real client is a better schema than one frozen against a design document.
Building the site first would mean doing UI work against contracts that are still moving, and UI is the most expensive thing to redo.
## Alternatives considered
[Section titled “Alternatives considered”](#alternatives-considered)
**Site first** — visible, shareable, and it would test the readers against real uploads early. Rejected for the contract-churn risk. The sample pair (ADR-0013) covers the “something to show” need in the meantime.
**Both in lockstep.** Rejected: slower to any release, and it splits attention during the phase when the schema most needs a single demanding consumer.
## Consequences
[Section titled “Consequences”](#consequences)
Positive: contracts settle early against a real client; the persona is served first; the site inherits stable outputs.
Negative: the visible, shareable artefact arrives last, so there is a period with a released library and no public demo. The sample pair and the benchmark report partly fill that gap.
## Revisit when
[Section titled “Revisit when”](#revisit-when)
If an opportunity makes a public demo urgent (a talk, a launch, a course), the order can be flipped — accepting the churn cost knowingly.
## Related
[Section titled “Related”](#related)
ADR-0002, ADR-0013, ADR-0017, ADR-0019.
# ADR-0021: Make alignment quality measurable with our own benchmark
**Status:** Accepted **Date:** 2026-08-26 **Deciders:** houfu
## Context
[Section titled “Context”](#context)
The structural thesis (ADR-0001) rests entirely on alignment quality. If blocks are matched wrongly, every downstream claim — moves, renumbering, semantic summaries, verify — is wrong with confidence, which is worse than a flat diff that is merely coarse.
One public benchmark exists, `neurotic_docx_bench` (AGPL-3.0, by Jandira Technologies, who also make the closed jubarte engine it tops). It takes 763 `base.docx`/`next.docx` pairs, asks each tool for a tracked-changes DOCX, renders candidate and Word’s own output to PDF through LibreOffice, and scores pixel similarity. It has already measured redlines 0.6.1 via a third-party adapter at 45.9 mean with 18 failures — bottom of the table, which is exactly what a text-only differ should score on a pixel metric.
That benchmark cannot measure what we care about. A tool could find every correspondence perfectly and still score badly for dropping formatting; a tool could align nothing and score well by preserving the document. **No alignment metric exists in this field.**
## Decision
[Section titled “Decision”](#decision)
Build one, and build it *before* tuning alignment, not after.
Two corpora. A **synthetic-mutation** corpus: take real documents and apply known moves, splits, renumberings and edits programmatically, keeping the labels — ground truth for free, at any volume. And a small **hand-labelled** set of ten real before/after pairs, where the mutations are whatever really happened.
Metrics: block correspondence precision and recall, move detection recall, renumbering recall. Baselines: flat redlines 0.6 as the floor, and python-redlines as a comparator once DOCX reading exists.
Semantic role and span precision is reported on a hand-labelled sample but not gated in 1.0.
The move gate from ADR-0009 is enforced here. The benchmark report is published with the release and linked from the README.
## Alternatives considered
[Section titled “Alternatives considered”](#alternatives-considered)
**Reuse the visual benchmark only.** Rejected: it measures a different thing, and optimising for it would push the project toward OOXML fidelity, against ADR-0001.
**Tune first, measure later.** Rejected as the standard way to end up with thresholds that fit whatever documents were on hand.
## Consequences
[Section titled “Consequences”](#consequences)
Positive: the structural claims become checkable rather than rhetorical. It gives the ADR-0008 review gate its evidence, and it is itself a contribution — an alignment metric and corpus that others could adopt is the kind of thing that gets cited.
Negative: real cost, mostly human. Hand-labelling ten pairs is tedious and the synthetic generator is a small project of its own. There is also a self-marking risk: a benchmark we design could flatter us. Mitigations are publishing the generator and the labels, reporting the 0.6 baseline honestly, and keeping the hand-labelled set separate from the synthetic one.
`neurotic_docx_bench`’s 763 pairs are a useful text source for the corpus (AGPL, so usable for evaluation, not for bundling), and running our text through its adapter path would give a like-for-like comparison with the published 0.6.1 numbers.
## Revisit when
[Section titled “Revisit when”](#revisit-when)
Expand the hand-labelled set continuously. If someone else publishes a credible alignment benchmark, adopt theirs and stop maintaining ours.
## Related
[Section titled “Related”](#related)
ADR-0001, ADR-0008, ADR-0009, ADR-0019.
# ADR-0022: Keep the name redlines
**Status:** Accepted **Date:** 2026-08-26 **Deciders:** houfu
## Context
[Section titled “Context”](#context)
The name collides. `python-redlines` on PyPI is a different, actively maintained project doing DOCX comparison. `redlines.opensource.legal` is its demo. `redlines.free` is jubarte’s commercial site. Searching for “redlines” surfaces all of them.
A rename before a 1.0 rewrite would be the cheapest moment to do it.
## Decision
[Section titled “Decision”](#decision)
Keep the name. `redlines` stays the package, the repository and the project.
## Rationale
[Section titled “Rationale”](#rationale)
The name carries the distribution: 3.7M downloads, the course material, existing notebooks and blog posts, and whatever search authority the project has. A rename would forfeit all of it to solve a marketing nuisance rather than a product problem — nobody has ever installed the wrong package and been unable to tell.
The demo site’s domain is a separate and much cheaper decision, and can differentiate without touching the package.
## Alternatives considered
[Section titled “Alternatives considered”](#alternatives-considered)
**Rename to escape the collision.** Rejected for the reasons above. If a rename ever becomes necessary, the moment is a 2.0 with a transitional meta-package, not a quiet swap.
## Consequences
[Section titled “Consequences”](#consequences)
Positive: continuity, and no migration work.
Negative: ongoing confusion in search results and in conversation, particularly with python-redlines, whose maintainer is a plausible collaborator (ADR-0014) — which makes clear positioning in the README more important than usual. Both projects should probably say what the other is.
## Revisit when
[Section titled “Revisit when”](#revisit-when)
If a legal conflict arises, or if the collision demonstrably costs adoption (measurable as people arriving at the wrong project and saying so).
## Related
[Section titled “Related”](#related)
ADR-0014, ADR-0019.
# ADR-0023: Keep Python 3.10+ and strict typing
**Status:** Accepted **Date:** 2026-08-26 **Deciders:** houfu
## Context
[Section titled “Context”](#context)
redlines 0.6 supports Python 3.10 through 3.14, having dropped 3.8 and 3.9, and runs strict mypy with a `py.typed` marker. adeu, a comparable project, requires 3.12+.
A rewrite is the natural moment to raise the floor and use newer syntax.
## Decision
[Section titled “Decision”](#decision)
Keep the 3.10 floor. Keep strict mypy and `py.typed`. Use frozen dataclasses for the block and change models.
## Rationale
[Section titled “Rationale”](#rationale)
Nothing in the design needs 3.11 or 3.12 syntax. Notebook environments, managed platforms and corporate installations lag, and the primary persona (ADR-0002) works in exactly those places. Raising the floor would cost users to buy nothing.
Strict typing matters more than usual here because the block and change models are the public interface (ADR-0011); a typed model is self-documenting for both humans and the agents that will consume it.
Frozen dataclasses give cheap structural sharing and make it hard to mutate a tree accidentally during alignment.
## Alternatives considered
[Section titled “Alternatives considered”](#alternatives-considered)
**Raise to 3.12.** Rejected: no need, real cost.
**Runtime validation with pydantic.** Considered: it would give schema generation for free (ADR-0011). Rejected for core under ADR-0004 — a dependency for something the stdlib plus a hand-maintained JSON Schema can do. Reasonable to revisit if schema maintenance becomes a chore.
## Consequences
[Section titled “Consequences”](#consequences)
Positive: the widest install base, self-documenting models, no accidental mutation.
Negative: `nupunkt` requires 3.11+, so that extra is unavailable to a slice of supported users — already true today. The JSON Schema must be maintained by hand and kept in step with the dataclasses, which needs a test that generates one from the other and compares.
## Revisit when
[Section titled “Revisit when”](#revisit-when)
When 3.10 reaches end of life, or if hand-maintaining the schema proves error-prone.
## Related
[Section titled “Related”](#related)
ADR-0002, ADR-0004, ADR-0011.
# ADR-0024: No inline formatting change detection in 1.0
**Status:** Accepted **Date:** 2026-08-26 **Deciders:** houfu
## Context
[Section titled “Context”](#context)
“Text unchanged, but it became bold” is a change class that Word reports (`w:rPrChange`) and that python-redlines/Docxodus detects at run, paragraph and section level. An earlier plan for redlines made “rich tokens” carrying bold, italic and underline the foundational first phase of the whole rewrite, with formatting-change detection as a headline capability.
Two things argue against that ordering. Formatting changes are a DOCX-shaped concern, and redlines’ own users — notebook and agent developers comparing text and markdown — have not asked for them; the change classes they care about are content, structure and meaning. And making inline formatting foundational imposes a cost on *every* reader and on the token model, in service of a capability only one deferred format can supply.
There is also a design trap: naive implementations create a token per formatting run rather than per word, so the diff algorithm matches badly and “text unchanged, formatting changed” gets reported as a delete plus an insert — the opposite of the intent.
## Decision
[Section titled “Decision”](#decision)
No formatting-change category in 1.0. `attrs` may carry run-level formatting from readers that have it, and the semantic layer has an `emphasis` span type (ADR-0005) for readers where emphasis is meaningful, such as markdown. But formatting does not participate in alignment or diffing and is not reported as a change kind.
The block model, not an inline token model, is the foundation.
## Alternatives considered
[Section titled “Alternatives considered”](#alternatives-considered)
**Rich tokens with formatting as phase one.** The earlier plan. Rejected on priority: it front-loads cost for a deferred format’s benefit and delays the block model, which is what everything else depends on.
## Consequences
[Section titled “Consequences”](#consequences)
Positive: readers stay cheap; no token-explosion trap; the foundational phase is the one that carries the thesis.
Negative: when the DOCX reader arrives (1.1), redlines will report content and structure changes but not “this clause became bold”, which python-redlines does. For contract review that is a real gap, and it should be stated plainly rather than discovered.
## Revisit when
[Section titled “Revisit when”](#revisit-when)
After the DOCX reader ships, if users ask for it. The clean way to add it later is as a separate change category computed from `attrs` on aligned pairs — which the current design permits, since aligned pairs already carry both blocks’ attributes.
## Related
[Section titled “Related”](#related)
ADR-0005, ADR-0013, ADR-0014.
# ADR-0025: Treat the CLI and the MCP server as two skins over one function table
**Status:** Accepted **Date:** 2026-08-26 **Deciders:** houfu
## Context
[Section titled “Context”](#context)
redlines has a click-based CLI with `text`, `markdown`, `stats` and `json` subcommands and a command-less default that emits JSON — deliberately agent-friendly. The new capabilities (structural compare, annotate, summary, verify, profiles) need surfacing there too, and the question of what that costs came up during planning.
## Decision
[Section titled “Decision”](#decision)
The existing CLI is preserved unchanged, including the command-less default. Three or four new subcommands are added — `compare`, `summary`, `annotate`, `verify` — with `--profile` and `--format` options.
CLI and MCP share **one argument-normalisation layer**: resolving a path, stdin or inline content; applying a format hint; loading a profile from a path or inline; enforcing size limits. The CLI and the MCP tools are two thin skins over the same function table, and they are written together.
Estimated cost: about a day including tests, because all the logic lives in core.
## Rationale
[Section titled “Rationale”](#rationale)
Two benefits beyond the obvious. Shared normalisation means the two surfaces cannot drift in how they interpret inputs, which is the usual way a CLI and a server develop subtly different behaviour. And the CLI is the fastest harness for exercising the core — it exists before the MCP package does, so it is how the new capabilities get their first real use.
## Alternatives considered
[Section titled “Alternatives considered”](#alternatives-considered)
**Rebuild the CLI around the new model.** Rejected under ADR-0003.
**Skip new subcommands; rely on the MCP server and the Python API.** Rejected: the CLI is nearly free given the shared layer, it is how many users first meet the tool, and giving it up would remove the cheapest testing surface.
## Consequences
[Section titled “Consequences”](#consequences)
Positive: negligible marginal cost, guaranteed consistency between surfaces, an early test harness.
Negative: the shared layer is a small piece of infrastructure that must be designed before either skin, so it cannot be deferred to whichever surface is built second. The CLI also grows to eight or nine subcommands, which needs care in `--help` organisation so the old and new models do not read as one confusing whole.
## Revisit when
[Section titled “Revisit when”](#revisit-when)
If the CLI surface becomes unwieldy, group the legacy subcommands under a namespace with aliases preserved — a documentation change more than a code one.
## Related
[Section titled “Related”](#related)
ADR-0003, ADR-0016, ADR-0017, ADR-0018.
# ADR-0026: Publish the documentation with Astro Starlight, in the same site as the demo
**Status:** Accepted **Date:** 2026-08-27 **Deciders:** houfu
## Context
[Section titled “Context”](#context)
Documentation today is `pdoc` run over the package on release, deployed to GitHub Pages. It renders docstrings and nothing else: there is no way to publish a hand-written page.
1.0 needs hand-written pages. N6 requires the JSON v2 schema and the profile schema published with examples, and the agent guide rewritten for compare, summary, annotate and verify. ADR-0021 produces a benchmark report meant to be an external quality signal, which has to be readable somewhere. ADR-0006 makes profiles a user-authored artefact, so profile authoring needs a guide. The ADRs themselves are worth publishing. None of that is a docstring, so M4’s exit criterion — “agent guide live” — currently has nowhere to land.
Separately, ADR-0019 commits to a client-side demo site under `site/`, built by CI and served from GitHub Pages. Left alone, that is two static sites, two build systems and two deployments on one domain, and PRD section 13 still carries “where the site lives” as an open question.
## Decision
[Section titled “Decision”](#decision)
**Astro Starlight is the documentation platform**, replacing pdoc as the publishing surface, in **one Astro project under `site/`** that serves both the documentation and the demo.
* The demo of ADR-0019 becomes a route in that project, with the Pyodide worker as an island. One build, one deploy, one domain. This resolves PRD section 13’s open question in favour of the main repository.
* **pdoc is kept as the API-reference generator.** Its HTML is built into the site and served under `/api/`. Docstrings remain the source of truth for the API reference, so CONTRIBUTING’s instruction to edit documentation in the source files still holds.
* The migration happens in the **0.6.x hygiene release**, before any 1.0 documentation is written, so that no page is written twice — first for pdoc, then again for the new site.
## Rationale
[Section titled “Rationale”](#rationale)
The deciding argument is that the demo needs a JavaScript build regardless. ADR-0019 already commits to a Pyodide UI with a web worker, file drop, expandable change lists and a profile selector, which is a front-end application whatever hosts it. Given that a JS toolchain is arriving anyway, running one of them is cheaper than running a Python docs generator beside it, and it collapses two deployments into one.
Starlight in particular: it is a documentation theme rather than a framework to be assembled, so the default output is a sidebar, search and dark mode with no design work; MDX lets the schema pages embed a real worked example instead of a code fence copied by hand; and an Astro route sitting next to the docs is exactly the shape the demo needs.
There is a second-order benefit. The demo is the best documentation this project has — a visitor who can paste two clauses and see a change tree understands the thesis faster than any prose. Putting it inside the docs site rather than beside it means every documentation page is one click from a live example.
## Alternatives considered
[Section titled “Alternatives considered”](#alternatives-considered)
**Keep pdoc alone.** Free and already working. Rejected: it cannot publish a hand-written page at all, which is the entire requirement.
**MkDocs Material.** The conventional choice for a Python project, and a strong one: `mkdocstrings` gives a better API reference than pdoc, it stays inside the Python toolchain, and contributors would need no Node. Rejected on the demo. The Pyodide UI needs a bundler and a component model either way, so choosing MkDocs means maintaining a Python docs build *and* a JS app build, and deploying two artefacts to one Pages site. If ADR-0019 were ever reversed and the demo dropped, MkDocs Material would be the right answer and this decision should be revisited.
**Sphinx.** The most capable of the three and the standard for large Python projects. Rejected as disproportionate: RST or MyST plus autodoc ceremony is a lot of machinery for a library with six source files, and it has the same two-toolchain problem as MkDocs.
**Docusaurus.** Same JS-toolchain argument as Starlight, but a heavier React application for the same result, and its own routing conventions to work around for the demo.
**Docs and demo as separate builds.** Keeps the demo’s 10–15 MB Pyodide payload fully isolated from the documentation. Rejected: the isolation is already achieved by the worker and by route-level code splitting, and two builds means two deploy workflows, two dependency sets and a broken link between them the first time a path moves.
## Consequences
[Section titled “Consequences”](#consequences)
Positive: a home for everything 1.0 has to publish; the ADRs and the benchmark report become readable artefacts rather than repository files; M6 shrinks from “build and deploy a site” to “add a route”; and one deployment can never drift from the other.
Negative, and this is the real cost: **a Python repository acquires a Node toolchain**. A package manifest and lockfile, a `node_modules` in contributor checkouts, a Node step in CI, and a JavaScript dependency surface to keep patched — on a project whose central discipline (ADR-0004) is a dependency-free core. The mitigation is a boundary, not a promise: nothing in `site/` is imported by the wheel, nothing in the wheel depends on the site building, and a broken site build must never be able to block a release.
The API reference also becomes a two-step build — pdoc, then Astro — rather than one command, which is a sharper edge for a contributor previewing documentation locally than the single `uv run pdoc` it replaces.
## Revisit when
[Section titled “Revisit when”](#revisit-when)
If the demo is ever dropped or moved out of the repository, the JS toolchain loses its justification and MkDocs Material becomes the better answer. Revisit also if keeping the Node dependencies patched turns into recurring maintenance out of proportion to a documentation site, or if the demo’s payload turns out to degrade documentation page loads despite code splitting.
## Related
[Section titled “Related”](#related)
ADR-0004, ADR-0006, ADR-0019, ADR-0020, ADR-0021.
# ADR-0027: Agent-facing documentation is a machine surface and a contract page, not a guide
**Status:** Accepted **Date:** 2026-08-28 **Deciders:** houfu
## Context
[Section titled “Context”](#context)
ADR-0002 makes LLM and agent pipeline developers the primary persona. The artefact that serves them today is `AGENT_GUIDE.md`: 1,056 lines, last stamped 2025-10-22 against “Version 0.6.0+”. N6 and the M4 roadmap row both say it is to be “rewritten” for 1.0, which assumes the answer to a bad guide is a better guide. Before writing it twice — first for pdoc, now for the Starlight site of ADR-0026 — it is worth asking whether a guide is the right shape at all.
Three problems with the one we have.
**It serves two readers at once.** An assistant *writing integration code* wants prose, patterns and worked examples, and has context to spend on them. An agent *calling redlines at runtime* wants one thing: invocation form, output shape, exit codes, failure modes, size limits. The guide interleaves both, so neither is crisp; its Quick Reference Card is the second document trying to escape the first. At roughly 12–15k tokens, loading it costs the primary persona a meaningful slice of a working budget for content that is mostly justification.
**Nothing in it is machine-addressable.** Its “JSON Schema Reference” section is prose describing fields. It cannot be fetched, validated against, or diffed between versions. R19 and ADR-0011 already commit to a JSON schema with `schema_version`; R48 commits to publishing it and the profile schema. Once those exist as files at stable URLs, most of that section is a link.
**It restates generated surfaces, so it drifts.** It duplicates `--help`, docstrings, and — from M5 — the MCP tool descriptions and skill text, which is what a runtime agent actually reads. The drift has already started: the footer names 0.6.0 against a 0.6.1 package, and the header claims “All examples are copy-paste ready and tested” while nothing under `tests/` executes either the guide’s snippets or `examples/`. M0’s `autojunk` change and the punctuation-merge cleanup will alter reported change counts in those snippets, and no test will say which.
Meanwhile 1.0 adds surfaces that do the runtime job better than prose can: MCP tool descriptions and skill text (R32, R32c), the change-tree schema (R19), the profile schema (R1d, R48).
## Decision
[Section titled “Decision”](#decision)
**Agent-facing documentation is three tiers, not one document.**
**Tier 0 — the machine surface.** No prose written by hand, nothing to keep in sync: `llms.txt` and a per-page markdown export at the site root; the change-tree and profile schemas as fetchable files at stable URLs; the pdoc API reference under `/api/`; and the MCP skill text of R32c hosted rather than paraphrased. Docstrings remain the source of truth for the API reference, exactly as ADR-0026 has it.
**Tier 1 — one contract page**, short enough to be read whole in a single fetch. The verbs (`compare`, `summary`, `annotate`, `verify`), the invocation forms of the shared argument layer, the output shape linking to the schema rather than restating it, exit codes, failure modes and size limits. Everything else is a link out.
**Tier 2 — task pages for integrators.** CI check, pre-commit, batch comparison, profile authoring: today’s Integration Examples, kept but demoted. These pages **include their code from `examples/`** rather than pasting it, and `examples/` becomes executed by CI, so “tested” is a fact rather than a claim.
**Sequencing.** M0 stands up the Tier 0 mechanics and migrates `AGENT_GUIDE.md` to the site whole, as one page marked as the 0.6 guide with its date visible. M4’s “agent guide rewritten” becomes the decomposition into Tier 1 and Tier 2 described here.
## Rationale
[Section titled “Rationale”](#rationale)
The contract belongs where the caller already is. A runtime agent reads a tool description, a `--help` output or a schema; it does not read a document unless something tells it to. Every sentence of contract that lives only in prose is a sentence the caller may never see, and one more thing to keep true.
A document that duplicates a generated surface drifts, and here the evidence is not hypothetical — the version stamp and the tested-examples claim are both already wrong. Tiering does not merely shorten the prose; it removes the duplication that causes the drift.
Splitting the existing guide into pages now would be churn ahead of a rewrite that M4 already owns, and would not fix machine-addressability, which is the actual defect. Migrating it whole in M0 and decomposing it in M4 writes each page once, which is the same argument ADR-0026 used to put the site before the 1.0 documentation.
Including Tier 2 code from `examples/` costs one test and repays it every release: the snippets an integrator copies are the snippets CI runs.
## Alternatives considered
[Section titled “Alternatives considered”](#alternatives-considered)
**Rewrite the long guide for 1.0**, the plain reading of N6. Rejected: it preserves both the two-audience conflation and the duplication that produces drift, and it spends M4 effort on prose that MCP tool descriptions and the schemas will serve better.
**Split it into six pages during M0.** Rejected: churn ahead of the M4 rewrite, and a six-page guide is still unfetchable, unvalidatable prose.
**Drop the prose entirely and rely on MCP plus schemas.** Tempting, and it is where a pure runtime story ends. Rejected because it strands everyone who arrives from PyPI or GitHub and integrates the CLI or the library directly, which on current install numbers is most of the user base.
**Use `AGENTS.md` as the vehicle.** Rejected on scope: `AGENTS.md` addresses agents working inside this repository, not agents consuming the library. It may be worth adding for contributors, but it is a different document with a different reader.
## Consequences
[Section titled “Consequences”](#consequences)
Positive: the contract exists in exactly one place per surface; N6 becomes checkable rather than aspirational; M4’s documentation work shrinks to writing one short page and demoting the rest; and `examples/` stops being decorative.
Negative: Tier 0 is build machinery living in `site/` — an `llms.txt` generator and a markdown export — which is more of precisely the JavaScript dependency surface ADR-0026 already named as its real cost. If those plugins go unmaintained we own a small generator. The boundary of R50 still applies: none of it may block a release.
Operationally, the guide’s URL moves. `AGENT_GUIDE.md` at the repository root becomes a stub pointing at the site, and the “Agent Guide” entry in `[project.urls]` repoints there, so PyPI does not keep sending readers to a frozen blob.
There is also a standing judgement call this decision creates rather than settles: for each new piece of prose, whether it is contract (Tier 1) or task (Tier 2). That is a discipline, not a mechanism, and it will be got wrong occasionally.
## Revisit when
[Section titled “Revisit when”](#revisit-when)
If `llms.txt` fails to become something agents actually fetch, Tier 0 loses a limb and the contract page carries more. If MCP becomes the only way anyone integrates, Tier 1 collapses into the skill text of R32c and this ADR is superseded by one that says so. And if the contract page grows past what can be read in a single fetch, that is the signal that contract has leaked back into prose and needs pushing down into schemas and tool descriptions.
## Related
[Section titled “Related”](#related)
ADR-0002, ADR-0011, ADR-0018, ADR-0025, ADR-0026. Requirements N6, R19, R32, R32c, R48.
# Contributing to redlines
> How to report a bug, propose a change, and build the documentation.
First off, thanks for taking the time to contribute! ❤️
All types of contributions are encouraged and valued. See the [Table of Contents](#table-of-contents) for different ways to help and details about how this project handles them. Please make sure to read the relevant section before making your contribution. It will make it a lot easier for us maintainers and smooth out the experience for all involved. The community looks forward to your contributions. 🎉
> And if you like the project, but just don’t have time to contribute, that’s fine. There are other easy ways to support the project and show your appreciation, which we would also be very happy about:
>
> * Star the project
> * Tweet about it
> * Refer this project in your project’s readme
> * Mention the project at local meetups and tell your friends/colleagues
## Table of Contents
[Section titled “Table of Contents”](#table-of-contents)
* [Code of Conduct](#code-of-conduct)
* [I Have a Question](#i-have-a-question)
* [I Want To Contribute](#i-want-to-contribute)
* [Reporting Bugs](#reporting-bugs)
* [Suggesting Enhancements](#suggesting-enhancements)
* [Your First Code Contribution](#your-first-code-contribution)
* [Improving The Documentation](#improving-the-documentation)
* [Styleguides](#styleguides)
* [Commit Messages](#commit-messages)
* [Join The Project Team](#join-the-project-team)
## Code of Conduct
[Section titled “Code of Conduct”](#code-of-conduct)
This project and everyone participating in it is governed by the [CONTRIBUTING.md Code of Conduct](https://github.com/houfu/redlines/blob/main/Code_of_Conduct.md). By participating, you are expected to uphold this code. Please report unacceptable behavior to \.
## I Have a Question
[Section titled “I Have a Question”](#i-have-a-question)
> If you want to ask a question, we assume that you have read the available [Documentation](https://houfu.github.io/redlines).
Before you ask a question, it is best to search for existing [Issues](https://github.com/houfu/redlines/issues) that might help you. In case you have found a suitable issue and still need clarification, you can write your question in this issue. It is also advisable to search the internet for answers first.
If you then still feel the need to ask a question and need clarification, we recommend the following:
* Open an [Issue](https://github.com/houfu/redlines/issues/new).
* Provide as much context as you can about what you’re running into.
* Provide project and platform versions (python version, platform like streamlit, colab etc), depending on what seems relevant.
We will then take care of the issue as soon as possible. (As we work on this in our free time, fixes and reviews may take a while, but we usually respond first within days.)
## I Want To Contribute
[Section titled “I Want To Contribute”](#i-want-to-contribute)
> ### Legal Notice
>
> [Section titled “Legal Notice”](#legal-notice)
>
> When contributing to this project, you must agree that you have authored 100% of the content, that you have the necessary rights to the content and that the content you contribute may be provided under the project license.
### Reporting Bugs
[Section titled “Reporting Bugs”](#reporting-bugs)
#### Before Submitting a Bug Report
[Section titled “Before Submitting a Bug Report”](#before-submitting-a-bug-report)
A good bug report shouldn’t leave others needing to chase you up for more information. Therefore, we ask you to investigate carefully, collect information and describe the issue in detail in your report. Please complete the following steps in advance to help us fix any potential bug as fast as possible.
* Make sure that you are using the latest version.
* Determine if your bug is really a bug and not an error on your side e.g. using incompatible environment components/versions (Make sure that you have read the [documentation](https://houfu.github.com/redlines). If you are looking for support, you might want to check [this section](#i-have-a-question)).
* To see if other users have experienced (and potentially already solved) the same issue you are having, check if there is not already a bug report existing for your bug or error in the [bug tracker](https://github.com/houfu/redlines/issues?q=label%3Abug).
* Also make sure to search the internet to see if users outside of the GitHub community have discussed the issue.
* Collect information about the bug:
* Stack trace (Traceback)
* OS, Platform and Version (Windows, Linux, macOS, x86, ARM)
* Python version, runtime environment, package manager, depending on what seems relevant.
* Possibly your input and the output
* Can you reliably reproduce the issue? And can you also reproduce it with older versions?
#### How Do I Submit a Good Bug Report?
[Section titled “How Do I Submit a Good Bug Report?”](#how-do-i-submit-a-good-bug-report)
> You must never report security related issues, vulnerabilities or bugs including sensitive information to the issue tracker, or elsewhere in public. Instead sensitive bugs must be sent by email to \.
We use GitHub issues to track bugs and errors. If you run into an issue with the project:
* Open an [Issue](https://github.com/houfu/redlines/issues/new). (Since we can’t be sure at this point whether it is a bug or not, we ask you not to talk about a bug yet and not to label the issue.)
* Explain the behavior you would expect and the actual behavior.
* Please provide as much context as possible and describe the *reproduction steps* that someone else can follow to recreate the issue on their own. This usually includes your code. For good bug reports you should isolate the problem and create a reduced test case.
* Provide the information you collected in the previous section.
Once it’s filed:
* The project team will label the issue accordingly.
* A team member will try to reproduce the issue with your provided steps. If there are no reproduction steps or no obvious way to reproduce the issue, the team will ask you for those steps and mark the issue as `needs-repro`. Bugs with the `needs-repro` tag will not be addressed until they are reproduced.
* If the team is able to reproduce the issue, it will be marked `needs-fix`, as well as possibly other tags (such as `critical`), and the issue will be left to be [implemented by someone](#your-first-code-contribution).
### Suggesting Enhancements
[Section titled “Suggesting Enhancements”](#suggesting-enhancements)
This section guides you through submitting an enhancement suggestion for CONTRIBUTING.md, **including completely new features and minor improvements to existing functionality**. Following these guidelines will help maintainers and the community to understand your suggestion and find related suggestions.
#### Before Submitting an Enhancement
[Section titled “Before Submitting an Enhancement”](#before-submitting-an-enhancement)
* Make sure that you are using the latest version.
* Read the [documentation](https://houfu.github.io/redlines). carefully and find out if the functionality is already covered, maybe by an individual configuration.
* Perform a [search](https://github.com/houfu/redlines/issues) to see if the enhancement has already been suggested. If it has, add a comment to the existing issue instead of opening a new one.
* Find out whether your idea fits with the scope and aims of the project. It’s up to you to make a strong case to convince the project’s developers of the merits of this feature. Keep in mind that we want features that will be useful to the majority of our users and not just a small subset. If you’re just targeting a minority of users, consider writing an add-on/plugin library.
#### How Do I Submit a Good Enhancement Suggestion?
[Section titled “How Do I Submit a Good Enhancement Suggestion?”](#how-do-i-submit-a-good-enhancement-suggestion)
Enhancement suggestions are tracked as [GitHub issues](https://github.com/houfu/redlines/issues).
* Use a **clear and descriptive title** for the issue to identify the suggestion.
* Provide a **step-by-step description of the suggested enhancement** in as many details as possible.
* **Describe the current behavior** and **explain which behavior you expected to see instead** and why. At this point you can also tell which alternatives do not work for you.
* You may want to **include screenshots and animated GIFs** which help you demonstrate the steps or point out the part which the suggestion is related to. You can use [this tool](https://www.cockos.com/licecap/) to record GIFs on macOS and Windows, and [this tool](https://github.com/colinkeenan/silentcast) or [this tool](https://github.com/GNOME/byzanz) on Linux.
* **Explain why this enhancement would be useful** to most users. You may also want to point out the other projects that solved it better and which could serve as inspiration.
### Your First Code Contribution
[Section titled “Your First Code Contribution”](#your-first-code-contribution)
Be relaxed and open to comments. We are all here to learn and grow together.
### Improving The Documentation
[Section titled “Improving The Documentation”](#improving-the-documentation)
The Documentation is contained in the python source files as docstrings. Edit them directly. Please be clear, concise and respectful in your tone.
To preview documentation, clone the project, install the development dependencies, then run `npm install && npm run dev` in `site/` — the dev server regenerates the API reference from your docstrings before it starts.
## Documentation
[Section titled “Documentation”](#documentation)
Documentation is published from `site/`, an Astro Starlight project that serves the hand-written pages and, under `/api/`, the API reference that `pdoc` generates from docstrings. See [ADR-0026](/redlines/project/adr/0026-docs-site-on-astro-starlight/) for why, and [ADR-0027](/redlines/project/adr/0027-agent-docs-machine-surface/) for the shape of the agent-facing pages.
### How It Works
[Section titled “How It Works”](#how-it-works)
* The site deploys **on every push to `main`**, and again when a release is published. A documentation fix does not have to wait for a release; the consequence is that `/api/` documents `main` rather than the last tag.
* Every pull request builds the site without deploying it, so a release is never the first time the site is built.
* **A broken site never blocks a release.** The website workflow is separate from the packaging workflows, nothing depends on it, and it must not be made a required status check.
### To Update Documentation
[Section titled “To Update Documentation”](#to-update-documentation)
* **API reference:** edit the docstrings in `redlines/`. They remain the source of truth; `pdoc` renders them and nothing under `site/public/api/` should be edited by hand — it is generated and not committed.
* **Everything else:** edit the pages under `site/src/content/docs/`.
### Docstring conventions
[Section titled “Docstring conventions”](#docstring-conventions)
Two conventions, both there for the same reason: the API reference generator is expected to change at M4, and these keep that a configuration change rather than a rewrite of every docstring written between now and then.
* **Write docstrings in reST style** — `:param name:`, `:type name:`, `:return:`, `:rtype:` — which is what the package already uses throughout. Do not mix in Google or numpy sections. Both pdoc and `griffe`, the extractor every plausible successor is built on, read reST; a mixed codebase means one of them renders half the parameters as literal text.
* **Do not add `@private` to a docstring.** It is a pdoc-specific pragma that no other generator recognises: under `griffe` the member reappears in the reference with a literal `@private` line in its body. To keep something out of the published reference, use `__all__`, or raise it in the pull request so the module can be excluded by configuration. The nine existing pragmas in `redlines/cli.py` are doing a real job for pdoc today and stay until the generator changes; the rule is about not adding a tenth.
### Building the site locally
[Section titled “Building the site locally”](#building-the-site-locally)
```bash
cd site
npm install
npm run dev # runs pdoc first, then serves on http://localhost:4321/redlines/
```
`npm run build` does the same for a production build, and `npm run api` regenerates only the pdoc output. All three need the Python development environment (`uv sync --all-extras --dev`) available in the repository root.
### Written documentation
[Section titled “Written documentation”](#written-documentation)
Not everything belongs in a docstring. `docs/` holds the documents that are written by hand rather than generated:
* `docs/adr/` — architecture decision records: why a design choice was made, what was rejected, and when to revisit it. Adding a decision means adding a file here; see [docs/adr/README.md](/redlines/project/adr/) for the conventions.
* `docs/PRD.md` — what redlines 1.0 is and who it is for.
* `docs/competitive-landscape-2026-08.md` — the survey the 1.0 decisions rest on.
* `ROADMAP.md` — which release each feature lands in.
## Styleguides
[Section titled “Styleguides”](#styleguides)
For code, we use the [Black formatter](https://black.readthedocs.io/en/stable/index.html). Please be encouraged to use that too when submitting code contributions.
Please write tests to explain your work.
Please ensure all the tests pass before we can consider mergining your work.
### Commit Messages
[Section titled “Commit Messages”](#commit-messages)
It’s helpful to use a “verb” (like add, fix etc) and a short description of what you are trying to accomplish.
## Join The Project Team
[Section titled “Join The Project Team”](#join-the-project-team)
Please \[email me]\(mailto:houfu\@OUTLOOK dot sg) or file an [issue](https://github.com/houfu/redlines/issues).
## Attribution
[Section titled “Attribution”](#attribution)
This guide is based on the **contributing.md**. [Make your own](https://contributing.md/)!
# PRD: redlines 1.0 — a structural redliner
> What redlines 1.0 is, who it is for, and what it must do.
**Status:** reconciled with the adopted ROADMAP, 30 August 2026 (revision 10: ROADMAP.md was adopted rather than left as a candidate for curation, so this revision annotates every Must/Should requirement the roadmap’s section 5 trims below the tag stated here, with its actual 1.0/1.1 release — the roadmap wins on release assignment, this document is updated to match. Also records the #96 finding on the 18 `neurotic_docx_bench` failures in section 11, and prunes section 13’s open questions that the roadmap has since answered. Revision 9: the agent-facing documentation is reshaped by ADR-0027 into a machine surface plus a contract page, which rewrites N6 and R48. Revision 8: the documentation platform is decided in ADR-0026 — Astro Starlight, in one site with the demo — which adds section 7.12 and resolves the last of section 13’s site questions. Revision 7: decisions hived off into ADRs under `docs/adr/`; section 6 is now an index and a D-number map) **Owner:** houfu **Companions:** [`docs/adr/`](/redlines/project/adr/) (the decisions, with rationale and alternatives — section 6 maps the old D-numbers onto them), [`docs/competitive-landscape-2026-08.md`](https://github.com/houfu/redlines/blob/main/docs/competitive-landscape-2026-08.md) (the landscape survey the decisions rest on), and [`ROADMAP.md`](/redlines/project/roadmap/) (the authority on which release each feature is in; where the Must/Should/1.1 tags below disagree with it, the roadmap wins once curated)
***
## 1. Why this document exists
[Section titled “1. Why this document exists”](#1-why-this-document-exists)
redlines is a 2023 design: flatten a document into one token stream, mark paragraph boundaries with a `¶` token, run one `difflib.SequenceMatcher`, render the opcodes. It has done well on that design (3.7M downloads, the DeepLearning.AI course, an agent-first CLI), and it is now stuck on it. The 2026 field has moved to typed document models and tracked-changes output for AI agents, but every competitor is bound to OOXML and none exposes a change model above “these words changed in paragraph 37”.
This PRD defines redlines 1.0 as a **format-neutral structural comparison engine**: documents become trees of blocks, blocks are aligned before text inside them is diffed, and the result is a change tree with human-readable addresses that a person can review and an agent can act on. It records the decisions that shape that product, with alternatives and status, so they can be argued with before code exists.
## 2. Problem
[Section titled “2. Problem”](#2-problem)
Three groups have a problem the flat spine cannot solve.
Developers building LLM and agent pipelines need to answer “what did the model change, where, and was that all it was supposed to change” in a form they can log, assert on and show to a human. Today they get a global list of token spans and a markdown string. They cannot ask about clause 7.2, cannot tell a move from a delete-plus-insert, and cannot verify scope.
People comparing two versions of a structured document (contracts, policies, legislation, long markdown) think in clauses, sections and rows. A flat diff of a document where a clause moved shows a large deletion and a large insertion, and a renumbering shows every label as changed. Word Compare and the commercial tools handle some of this inside DOCX; nothing does it for markdown, plain text, PDF-extracted text or HTML, which is where most agent-era content lives.
redlines’ own users hit silent cliffs in the current engine: `autojunk` marks whole repetitive schedules as replaced, adjacent edits split by punctuation count as two changes, and sentence mode discards paragraph structure. These are symptoms of structure living in a token instead of a model.
## 3. Goals and non-goals
[Section titled “3. Goals and non-goals”](#3-goals-and-non-goals)
**Goals for 1.0**
G1. Compare two documents of any supported format and return a change tree whose nodes are block-level operations (insert, delete, modify, move, renumber) containing inline operations, each addressable by a stable, human-readable path.
G2. Keep the existing string API, markdown styles, JSON v1 output and CLI working unchanged, implemented over the new core.
G3. Give agent pipelines two first-class operations beyond compare: a deterministic plain-text summary of changes for LLM consumption, and a verification that an edited document changed only what it was permitted to change.
G4. Read plain text and markdown in the core with no third-party dependencies; read DOCX through an optional extra.
G5. Produce tracked-changes DOCX by delegation to an existing applier rather than by writing OOXML.
G6. Publish an alignment benchmark (metric, corpus, baselines) so that the structural claims are measurable and so that the project has an external quality signal beyond the visual-fidelity benchmark it currently sits at the bottom of.
**End deliverables.** Two things must exist at the end of this cycle, and the library work above is in service of them:
G7. **An MCP server** (`redlines-mcp`, a separate package) that gives agents compare, summary, annotate, verify and block-reading tools over local files or inline content, and, distinctively, lets a model *author a structure profile* for a new document family through a prompt-plus-preview loop (D26, D30) without any model call inside the library. It ships with a skill file and registry listings, first, immediately after the 1.0 core, because it reaches the primary persona directly and forces the change-tree, summary, verify and profile contracts to be right.
G8. **A demo website** where a visitor drops two plain-text or markdown files, or pastes text, and gets the structural comparison rendered: block-level changes with roles and addresses, inline redlines within them, the annotated document, the LLM summary, and the JSON to copy. It runs entirely in the browser (Pyodide/WASM), so documents never leave the visitor’s machine and hosting is static. It is a usable tool, not a screenshot: it must handle a 100-page contract in markdown. A built-in sample pair (section 3a) shows every capability in one click.
**The 1.0 slice.** 1.0 is deliberately the smallest thing that demonstrates the thesis end to end: text and markdown readers, the semantic pass, block alignment with moves and renumbering, the change tree, the annotated and summary renderers, verify, the CLI, the MCP server, and the site. Every other format is 1.1 or later. The reason is that the top requirements (semantic roles, alignment, moves) need clean text and nothing else; PDF brings layout analysis and OCR questions, DOCX brings a dependency and a reader to maintain, and neither adds anything to what the demo has to prove.
**Non-goals for 1.0**
DOCX, PDF and HTML reading (1.1). A bespoke OOXML parser or writer. Inline formatting change detection (bold, italic, style) as a diff category. Comments, footnotes, headers and footers. Images. **OCR of any kind, and any call to an LLM from inside the library**: the semantic pass is deterministic heuristics over text, always. Real-time or collaborative diffing. A desktop or native GUI. Three-way merge. Server-side processing or storage of uploaded documents on the demo site.
## 3a. The demo scenario
[Section titled “3a. The demo scenario”](#3a-the-demo-scenario)
The site, the CLI examples, the MCP `SKILL.md` and the README all use one sample pair, so that a visitor sees every capability without uploading anything. The pair is a short services agreement in markdown, about forty clauses with a definitions section, a schedule and one table, and an amended version that contains, deliberately, one of each thing the engine detects: a definition whose text changed (“Confidential Information”); a clause moved from section 7 to section 9 with a small edit inside it; a renumbering caused by an inserted clause; a cross-reference updated to follow the renumbering; a deleted sub-clause; an inserted table row; a whitespace-only change that should be reported as nothing; and an edit inside a repetitive schedule, which the flat 0.6 engine gets wrong and the structural engine gets right. The expected change tree for this pair is a golden file; it is the first test written and the last one allowed to fail.
## 4. Users
[Section titled “4. Users”](#4-users)
The primary user is the **LLM/agent pipeline developer**: writes Python, uses redlines in a notebook, a Streamlit app, a test suite or an agent loop; wants JSON, deterministic output, and a CLI; increasingly wants an MCP surface. Everything in 1.0 is sequenced for this person, and the MCP server (G7) is their front door.
The secondary user is the **document engineer** with two versions of a structured document who wants clause-level comparison without Word. This person drives the block model and alignment requirements but does not drive DOCX fidelity work in 1.0. The demo site (G8) is their front door, and the one that turns a curious visitor into a library user.
The user who must not be harmed is the **course learner and notebook author** who does `Redlines(a, b).output_markdown` and expects the same string tomorrow.
## 5. Product principles
[Section titled “5. Product principles”](#5-product-principles)
Format-neutral at the boundary: anything that can be turned into blocks can be compared. Inspectable in the middle: the change tree is data first and rendering second. Addressable: every change has a path a human would recognise. Honest about scope: when a reader drops content (a table, a header), the output says so. Zero-install core: `pip install redlines` keeps working with stdlib, click and rich only.
## 6. Decisions
[Section titled “6. Decisions”](#6-decisions)
The decisions behind this PRD, with their alternatives, rationale and revisit conditions, live as architecture decision records in [`docs/adr/`](/redlines/project/adr/). They were moved out of this document deliberately: a PRD gets rewritten as the product changes, and rewriting erases reasoning. An ADR is written once and superseded rather than edited, so the trail survives.
Read [`docs/adr/README.md`](/redlines/project/adr/) for the index and the conventions. The load-bearing ones, if you read only four: [ADR-0001](/redlines/project/adr/0001-format-neutral-structural-engine/) (why a structural engine at all), [ADR-0005](/redlines/project/adr/0005-minimal-core-open-semantic-layer/) and [ADR-0006](/redlines/project/adr/0006-structure-profiles/) (the semantic layer and declared structure, which are what set this apart), and [ADR-0013](/redlines/project/adr/0013-the-1-0-slice-text-and-markdown/) (why 1.0 is only text and markdown).
### Traceability from earlier revisions
[Section titled “Traceability from earlier revisions”](#traceability-from-earlier-revisions)
Revisions 1 to 6 of this PRD numbered decisions D1 to D30 in a table here. That numbering is referenced in the requirements below, in `ROADMAP.md` and in earlier conversation, so the mapping is preserved:
| Was | Now |
| ------------- | ------------------------------------------------------------------------------ |
| D1 | ADR-0001 Format-neutral structural engine |
| D2 | ADR-0002 Primary persona |
| D3 | ADR-0003 Compatibility facade |
| D4 | ADR-0004 Stdlib core, optional extras |
| D5 | ADR-0005 Minimal core, open semantic layer |
| D6 | ADR-0008 Multi-pass block alignment |
| D7 | ADR-0009 Moves before splits |
| D8, D9 | ADR-0010 Keep difflib for leaf diffs |
| D10 | ADR-0011 JSON canonical, annotated renderer |
| D11 | ADR-0012 HTML-like addresses |
| D12, D16, D24 | ADR-0013 The 1.0 slice: text and markdown |
| D13 | ADR-0014 No OOXML writing |
| D14 | ADR-0015 Verify mode in 1.0 |
| D15 | ADR-0016 Summary renderer in core |
| D17 | ADR-0006 Structure profiles (the plain-text reader’s rules are profile-driven) |
| D18 | ADR-0017 Separate MCP package |
| D19 | ADR-0021 Alignment benchmark |
| D20 | ADR-0022 Keep the name |
| D21 | ADR-0023 Python support and typing |
| D22 | ADR-0024 No formatting change detection |
| D23, D27, D28 | ADR-0019 Client-side demo site |
| D25 | ADR-0020 MCP before site |
| D26 | ADR-0018 MCP tools, prompts and resources |
| D29 | ADR-0025 CLI as a thin skin |
| D30 | ADR-0006 Structure profiles |
| (non-goal) | ADR-0007 No OCR, no LLM in the library |
Decisions taken after revision 7 have no D-number and appear only as ADRs: [ADR-0026](/redlines/project/adr/0026-docs-site-on-astro-starlight/), the documentation platform.
## 6a. Change-tree wire format
[Section titled “6a. Change-tree wire format”](#6a-change-tree-wire-format)
The JSON-versus-XML question and the evidence behind it are recorded in [ADR-0011](/redlines/project/adr/0011-json-canonical-annotated-renderer/). Summary: JSON is canonical, schema-published and versioned; an annotated-document renderer (changes marked in place, CriticMarkup for text and markdown) is the second first-class representation, and is what a model should read when it needs surrounding context.
## 6b. How plain text becomes structure (design note; the decision is ADR-0006)
[Section titled “6b. How plain text becomes structure (design note; the decision is ADR-0006)”](#6b-how-plain-text-becomes-structure-design-note-the-decision-is-adr-0006)
The plain-text reader runs five mechanical stages, and then the semantic pass runs on the result. Each stage records what it decided and why, so a mis-parse is visible.
**Normalise and segment.** Line endings and whitespace are normalised; the text is split into candidate paragraphs on blank lines; hard-wrapped lines are re-joined when a line ends mid-sentence and the next begins lowercase. This is what 0.6 does today, made explicit.
**Detect labels.** Each candidate paragraph is tested for a leading label from the active profile’s patterns (`1.`, `1.1`, `(a)`, `(i)`, `A.`, `Article 5`, `Section 3`, `Schedule 2`, `§ 4`, `4.—(1)`). The label is stripped and kept as `label`; its style (decimal-dotted, alpha-paren, roman-paren, word-prefixed) is kept for the next stage. Unlabelled paragraphs are continuation candidates.
**Infer hierarchy.** Decimal-dotted labels carry their own depth and nest by arithmetic. Alpha and roman labels are ambiguous in isolation (“(i)” after “(h)” is alphabetic; after “7.2” it is roman and one level deeper), so depth is resolved from the label-style stack: a style already on the stack pops back to its level, a new style pushes one deeper. Headings the profile marks as numbering resets (schedules, annexes, parts) open a new section and clear the stack, which is what makes labels unambiguous again after a schedule boundary. Indentation is a secondary signal where it survives.
**Attach continuations.** Unlabelled paragraphs following a labelled block become its body children unless a heading rule claims them.
**Recognise headings.** Short lines in all caps or title case without terminal punctuation, followed by labelled content, score as headings; the score is kept rather than thresholded away, and a profile can tighten or loosen the rule.
**Semantic pass.** On the tree, not the text: a section whose heading matches the profile’s definitions rule, or whose children mostly match the *quoted term, “means”, text* shape, gets role `definitions` and its children role `definition` with a `defined_term` span; blocks under a schedule heading get role `schedule`; text matching the profile’s citation patterns becomes a `cross_reference` span carrying the referenced label, which is what lets the engine later say “cross-reference updated to follow renumbering” rather than “text changed”; parties, dates and amounts are span regexes; emphasis exists only where the format has it (markdown).
**Markdown.** The markdown reader replaces the first three stages with the syntax itself (`#` depth, list markers, pipe tables, fences) and then runs the same label detection on list-item and paragraph text, the same continuation logic, and the same semantic pass under the `markdown` profile. A markdown contract with `## 7. Termination` and `1.` list items therefore gets the same roles and labels as its plain-text twin.
**Profiles (D30).** All of the above is parameterised by a profile: which label patterns exist and their precedence; which headings reset numbering; heading rules; role rules; span extractors. The built-in `contract`, `legislation`, `markdown` and `generic` profiles cover the demo and the primary persona; auto-selection scores a sample of the document against each and reports the winner and confidence. **1.0 ships `contract`, `markdown` and `generic`; `legislation` and auto-selection move to 1.1 (ROADMAP § 5.1–5.2, accepted) — 1.0 defaults to `contract` for plain text and `markdown` for `.md`, with `--profile` to override.** A profile is short enough for a person to write for their own precedent bank in half an hour and for an LLM to draft from a sample document in one prompt, outside the library. The library’s contract is: apply the profile deterministically, report `matched_by` and confidence per block, and degrade to one block per paragraph, with alignment still working, when nothing matches.
**Known hard cases,** to be in the test corpus from the start: alpha/roman ambiguity at `(i)`; numbering that restarts inside schedules; one-line clauses that look like headings; definitions written as a run-on paragraph rather than a list; cross-references in prose (“the preceding sub-clause”); documents that mix two label styles because two drafters edited them; and text extracted from PDF with page headers interleaved, which is out of 1.0 but should not crash the reader.
## 7. Functional requirements
[Section titled “7. Functional requirements”](#7-functional-requirements)
Requirements are numbered for traceability and written so each is testable. “Must” is 1.0; “Should” is 1.x; “May” is later.
### 7.1 Block model
[Section titled “7.1 Block model”](#71-block-model)
R1. A document is an ordered tree of blocks. Each block has a kind from a closed set (`document`, `section`, `heading`, `paragraph`, `list_item`, `table`, `row`, `cell`, `unknown`), text, an optional label, a depth, a path derived from position, and an `attrs` mapping for reader-specific detail. **Must.**
R1a. Each block may carry an optional semantic `role` and a list of semantic `spans` (type, start, end) per D5; the vocabulary is open, with a recommended set documented and used by the built-in heuristics. **Must.**
R1b. A pluggable semantic pass runs after reading and before alignment, assigning roles and spans from the active profile’s rules (clause labels, “means” definitions, quoted defined terms, cross-references to labels, party names, dates, amounts). The built-in profiles ship in core; users can register their own passes and profiles. **Must.**
R1d. Structure profiles per D30: a documented declarative format; built-in `generic`, `contract`, `legislation`, `markdown`; explicit selection or auto-selection with reported confidence; per-block `matched_by` and confidence; a tree-level fallback count. **Must** for the format, the `generic`/`contract`/`markdown` profiles, explicit selection, and per-block `matched_by`/confidence/fallback count. The `legislation` profile and auto-selection with confidence are **1.1** (ROADMAP § 5.1–5.2, accepted).
R1e. A profile can be loaded from a file or passed as a mapping at call time, from the CLI (`--profile`), from the MCP tools, and by pasting on the site, so a profile drafted by a model in one place is reusable everywhere. **Must.**
R1f. The profile format is flat, plainly named and commented, with a published schema, so that a model given the schema and one built-in example can write a valid profile for a new document family in a single turn; this legibility is a design requirement, not a documentation nicety. **Must.**
R1c. Change nodes carry the role of the block they affect and the span types touched, so summaries can say “definition modified” or “cross-reference inserted”. **Must.**
R2. Block text is the comparison key; `attrs`, roles and spans never drive alignment or diffing in 1.0, except that role may break ties between otherwise equal fuzzy candidates. **Must.**
R3. Every reader reports what it dropped (kinds and counts) so output can disclose scope. **Must.**
### 7.2 Readers
[Section titled “7.2 Readers”](#72-readers)
R4. Plain-text reader per D17. **Must.** R5. Markdown reader per D16, stdlib regex only. **Must.** R6. DOCX reader per D12 as the `[docx]` extra. **1.1.** R7. A reader interface so users can supply their own (HTML, DOCX, Akoma Ntoso, JSON, anything) by producing blocks; documented with a worked example so a third party can contribute a reader without touching the core. **Must.** R8. HTML reader on the stdlib parser. **1.1.** R8a. PDF reader as the `[pdf]` extra on pypdf, text extraction only, structure flagged as inferred; never OCR. **1.1.** R8b. Format detection from extension and content sniffing for the 1.0 formats (txt, md), extensible for later readers; unknown types are reported, not guessed. **Must.**
### 7.3 Alignment
[Section titled “7.3 Alignment”](#73-alignment)
R9. Multi-pass alignment per D6 with configurable thresholds and a record, per aligned pair, of which pass matched it. **Must.** R10. Move detection: an unmatched deleted block that matches an unmatched inserted block elsewhere at or above the fuzzy threshold is reported as a move, with both addresses. **Must.** R11. Renumbering: matched content with differing labels is reported as a renumber, not as inline text edits of the label. **Must.** R12. Split and merge detection. **Should.** R13. Alignment is deterministic for identical inputs and configuration. **Must.** R14. Tables align row by row, then cell by cell; row insert/delete is reported at row level. **Must** for DOCX and markdown pipe tables. 1.0 scope is kept minimal — row insert/delete and cell-level inline diff for markdown pipe tables only, no column operations and no merged cells (ROADMAP § 5.8, accepted); DOCX tables apply once the DOCX reader lands in 1.1.
### 7.4 Leaf diff
[Section titled “7.4 Leaf diff”](#74-leaf-diff)
R15. Word-level diff inside aligned pairs with `autojunk` disabled and the cleanup pass from D8. **Must.** R16. Sentence-level leaf tokenisation as an option, paragraphs preserved (D9). **Must.** R17. Leaf diff is pluggable (processor interface retained). **Must.**
### 7.5 Change tree and outputs
[Section titled “7.5 Change tree and outputs”](#75-change-tree-and-outputs)
R18. A change tree per D10 with block operations `insert`, `delete`, `modify`, `move`, `renumber` and, from 1.1, `split`, `merge`; `modify` nodes contain inline `insert`, `delete`, `replace`. **Must.** R19. JSON v2 serialisation with a published schema; v1 JSON unchanged and still produced by the existing method. **Must.** R20. Markdown, rich-terminal and HTML renderers over the tree, byte-identical to 0.6 output for the plain-string path on the existing test suite. **Must.** The HTML renderer’s 1.0 scope is minimal — a block list with roles and addresses, expandable inline redlines; richer interaction is the demo site’s job, not a second renderer (ROADMAP § 5.7, accepted). R21. LLM summary renderer per D15, built in the MCP milestone, implemented in core. **Must.** R21a. Annotated-document renderer per section 6a: the source text with changes marked in place (CriticMarkup for text and markdown; tag variant for HTML), block roles shown as prefixes. **Must** in 1.0 — the MCP server’s `summary` and `annotate` tools ship alongside 1.0 and need it, so the PRD’s original hedge (“Should” pending “Must before MCP ships”) resolves to a plain Must (ROADMAP § 5.6, accepted). R22. Per-block and per-section statistics; change density by section. **Must.** R23. Filtering: changes by kind, by address prefix, by label, by minimum size. **Must.**
### 7.6 Verify
[Section titled “7.6 Verify”](#76-verify)
R24. `verify(original, edited, allowed)` where `allowed` is a set of addresses, labels or text anchors; returns a result with pass/fail, out-of-scope changes, and structural side effects. **Must** for `allowed` scoped by address or label. Free-text-anchor scope is **1.1** (ROADMAP § 5.3, accepted) — anchors reopen the ambiguity problem adeu is fighting. R25. Verify treats whitespace-only and label-only changes as configurable exemptions. **Must.**
### 7.7 DOCX output by delegation
[Section titled “7.7 DOCX output by delegation”](#77-docx-output-by-delegation)
R26. Export the change tree as an adeu edit batch (`ModifyText` with `target_text`, table row operations) and as a superdoc-redlines edits file keyed by block. **Deferred to 1.1** (D13). R27. When an applier is installed, a convenience call runs the full compare-and-apply round trip and returns DOCX bytes. **Deferred to 1.1** (D13). R27a. The change tree’s design must not preclude R26: every inline change must be recoverable as (block address, old text, new text) with enough surrounding context to anchor a text search. **Must** in 1.0.
### 7.8 CLI and agent surface
[Section titled “7.8 CLI and agent surface”](#78-cli-and-agent-surface)
R28. `redlines compare A B` accepts .txt, .md, `-` for stdin and bare strings; defaults to the new tree output when either input is a file, v1 JSON when both are bare strings (compatibility); `--format` overrides. **Must.** R29. `redlines summary A B`, `redlines annotate A B` and `redlines verify A B --allow ...`. **Must.** R30. Existing subcommands (`text`, `markdown`, `stats`, `json`) and the command-less default unchanged. **Must.** R30a. CLI and MCP share one argument-normalisation layer (path, stdin or inline content; format hint; size limit) so behaviour is identical across the two skins (D29). **Must.**
### 7.9 MCP server (`redlines-mcp`, deliverable G7)
[Section titled “7.9 MCP server (redlines-mcp, deliverable G7)”](#79-mcp-server-redlines-mcp-deliverable-g7)
R31. Separate package depending on a pinned compatible range of `redlines`; `redlines-mcp` console entry point; stdio and streamable HTTP transports. **Must** for the package and the stdio transport. The streamable HTTP transport is **1.1** (ROADMAP § 5.4, accepted) — Claude Code, Claude Desktop and Cursor all use stdio; HTTP matters for hosted agents, which are not the first audience. R32. Tools per D26: `compare` (change tree JSON), `summary` (summary text), `annotate` (annotated document), `verify` (verification result), `read_blocks` (one document’s block tree, for picking addresses), `preview_structure` (block tree with `matched_by`, confidence and fallback count under a given profile), `validate_profile` (schema and pattern errors, with line references). Each accepts a path or inline content, an optional format hint, and an optional profile by path or inline. **Must.** R32a. Prompts per D26: `draft_profile`, `refine_profile`, `explain_changes`; each is a template the server fills from its resources and the user’s document sample, so the model receives the profile format, a worked example and the target text in one turn. **Must** for `draft_profile` and `refine_profile` — the profile-authoring loop is the distinctive thing. `explain_changes` is **1.1** (ROADMAP § 5.5, accepted); the `summary` tool already gives a model what it needs. R32b. Resources: the profile schema, every built-in profile, the change-tree schema, and the skill text, each addressable by URI so a client can read them without a tool call. **Must.** R32c. The profile-authoring loop (draft → `preview_structure` → refine → `compare`) is documented as the canonical workflow in the skill text, with a worked transcript against the section 3a sample; a golden test replays it with a fixed profile draft. **Must.** R33. Tool and prompt descriptions and a `SKILL.md` written for models: when to use which tool, how to read addresses and confidence, how to build an `allowed` scope, how to author and save a profile. **Must.** R34. Size guards: inputs above a configurable limit return an error naming the limit rather than timing out; responses can be truncated by block count with a continuation hint. **Must.** R35. Listed in the MCP registries agents actually use (the official registry, Smithery, glama), with a one-line install for Claude Code, Claude Desktop and Cursor. **Must.** R36. Golden tests that call every tool over stdio with fixture documents and compare against stored JSON. **Must.**
### 7.10 Demo site (deliverable G8)
[Section titled “7.10 Demo site (deliverable G8)”](#710-demo-site-deliverable-g8)
R37. Static site, no backend; loads the published `redlines` wheel into Pyodide in a web worker so the UI stays responsive. No extras needed in 1.0. **Must.** R38. Two inputs, each either a dropped/uploaded file (txt, md) or pasted text; format detected per R8b; other file types produce a friendly “not yet, coming in 1.1” message; the section 3a sample pair loads with one click and is the default state of the page. **Must.** R39. Output views: block-change list with addresses and change kinds (insert, delete, modify, move, renumber), each expandable to the inline redline; the summary text; JSON v2 with a copy button; per-file `dropped` notice; per-section change density. **Must**, except per-section change density, which is **1.1** (ROADMAP § 5.9, accepted) — the stats already exist in the JSON from M2; drawing them is UI work that does not prove anything new. R40. Handles a 100-page contract pair in markdown (roughly 2,000 blocks) within ten seconds after the runtime has loaded, with a visible progress state. **Must.** R41. Nothing leaves the browser: no upload endpoint, no content analytics; a plain statement to that effect on the page. **Must.** R42. Works without install on current Chrome, Firefox and Safari; shows a clear message when Pyodide fails to load rather than a blank page. **Must.** R43. Side-by-side pane view with synchronised scrolling. **Should** (1.1). R44. Shareable permalink that encodes both inputs client-side (compressed in the URL fragment) for small documents. **May.**
### 7.11 Compatibility
[Section titled “7.11 Compatibility”](#711-compatibility)
R45. The 0.6 public API (`Redlines`, `compare`, `output_markdown`, `output_rich`, `output_json`, `changes`, `get_changes`, `stats`, `opcodes`, processors, `Document`, `PlainTextFile`) keeps working with no code changes; the existing test suite passes unmodified. **Must.** R46. Deprecation warnings, not removals, for anything superseded. **Must.**
### 7.12 Documentation site (ADR-0026)
[Section titled “7.12 Documentation site (ADR-0026)”](#712-documentation-site-adr-0026)
R47. Documentation is published from a single Astro Starlight project in `site/`, deployed to GitHub Pages, replacing pdoc as the publishing surface. Hand-written pages and the generated API reference live together, the latter built with pdoc and served under `/api/`; docstrings stay the source of truth for the API reference. **Must.** R48. By 1.0 the site carries: quickstart and install; the agent-facing documentation of ADR-0027 — a machine surface (`llms.txt`, per-page markdown, fetchable schemas) with one contract page over compare, summary, annotate and verify, and task pages whose code comes from `examples/`; the JSON v2 schema and the profile schema, each with a worked example; a profile-authoring guide; the alignment benchmark report of ADR-0021; and the ADR index. **Must.** R49. The demo site of 7.10 is a route in that same project, sharing its build and deployment; documentation pages link to it and it links back. **Must.** R50. The boundary holds in both directions: nothing under `site/` is imported by the wheel, nothing in the wheel depends on the site building, and a failing site build never blocks a release. **Must.**
## 8. Non-functional requirements
[Section titled “8. Non-functional requirements”](#8-non-functional-requirements)
N1. Determinism: identical inputs and configuration produce identical trees, JSON and summaries. N2. Performance: a 200-page contract (roughly 60k words, 2,000 blocks) compares in under five seconds on a laptop in pure stdlib mode and under ten seconds in the browser; the alignment step is at worst quadratic in block count with early exit on exact matches. N3. Memory: block trees hold text once; inline diffs are computed per pair, not over the whole document. N4. Typing: strict mypy, `py.typed`. N5. Packaging: uv-managed, hatchling, extras as in D4; the core wheel and every 1.0 extra import under Pyodide, checked in CI (D28). N6. Documentation: everything in 7.12 — the JSON v2 and profile schemas published with worked examples, the agent-facing documentation of ADR-0027 covering compare, summary, annotate and verify, and the benchmark report — on the Starlight site of ADR-0026, which is stood up in the 0.6.x hygiene release so that no 1.0 page is written twice; the MCP package and the demo each link back to it. N7. The site and the MCP server are thin: no comparison logic lives outside the core library, so a behaviour seen on the site is reproducible from Python with the same inputs.
## 9. Interface sketch (names, not code)
[Section titled “9. Interface sketch (names, not code)”](#9-interface-sketch-names-not-code)
The new entry point is a single function that takes two inputs of any supported kind and returns a comparison object. The comparison object exposes the source and test block trees, the change tree, statistics, filters, and render methods for markdown, rich, HTML, JSON v2 and summary. A separate `verify` function takes the same inputs plus an allowed scope and returns a verification result. Readers are classes with one method that turns bytes or text into a block tree; the DOCX reader lives behind the extra. The existing `Redlines` class becomes a thin facade that builds a one-block-per-paragraph tree and renders through the same renderers.
I have deliberately not written signatures; that is the first design task once this PRD is agreed.
## 10. Success metrics
[Section titled “10. Success metrics”](#10-success-metrics)
For the alignment benchmark (D19): precision and recall of block correspondences, move detection recall, and renumbering recall on the labelled corpus; target ≥ 0.95 correspondence F1 on synthetic mutations and ≥ 0.85 on hand-labelled real pairs at 1.0, with flat redlines 0.6 as the floor and python-redlines as the comparator on DOCX. The hand-labelled set is capped at ten pairs for 1.0 (ROADMAP § 5.10, accepted) — enough to catch a wrong move; expansion beyond ten is ongoing 1.1 work, not gated. Per D7, **move detection is a release gate**: 1.0 does not ship with move recall below 0.9 on synthetic mutations or with any move false positive on the hand-labelled set that a reviewer would call wrong. For the semantic layer (D5): precision of role and span assignment on a hand-labelled sample of contracts, reported but not gated in 1.0. For compatibility: the 0.6 test suite passes unmodified, and the course example string is byte-identical. For the MCP server: installable in Claude Code in one command, listed in at least two registries, every tool passing golden tests, and used by at least one external project within a quarter. For the site: loads and runs a 100-page pair within the R40 budget on a mid-range laptop, zero server cost, and visible referral traffic to the PyPI page and repository. For adoption, over two quarters after release: JSON v2 and summary usage visible in issues and dependents; at least one applier integration used in the wild; the benchmark cited by another project.
## 11. Sequencing
[Section titled “11. Sequencing”](#11-sequencing)
Each step ships on its own and is useful without the next.
1. **0.6.x hygiene release.** `autojunk` off, cleanup pass, sentence mode preserves paragraphs, regression corpus with a repetitive schedule, investigate the 18 benchmark failures. Small, immediate, no design risk. The documentation moves from pdoc to Starlight here (ADR-0026, 7.12): it touches no engine code, so it neither blocks nor is blocked, and every page step 5 owes then gets written once, onto a site that can hold it. **Shipped as 0.6.2, 30 August 2026.** The 18-failure investigation ([#96](https://github.com/houfu/redlines/issues/96)) found none of them are redlines failures: all 18 die in the benchmark adapter’s python-docx extraction step before `Redlines` is ever constructed, so nothing needed fixing in 0.6.x. The failures are carried forward as [#110](https://github.com/houfu/redlines/issues/110), a preview of the 1.1 DOCX reader’s failure modes with named reproducers.
2. **Model, semantics and readers.** Block model with the semantic layer (D5), reader interface with a worked third-party example, `dropped` reporting; plain-text and minimal markdown readers; the legal semantic pass; format detection; the section 3a sample pair and its golden tree. Pyodide import check added to CI here. Existing API untouched.
3. **Alignment and change tree.** D6 passes, moves, renumbering, tables; change tree; serialisation per the D10 decision; filters and per-section stats. The synthetic-mutation corpus and metric are built in this step, before alignment is tuned; the move-recall gate (section 10) is measured here.
4. **Renderers and compatibility.** Markdown, rich, HTML and annotated-document renderers over the tree; `Redlines` facade; 0.6 suite green.
5. **Verify, CLI, docs — the 1.0 release.** Verify mode, the three CLI subcommands over the shared argument layer (D29, about a day), agent guide, schema publication.
6. **`redlines-mcp` 0.1 (G7).** Started as soon as the tree serialisation is frozen in step 3, released within days of 1.0: tools, transports, the LLM summary renderer (D15), SKILL.md, golden tests, registry listings.
7. **Demo site (G8).** Started once step 4’s renderers are stable; released after the MCP server. A route in the documentation site from step 1 (ADR-0026), so this step is the demo itself and not a second build: Pyodide worker, text and markdown inputs, the output views, the sample pair as the default state, privacy statement.
8. **1.1.** HTML reader, then DOCX reader (D12), then PDF text reader; adeu and superdoc-redlines export (D13); split/merge; side-by-side view on the site; `[fuzzy]` tuning from benchmark results; markdown-it reader only if the regex one proves insufficient.
## 12. Risks
[Section titled “12. Risks”](#12-risks)
Alignment quality on real documents is the thesis risk: fuzzy thresholds that work on contracts may misfire on prose or on tables of near-identical rows. Mitigation: the corpus and metric are built before tuning, thresholds are configurable and reported, and every match records its pass.
Compatibility drift: re-implementing `output_markdown` over a tree could change whitespace or paragraph handling in edge cases the test suite does not cover. Mitigation: golden-file tests generated from 0.6 across the README, course and issue examples before the facade is written.
Regex markdown and clause-label heuristics will have false positives. Mitigation: heuristics opt-in where risky, `dropped` and `matched_by` reporting so users can see what happened, and an easy path to supply their own reader.
Delegated DOCX output depends on adeu’s `target_text` addressing, which is ambiguous on repeated text (their issues #28/#29). Mitigation: emit `match_mode: strict` with enough context, fall back to block-level replacement, and report ambiguities rather than guessing.
Scope creep toward OOXML. Mitigation: D12, D13 and D22 are explicit non-goals with rationale; revisit only with user demand in hand.
Pyodide constraints bite late. A dependency that works on CPython may be missing from the Pyodide index or too slow under WASM. The 1.0 reader set is confirmed to fit (D28), but rapidfuzz is browser-unavailable, so alignment quality on the site is the difflib-ratio floor, not the tuned `[fuzzy]` result; if the two diverge noticeably, the site will under-sell the engine. Mitigation: the CI import check lands in step 2, extras degrade gracefully by design, and the site budget (R40) is measured on a real contract pair before the site is announced.
The 1.0 slice looks thin to a visitor with a Word file. The site turning away a .docx is a real first impression cost. Mitigation: the sample pair is the default state so the capability is visible before any upload; the “coming in 1.1” message is specific; and the DOCX reader is first in the 1.1 queue after HTML.
Semantic heuristics drift toward wanting an LLM. Once roles exist, the temptation is to improve them with a model call. Mitigation: the non-goal is explicit; the semantic pass is pluggable, so anyone who wants an LLM-backed pass can write one outside the library; and the benchmark measures the heuristic pass so its limits are known rather than felt.
Two packages drift. `redlines-mcp` can lag or break against a new core release. Mitigation: compatible-range pinning, the MCP golden tests run against the core’s main branch in the core’s CI, and releases of the two are cut together.
The demo becomes the product. A site that works well invites feature requests (accounts, history, DOCX download) that pull toward a hosted service. Mitigation: the non-goals in section 3 are explicit; the site’s job is to sell the library and the MCP server, and anything that needs a server is out of scope by construction.
## 13. Open questions for you
[Section titled “13. Open questions for you”](#13-open-questions-for-you)
D10 and D14 are now decided per the recommendations. Next design questions, in order: the profile format (D30) — the built-in set is decided (`generic`, `contract`, `markdown` for 1.0; `legislation` deferred to 1.1, ROADMAP § 5.1) but the format itself is still to be designed, tracked as [#100](https://github.com/houfu/redlines/issues/100); the path syntax for D11; the JSON Schema for the change tree.
Whether to keep the name (D20; I recommend keeping it; the site’s domain is a separate, cheaper decision). Whether verify should also accept a natural-language instruction and ask an LLM to derive the allowed scope, or stay purely deterministic in 1.0 (I recommend deterministic; the LLM layer belongs in the caller, and on the site there is no LLM at all). Whether the MCP server should expose the 0.6 flat comparison as a separate tool for callers who just want the old markdown string (I recommend no; `compare` on two bare strings already returns a one-block-per-paragraph tree, and one fewer tool is easier for models). Whether to approach the adeu and Docxodus maintainers before or after 1.0 (I recommend before, once the JSON v2 schema is drafted, because the schema is the integration point).
**Resolved since revision 7.** Where the site lives: a `site/` directory in the main repository, one Astro project holding the documentation and the demo route together, per ADR-0026 and 7.12.
**Resolved since revision 9, by ROADMAP adoption (30 August 2026).** Whether the site’s side-by-side view is 1.0 or 1.1: 1.1 (ROADMAP § 3’s 1.1 table; already reflected in R43 above, which was never in conflict) — the block-change list with expandable redlines is the view that shows what is different about this engine, and side-by-side is what every competitor already has.
# Roadmap to redlines 1.0
> Which release each feature lands in.
**Status:** adopted, 30 August 2026. M0 shipped as 0.6.2 (PyPI, docs site live). Milestones M0–M6 exist on GitHub mirroring this file; all 12 M1 issues cite it. The ten calls in section 5 are recorded as accepted. **Relationship to the other documents:** the `PRD` reference in each row is a requirement number (R\*) or a decision number (D\*). Decisions now live as ADRs in [`docs/adr/`](/redlines/project/adr/); PRD section 6 maps every old D-number onto its ADR.
**Relationship to the PRD:** [`docs/PRD.md`](/redlines/project/prd/) says what each feature is and why it exists. This file says which release it is in. Where the two disagree, this file wins on release assignment; the PRD’s Must/Should tags were reconciled against the adopted plan in its revision 10 (30 August 2026).
## How to read this
[Section titled “How to read this”](#how-to-read-this)
Sizes are rough, for one experienced developer working with an agent: **S** is a day or two, **M** is up to a week, **L** is one to three weeks. They are for ranking, not estimating. Each feature cites its PRD requirement so you can look up the detail. Section 5 lists the places where I trimmed 1.0 below what the PRD currently says; those are the decisions this file exists to surface.
## 1. Releases at a glance
[Section titled “1. Releases at a glance”](#1-releases-at-a-glance)
| Release | What it proves | Contents in one line |
| ------------------------------- | ------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **0.6.x** (hygiene) | The flat engine is trustworthy while 1.0 is built, and there is somewhere to publish | autojunk off, cleanup pass, sentence mode keeps paragraphs, regression corpus, benchmark failure investigation, documentation moved from pdoc to Astro Starlight |
| **1.0** (the slice) | Structural, semantic, format-neutral comparison works end to end on text and markdown | block model + semantic layer + profiles; text and markdown readers; alignment with moves and renumbering; change tree and JSON schema; annotated, summary, markdown, rich and HTML renderers; verify; CLI; compatibility layer; benchmark |
| **redlines-mcp 0.1** (with 1.0) | Agents can compare, verify and author profiles | tools, prompts, resources, skill, golden tests; stdio transport |
| **Site 1.0** (after MCP) | A visitor sees the thesis in one click | A demo route added to the docs site: Pyodide, text and markdown inputs, sample pair by default, block-change list, annotated view, summary, JSON |
| **1.1** | More inputs and more detection, same core | HTML, DOCX and PDF readers; split/merge; applier export; profile auto-selection; `legislation` profile; MCP HTTP transport; side-by-side view |
| **Later** | Only with demand in hand | formatting changes, comments and footnotes, XML renderer, permalinks, batch API, Akoma Ntoso reader |
## 2. The 1.0 feature list, by milestone
[Section titled “2. The 1.0 feature list, by milestone”](#2-the-10-feature-list-by-milestone)
### M0 — 0.6.x hygiene release
[Section titled “M0 — 0.6.x hygiene release”](#m0--06x-hygiene-release)
| Feature | PRD | Size | Notes |
| ------------------------------------------------------------------------------------------ | ------------------------------- | ---- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `autojunk=False` on the SequenceMatcher, exposed as an option | D8 | S | One argument; measure speed on the 1,050-token repetitive case |
| Cleanup pass merging adjacent ops split only by punctuation or whitespace | D8, R15 | S | Fixes “thirty (30)” → two changes |
| Sentence mode preserves paragraph boundaries | D9, R16 | S | |
| Regression corpus including a repetitive schedule and the course examples as golden files | section 12 (compatibility risk) | S | These golden files are reused by M4 |
| Investigate the 18 `neurotic_docx_bench` failures | step 1 | S | Read-only investigation; fix only if trivial. **Resolved, #96:** none of the 18 are redlines failures — all die in the benchmark adapter’s python-docx extraction before `Redlines` is ever constructed. Nothing to fix in 0.6.x. Carried forward as #110 (a preview of the 1.1 DOCX reader’s failure modes) |
| Documentation site on Astro Starlight in `site/`, replacing pdoc as the publishing surface | ADR-0026, N6 | M | Scaffold, migrate what already exists (quickstart from the README, agent guide, ADR index, contributing), build pdoc into `/api/`, rewrite the Pages workflow. The agent guide moves whole and marked as the 0.6 guide, per ADR-0027; `llms.txt`, the per-page markdown export and a `/schemas/` location are stood up here so M4 writes into slots that exist. Nothing in `site/` may block a release |
**Exit:** 0.6.x on PyPI; existing test suite green; the repetitive-schedule case reports a two-token change; the docs site is live on GitHub Pages with the agent guide, the ADR index and the API reference under `/api/`.
**Shipped, 30 August 2026.** 0.6.2 is on PyPI; the docs site is live at [houfu.github.io/redlines](https://houfu.github.io/redlines/); the “M0 0.6.x hygiene” GitHub milestone is closed, 6 of 6 issues.
**Why this milestone.** The site work is independent of every engine milestone, so it neither blocks nor is blocked — and doing it first means the 1.0 pages that M4 owes (schemas, rewritten agent guide, benchmark report) are written once, onto a site that can hold them, instead of being written for pdoc and then migrated. It also settles the `site/` directory before M6 depends on it.
pdoc is kept here deliberately rather than replaced along with the publishing surface. The reference it produces is a foreign body on the site — its own theme, its own search, HTML only, so it is absent from `llms-full.txt` — but every one of those costs is lowest while the API is still being rebuilt, and the migration is scheduled into M4. Bring it forward if profile authoring in M1 needs API objects embedded in hand-written pages: that is a capability pdoc does not have at all, and it would be a reason to move early rather than on schedule.
### M1 — Block model, semantic layer, profiles, readers
[Section titled “M1 — Block model, semantic layer, profiles, readers”](#m1--block-model-semantic-layer-profiles-readers)
| Feature | PRD | Size | Notes |
| --------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------- | ---- | ------------------------------------------------------------------------------------- |
| Block model dataclasses: kind, text, label, level, path, children, attrs | R1 | S | Frozen dataclasses; no behaviour yet |
| Semantic fields: `role` on blocks, `spans` in blocks, open vocabulary with a recommended set | R1a, D5 | S | Vocabulary documented, not enforced |
| Profile format: flat, commented, schema-validated; loadable from file or mapping | R1d, R1e, R1f, D30 | M | The first design task; must satisfy R1f legibility |
| Built-in profiles: `generic`, `contract`, `markdown` | D30 | M | `legislation` moved to 1.1, see section 5 |
| Plain-text reader: normalise, segment, re-join wraps, detect labels, infer hierarchy, attach continuations, score headings | R4, D17, section 6b | L | The hard part is hierarchy inference; hard cases from 6b in the test set from day one |
| Markdown reader: ATX headings, lists with nesting, numbered clause patterns, pipe tables, fenced code, paragraphs; stdlib regex | R5, D16 | M | Reuses label detection and continuation logic from the text reader |
| Semantic pass: definitions, definition blocks with defined-term spans, schedules, cross-references carrying the referenced label, parties, dates, amounts | R1b, R1c | M | Rule-driven from the profile |
| Reader interface with a worked third-party example | R7 | S | |
| `dropped` reporting and per-block `matched_by` plus confidence; tree-level fallback count | R3, R1d | S | |
| Format detection for txt and md; unknown types reported, not guessed | R8b | S | |
| Section 3a sample pair and its expected block trees | section 3a | S | The change-tree golden comes in M2 |
| Pyodide import check in CI | D28, N5 | S | Do it here so nothing later depends on a dependency the browser cannot load |
**Exit:** the sample pair parses into the expected trees under `contract` and `markdown`; every 6b hard case has a test, passing or explicitly xfail; the wheel imports in Pyodide.
### M2 — Alignment, change tree, benchmark
[Section titled “M2 — Alignment, change tree, benchmark”](#m2--alignment-change-tree-benchmark)
| Feature | PRD | Size | Notes |
| --------------------------------------------------------------------------------------------------------------------------------------------- | --------------- | ---- | -------------------------------------------------------------------------------------------------------------------- |
| Multi-pass alignment: exact, label, fuzzy (difflib ratio; rapidfuzz if installed), positional; configurable thresholds; `matched_by` per pair | R9, D6 | L | |
| Move detection | R10, D7 | M | Release gate: move recall ≥ 0.9 on synthetic mutations, no reviewer-rejected false positive on the hand-labelled set |
| Renumbering detection | R11, D7 | S | Falls out of label-vs-content matching |
| Table alignment for markdown pipe tables: row insert/delete, cell-level inline diff; no column operations | R14 | M | Minimal, see section 5 |
| Determinism guarantee and test | R13, N1 | S | |
| Change tree: block ops insert, delete, modify, move, renumber; inline ops under modify; role and span types carried on change nodes | R18, R1c | M | |
| JSON serialisation with published schema and `schema_version` | R19, D10 | M | The integration point for MCP and the site; freeze early |
| Filters by kind, address prefix, label, role, minimum size | R23 | S | |
| Per-block and per-section statistics; change density by section | R22 | S | |
| R27a: inline changes recoverable as (address, old, new, context) | R27a | S | Design constraint, tested |
| Synthetic-mutation corpus generator: apply known moves, splits, renumberings, edits to real documents and keep the labels | D19 | M | Ground truth for free |
| Alignment metric: correspondence precision/recall, move recall, renumber recall; baselines flat 0.6 and, on DOCX pairs later, python-redlines | D19, section 10 | M | Report published with the release |
| Small hand-labelled set: ten real pairs | D19 | M | Labelling is the cost, not code |
**Exit:** the section 3a golden change tree passes; move gate met; benchmark report exists with numbers for 0.6 and 1.0 on the same corpus; JSON schema frozen.
### M3 — Renderers and compatibility
[Section titled “M3 — Renderers and compatibility”](#m3--renderers-and-compatibility)
| Feature | PRD | Size | Notes |
| -------------------------------------------------------------------------------------------- | --------- | ---- | ------------------------------------------------- |
| Markdown renderer over the tree, all six existing styles | R20 | M | Byte-identical to 0.6 on the golden files from M0 |
| Rich terminal renderer over the tree | R20 | S | |
| HTML renderer: block list with roles and addresses, expandable inline redlines | R20 | M | Minimal; the site builds on it, see section 5 |
| Annotated-document renderer: CriticMarkup for text and markdown, tag variant for HTML | R21a, D10 | M | Promoted to 1.0 Must, see section 5 |
| `Redlines` facade: existing class reimplemented as one-block-per-paragraph over the new core | R45, D3 | M | 0.6 suite passes unmodified |
| Deprecation warnings, no removals | R46 | S | |
**Exit:** 0.6 test suite green unmodified; README and course example strings byte-identical; annotated view of the sample pair reads correctly.
### M4 — Verify, CLI, docs: the 1.0 release
[Section titled “M4 — Verify, CLI, docs: the 1.0 release”](#m4--verify-cli-docs-the-10-release)
| Feature | PRD | Size | Notes |
| ---------------------------------------------------------------------------------------------------------------------------------- | ------------ | ---- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Verify: original, edited, allowed scope by address, label or role; pass/fail with out-of-scope changes and structural side effects | R24, D14 | M | Text-anchor scope moved to 1.1, see section 5 |
| Verify exemptions: whitespace-only, label-only | R25 | S | |
| Shared argument layer: path, stdin, inline; format hint; profile; size limit | R30a, D29 | S | Reused by the MCP package |
| CLI subcommands `compare`, `summary`, `annotate`, `verify`; `--profile`, `--format` | R28, R29 | S | About a day in total |
| Existing subcommands and command-less default unchanged | R30 | S | |
| Agent guide decomposed into one contract page plus task pages included from `examples/` | N6, ADR-0027 | M | Not a rewrite of one document: the contract goes on a single fetchable page, the tasks become pages whose code CI executes |
| JSON schema and profile schema published as pages, with a worked example each | N6, ADR-0026 | S | MDX, so the examples are real output rather than pasted |
| Benchmark report from M2 published on the docs site | ADR-0021, N6 | S | It is the external quality signal; it needs to be readable, not a file in the repository |
| Performance check: 2,000-block markdown pair under five seconds native | N2 | S | |
| API reference migrated off pdoc onto a `griffe`-based generator | ADR-0026, N6 | S | Deferred to here on purpose. The API roughly triples through M1–M3 and `Redlines` becomes a facade over a new core, so the reference’s job changes; and the Starlight-side tooling is weeks old, which only time can settle. Docstring conventions are fixed from M0 so this stays a configuration change. Trial on `claude/trial-starlight-pydocs` |
**Exit:** 1.0 on PyPI; agent guide live; benchmark report linked from the README.
### M5 — `redlines-mcp` 0.1
[Section titled “M5 — redlines-mcp 0.1”](#m5--redlines-mcp-01)
| Feature | PRD | Size | Notes |
| --------------------------------------------------------------------------------------------------------- | --------- | ---- | ------------------------------------------------------------------------- |
| Package skeleton depending on a compatible `redlines` range; console entry point; stdio transport | R31, D18 | S | HTTP transport moved to 1.1, see section 5 |
| Tools: `compare`, `summary`, `annotate`, `verify`, `read_blocks`, `preview_structure`, `validate_profile` | R32, D26 | M | Thin over the shared argument layer |
| Prompts: `draft_profile`, `refine_profile` | R32a | M | The profile-authoring loop; `explain_changes` moved to 1.1, see section 5 |
| Resources: profile schema, built-in profiles, change-tree schema, skill text | R32b | S | |
| Summary renderer, implemented in core, surfaced here | R21, D15 | M | |
| Skill text with the canonical loop and a worked transcript against the sample pair | R32c, R33 | M | |
| Size guards and truncation with continuation hints | R34 | S | |
| Golden tests over stdio for every tool and a replayed profile-authoring loop | R36, R32c | M | |
| Registry listings and one-line installs for Claude Code, Claude Desktop, Cursor | R35 | S | Marketing, not code; do it on release day |
**Exit:** a fresh Claude Code session can install the server, run the sample comparison, author a profile for a new document in the loop, and verify an edit, with no human intervention beyond the prompt.
### M6 — Site 1.0
[Section titled “M6 — Site 1.0”](#m6--site-10)
| Feature | PRD | Size | Notes |
| -------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- | ---- | ------------------------------------------------------------------------------------------- |
| Demo route in the docs site, Pyodide in a web worker, loads the published wheel | R37, D23, ADR-0026 | M | The site and its deployment already exist from M0; this adds a route and an island |
| Two inputs: drop or upload txt/md, or paste; sample pair loaded by default; friendly “coming in 1.1” for other types | R38 | M | |
| Views: block-change list with roles and addresses, expandable inline redlines; annotated document; summary; JSON with copy; `dropped` notice | R39 | M | Density view moved to 1.1, see section 5 |
| Profile selector with the built-ins and a paste box for a custom profile | R1e | S | |
| Progress state; 2,000-block pair under ten seconds after load | R40 | S | |
| Privacy statement; no upload endpoint; no content analytics | R41 | S | |
| Browser matrix and a clear failure message when Pyodide cannot load | R42 | S | |
| Demo ships from the same Astro project as the docs — no second build, no second deployment | section 13, ADR-0026 | S | PRD section 13’s “where the site lives” question, now decided |
| Documentation pages link to the demo, and the demo links back to the guides | ADR-0026 | S | The demo is the fastest explanation the project has; every page should be one click from it |
**Exit:** demo route live on GitHub Pages alongside the docs; sample pair renders on first load; a pasted 100-page markdown contract completes within budget on a mid-range laptop.
## 3. 1.1
[Section titled “3. 1.1”](#3-11)
| Feature | PRD | Size | Why not 1.0 |
| ------------------------------------------------------------------------------------------ | -------------- | ---- | --------------------------------------------------------------------- |
| HTML reader (stdlib parser) | R8 | M | Cheapest new reader; first in the queue |
| DOCX reader as `[docx]` extra on python-docx | R6, D12 | M | Your deferral; styles will improve the semantic pass, so early in 1.1 |
| PDF text reader as `[pdf]` extra on pypdf, structure flagged inferred, never OCR | R8a | M | Weakest path; after DOCX |
| Split and merge detection | R12, D7 | L | The main alignment feature after moves |
| Applier export: adeu edit batch, then superdoc-redlines edits file; round-trip convenience | R26, R27, D13 | M | Your deferral; R27a keeps the door open |
| Profile auto-selection with confidence | D30 | M | Trimmed from 1.0, see section 5 |
| `legislation` built-in profile | D30 | M | Trimmed from 1.0, see section 5 |
| Verify scope by text anchor | R24 | S | Trimmed from 1.0, see section 5 |
| MCP streamable HTTP transport | R31, D26 | S | Trimmed from 1.0, see section 5 |
| MCP `explain_changes` prompt | R32a | S | Trimmed from 1.0, see section 5 |
| Site side-by-side view with synchronised scrolling | R43 | M | Every competitor has it; the block list is what is different |
| Site per-section density view | R39 | S | Trimmed from 1.0, see section 5 |
| `[fuzzy]` threshold tuning from benchmark results | D6 review gate | M | Needs the benchmark to exist first |
| markdown-it based reader | D16 | M | Only if the regex reader proves insufficient |
| Expand hand-labelled benchmark set | D19 | M | Ongoing |
## 4. Later, only with demand
[Section titled “4. Later, only with demand”](#4-later-only-with-demand)
Inline formatting change detection (D22). Comments, footnotes, headers and footers as parts. XML renderer over the change tree (6a). Site permalinks encoding inputs in the URL fragment (R44). Batch comparison API. Akoma Ntoso and other structured-XML readers via the reader interface. Native OOXML revision writer (D13, and probably never). Three-way merge. Any OCR or in-library LLM call (never, per section 3 non-goals; a model-backed semantic pass belongs outside the library).
## 5. Calls that trim 1.0 below the PRD — accepted, 30 August 2026
[Section titled “5. Calls that trim 1.0 below the PRD — accepted, 30 August 2026”](#5-calls-that-trim-10-below-the-prd--accepted-30-august-2026)
All ten calls below are **accepted** as written. Nothing here has changed the milestone tables — M1 through M6 were already built on these calls, all 12 M1 issues cite this roadmap as authoritative, and issue [#101](https://github.com/houfu/redlines/issues/101) already treats call 5.1 as settled. Accepting them turns that existing fact into a recorded decision rather than a new one. [`docs/PRD.md`](/redlines/project/prd/) is annotated to match as of its revision 10.
1. **Accepted. `legislation` profile to 1.1.** The demo scenario is a contract and the primary persona’s inputs are contracts and LLM drafts. PLUS Explorer and legislation work stay a 1.1 case; revisit only if a statute needs to be in the demo before then.
2. **Accepted. Profile auto-selection to 1.1.** 1.0 defaults to `contract` for plain text and `markdown` for `.md`, with `--profile` to override. Auto-selection is scoring logic that can wait until there are more than three profiles.
3. **Accepted. Verify text-anchor scope to 1.1.** Addresses, labels and roles cover the agent use case; free-text anchors bring the ambiguity problem adeu is fighting.
4. **Accepted. MCP HTTP transport to 1.1.** Claude Code, Claude Desktop and Cursor all use stdio. HTTP matters for hosted agents, which are not the first audience.
5. **Accepted. MCP `explain_changes` prompt to 1.1.** Useful, but the profile loop is the distinctive thing and the summary tool already gives a model what it needs.
6. **Accepted. Annotated renderer promoted to 1.0 Must.** The PRD hedged; the MCP `summary` and `annotate` tools need it, so it is in M3.
7. **Accepted. HTML renderer kept minimal in 1.0.** Block list plus expandable redlines; the site adds interaction on top rather than a second renderer.
8. **Accepted. Table alignment kept minimal in 1.0.** Row insert/delete and cell inline diff for markdown pipe tables; no column operations, no merged cells. Enough for the sample pair’s inserted row.
9. **Accepted. Site density view to 1.1.** The stats exist in the JSON from M2; drawing them is UI work that does not prove anything new.
10. **Accepted. Hand-labelled benchmark set capped at ten pairs for 1.0.** Enough to catch a wrong move; expansion is ongoing work.
Any of these can still be overturned later; if one is, the size lands in the milestone named in the PRD reference and that milestone’s exit criteria should be re-read.
## 6. What is deliberately not sized here
[Section titled “6. What is deliberately not sized here”](#6-what-is-deliberately-not-sized-here)
The profile format design (M1) and the JSON schema (M2) are design tasks whose cost is thinking rather than typing; they are marked M but could be a week of argument each. The benchmark labelling (M2) is human time. Registry listings and the agent guide are writing. None of these should be squeezed to hit a date; they are the parts a fast implementation cannot make up for.