JUBARTE.DOCX
Menu

/ BENCHMARK · NEUROTIC_DOCX_BENCH 0.7 · GENERATED 2026-10-01

Scored against Word, failures included.

The oracle is Word’s own PDF export of each document (SHA-pinned). A converter’s PDF is scored 0–100 by pixel similarity; redlines are additionally opened in Word and compared against Word’s own compare. jubarte is author-affiliated (†) and plays by the same rules. Medians and means count a failed document as 0, so no tool gains by skipping a hard one.

Browse cases → Full viewer on GitHub Pages ↗ Harness & raw JSONL ↗ Live feed · coming soon →

Redline vs Word compare · median, failures = 0

84.34vs Docxodus 62.87

3,502 pairs; Docxodus produced a scorable redline for 2,612.

Perfect redlines (= 100)

241vs Docxodus 204

Redlines Word exported pixel-identically to its own compare.

Accept-all parity · median, failures = 0

99.70vs Docxodus 99.87

400 pairs; Word accepts every change, exports, scores. 0 missing files.

DOCX → PDF · median, failures = 0

80.90vs LibreOffice 63.43

6,427 documents, 0 failures. docxide-pdf 51.44.

Redlines vs Word’s compare

redlines_0929_full · 3,502 pairs · pixel scorer · opened in Word

Each tool redlines the pair; Word opens that redline and exports it to PDF, scored against Word’s own compare of the same pair. A compare with no scored PDF (no redline, or Word could not open it) counts 0.

#Tool · pinMedian · 95% CIMedian (fail = 0)Mean (fail = 0)ScoredFailed
1
jubarte-0.10.1 †
jubarte 0.10.1 (release)
= 100: 241 · ≥ 90: 636
84.34 74.82 3,496 6
2
jubarte-rust †
jubarte 0.10.0 (86b6b5d3)
= 100: 310 · ≥ 90: 658
81.73 72.23 3,499 3
3
jubarte-093 †
jubarte 0.9.3 (release)
= 100: 309 · ≥ 90: 634
79.07 68.62 3,338 164
4
docxodus
Docxodus 12.6.5 (C#)
= 100: 204 · ≥ 90: 482
62.87 53.94 2,612 890
5
superdoc
superdoc-sdk 2.16.0
= 100: 4 · ≥ 90: 7
0.00 6.66 380 3,122

Redlines accepted or rejected by Word

400 pairs · Word accepts or rejects every change · pixel scorer

Word accepts (or rejects) every change in each tool’s redline and exports it, scored against Word’s own compare resolved the same way. The calibration row is Word against itself. On accept-all, Docxodus leads by 0.17.

#Tool · pinMedian · 95% CIMedian (fail = 0)Mean (fail = 0)ScoredFailed
cal.
word-identity
Word’s own compare · accept all
= 100: 400
100.00 100.00 400 0
1
docxodus
docxodus 12.6.4 (npm) · accept all
= 100: 166 · ≥ 90: 299
99.87 90.07 394 6
2
jubarte-rust †
jubarte 0.9.3 · accept all
= 100: 181 · ≥ 90: 261
99.70 87.56 400 0
3
superdoc
superdoc-sdk 2.15.0 · accept all
= 100: 4 · ≥ 90: 52
0.00 13.95 58 342
cal.
word-identity
Word’s own compare · reject all
= 100: 400
100.00 100.00 400 0
1
jubarte-rust †
jubarte 0.9.3 · reject all
= 100: 186 · ≥ 90: 260
99.85 86.22 400 0
2
docxodus
docxodus 12.6.4 (npm) · reject all
= 100: 155 · ≥ 90: 275
99.80 85.38 385 15
3
superdoc
superdoc-sdk 2.15.0 · reject all
= 100: 1 · ≥ 90: 36
0.00 11.07 49 351

corpus/word:all — DOCX → PDF vs Word’s own PDF

docset a89ffc217bba · 6,427 docs · scorer v1 · 2026-10-01

Word’s PDF export of each DOCX in all four corpus states is the oracle; the candidate is the converter’s PDF of the same file. Sorted by ITT median; a failed document scores 0. 95% CI is a percentile bootstrap of the ITT median (2,000 resamples, seed 42).

#Tool · pinMedian · 95% CIMedian (fail = 0)Mean (fail = 0)ScoredFailed
1
jubarte †
jubarte 0.10.1
[80.58, 81.19]
80.90 77.03 6,427 0
2
soffice
LibreOffice 26.8.0.3
[62.84, 64.03]
63.43 61.95 6,359 68
3
docxide-pdf
docxide-pdf v0.17.1
[51.01, 51.84]
51.44 54.14 6,422 5

docxide_metrics — the same documents, docxide-pdf’s own yardstick

docset a89ffc217bba · 6,427 docs · Jaccard at 150 dpi · 2026-10-01

The same Word PDFs scored with docxide-pdf’s ink Jaccard (ranked) and text-boundary metrics, so no tool is judged only by our scorer. The text-boundary median is printed beside each interval.

#Tool · pinMedian · 95% CIMedian (fail = 0)Mean (fail = 0)ScoredFailed
1
jubarte †
jubarte 0.10.1
[65.12, 66.71] · text 97.62
65.91 61.42 6,385 42
2
soffice
LibreOffice 26.8.0.3
[22.06, 22.61] · text 80.00
22.29 26.01 6,294 133
3
docxide-pdf
docxide-pdf v0.17.1
[10.43, 11.04] · text 23.08
10.73 18.32 6,271 156

Below 3 pages: every converter on the same short documents

docset 27122f4d945e · 2,626 docs · scorer v1 · 2026-09-28

corpus/word:all restricted to documents Word lays out in 1 to 3 pages, so a converter that renders only the first 3 pages (unlicensed PyMuPDF Pro) is compared on equal terms. Older jubarte pin.

#Tool · pinMedian · 95% CIMedian (fail = 0)Mean (fail = 0)ScoredFailed
1
jubarte †
jubarte 0.9.3
[83.01, 83.28]
83.15 81.35 2,626 0
2
soffice
LibreOffice 26.8.0.3
[67.84, 68.83]
68.40 66.26 2,611 15
3
PyMuPDF Pro
pymupdf-pro 1.28.2 (first 3 pages)
[59.09, 60.77]
60.06 60.02 2,601 25
4
docxide-pdf
docxide-pdf v0.17.1
[53.81, 54.89]
54.24 56.00 2,626 0

Source DOCX → PDF, no redlines

docset c947065c59b2 · 398 docs · 2026-09-05 · older engine pin

The one headline table jubarte does not lead, kept here on purpose: on clean source documents at the 0.7.0 pin, the LibreOffice-based converters scored higher. The 0.10.1 run above, on a set sixteen times larger, is the current picture.

#Tool · pinMedian · 95% CIMedian (fail = 0)Mean (fail = 0)ScoredFailed
1
office2pdf
office2pdf 0.6.7
[79.83, 86.03]
84.34 73.82 390 8
1=
pdfitdown
pdfitdown 4.0.0
[79.83, 86.03]
84.34 73.82 390 8
3
libreoffice_convert_rust
LibreOffice Convert Rust v0.1.0
[72.14, 79.47]
75.26 73.98 398 0
4
jubarte †
jubarte 0.7.0
[66.94, 68.10]
67.39 66.25 398 0
4=
docxide-pdf
docxide-pdf v0.17.0
[63.27, 64.73]
63.97 65.65 398 0
6
rdocx
rdocx 0.7.0
[53.38, 55.31]
54.49 59.11 398 0
7
dxpdf
dxpdf 0.5.1
[52.15, 52.94]
52.52 56.73 381 17

By corpus state, corpus/word:all

median, failures = 0 · documents per state in brackets

clean (2,480)

jubarte †87.54
soffice68.72
docxide-pdf59.48

tracking, no comments (3,410)

jubarte †79.70
soffice62.99
docxide-pdf49.10

comments + tracking (420)

jubarte †72.07
soffice45.26
docxide-pdf46.57

comments, clean (117)

jubarte †70.31
soffice47.56
docxide-pdf50.94

Per-case viewer

Open this case full-size →

Every published case is browsable: Word’s page next to each engine’s page, with overlay, difference and multiply modes, page-count mismatches, the score table and the files themselves on Hugging Face. Below is convert/tracking-without-comments-2ad8d15e88 — press r inside the viewer for a random case.

jubarte.pro/cases#convert/tracking-without-comments-2ad8d15e88

Oracle: real Word

Every reference PDF is exported by Microsoft Word and SHA-pinned. Redlines are opened in Word too: if Word refuses the file, the tool scores 0 for that pair.

Intent-to-treat

Failures are not dropped. A crash, a timeout or an unopenable file counts as 0 in ITT mean and median, so a tool cannot improve its rank by skipping hard documents.

Paired bootstrap

Ranks tie (n=) when the paired bootstrap interval of the median difference to the row above includes 0. 2,000 resamples, seed 42.

Author-affiliated, same rules

jubarte is marked † everywhere. Pins, run dates and docset hashes are printed so any row can be re-run; the harness is public.

Figures are project-maintained and reproducible, not a neutral third-party benchmark. Competitors move: Docxodus in particular has evolved since several stamps above, so read the pins. Tables are regenerated by uv run bench report from results/bench.jsonl; this page copies RESULTS.md of 2026-10-01.