"""plank_deck_qa.py - render a finished deck, look at it, and say what is wrong. The deck builder's text budgets are a *pre-render* check: plank_deck.py measures a string against the box it has to fit in and refuses to draw a slide that cannot hold it. That catches a lot, and it catches it before anything is built. It cannot catch what only exists once a renderer has laid the deck out: a table whose rows grew, two shapes that landed on each other, a chart whose category labels overlap, a picture that did not decode, a slide that came out empty. This does the other half. It converts the deck to PDF, rasterises the pages, reads back where every word actually landed, and compares that against the geometry the deck declares. The PDF it produces along the way is a deliverable in its own right - a deck usually has to be handed over as both. python3 plank_deck_qa.py quarterly-review.pptx python3 plank_deck_qa.py brand-pitch.html BOTH DECK PATHS, ONE TOOL ------------------------- Plank builds decks two ways - `plank_deck.py` writes a .pptx, `plank_slides.py` writes HTML slides that print to PDF - and this checks both. Not by having two tools that agree: by having ONE, because the two formats differ in a couple of places at the edges and in none of the rules. * the READER. `read_pptx` pulls authored geometry out of OOXML; for HTML that is `read_slides_manifest`, which loads the `.slides.json` the builder writes beside the document. Both return the same list of SlideRender. * the RENDERER. `to_pdf` shells out to headless LibreOffice; `html_to_pdf` shells out to headless Chromium. Both produce a PDF, which is then rasterised and word-boxed by exactly the same code. Everything after that - every detector, every threshold, every finding, the report, the critique pass and the exit status - is shared, and shared by being the same lines of code rather than by two files being kept in step. That was the whole design already: "Every detector is a pure function of the geometry that has already been read." A second QA tool would have had to import this one to avoid drifting from it, and then a workspace has to curl two files and get an import path right - which is a failure mode, and a silent one. Findings print per slide, each naming the slide, what is wrong, and what to change. Exit status is 1 if anything is an error, so it drops straight into a build-check-repair loop. WHAT IT CAN AND CANNOT SEE -------------------------- It detects things with an unambiguous mechanical definition: * text that renders outside the shape it belongs to * text that renders outside the slide's safe margins * words that land on top of other words, including chart labels * shapes that partially overlap each other * slides that came out blank or near-blank * text left in from a template - "Lorem ipsum", "TBD", "[fill this in]" * pictures whose bytes do not decode, or that rendered as an empty frame * fonts the deck asks for that are not installed, so were substituted * text over a photograph that is not legible against the pixels actually behind it - see TEXT OVER IMAGERY below * on an HTML deck, text that left the slide entirely and was clipped out of the render before anything could see it - see BOTH DECK PATHS below * Russian typography that is mechanically wrong - see RUSSIAN TYPOGRAPHY * an inline SVG diagram that breaks the theme, accessibility, id or self-containment contract - see DIAGRAMS It does NOT detect whether the deck is any good. It has no opinion on hierarchy, colour, pacing, whether the chart is the right chart, or whether a slide earns its place. Those need a reader. A clean report means nothing is broken; it does not mean the deck is finished. RUSSIAN TYPOGRAPHY, AND THE OTHER KIND OF DELIVERABLE ----------------------------------------------------- Most people reading a Plank deliverable read Russian, and a document full of "straight quotes" where «ёлочки» belong, with hyphens standing in for dashes and ё left out, reads as foreign no matter how good its structure is. That is not a matter of taste and it does not need a reader: it is decidable from the bytes, so it is checked here rather than left as advice somebody follows when they remember to. Because a Russian document is a deliverable too, this tool takes one: python3 plank_deck_qa.py квартальный-отчёт.html python3 plank_deck_qa.py дашборд.html python3 plank_deck_qa.py заметки.md and proofreads its visible text with the same rules, the same report and the same exit status as a deck. Nothing is rendered in that mode - a document has no geometry to measure - so LibreOffice and poppler are not needed for it. WHICH JOB AN .html FILE IS -------------------------- So `.html` now names two different jobs: an HTML slide deck that gets rendered and measured, and a document or dashboard that gets proofread. `route` decides between them, and it decides on evidence rather than on a flag: .slides.json beside it -> slide deck. Render it. no manifest, but the markup carries plank_slides.py's own `class="slide"` and `data-qa=` blocks -> slide deck whose manifest is missing. That is an error with a rebuild instruction, not a document - see read_slides_manifest. neither -> document. Proofread it. The manifest is the right discriminator because it is what the deck path actually needs: `read_slides_manifest` cannot run without it, so "is there a manifest" and "can this be checked as a deck" are the same question. The markup sniff exists only so that a deck whose manifest was lost keeps failing loudly the way it used to, instead of being quietly downgraded to a proofread that passes. A `--as-document` / `--as-deck` flag was the alternative and is worse: the caller is usually a script that just built the file and should not have to remember which kind it built. Nothing is lost either way. The deck path runs the Russian rules too - they live in `inspect_slide`, which both readers feed - so an HTML deck is proofread as well as measured. The document path skips only the geometry checks, and a document has no geometry. What is checked, and what is deliberately not, is written out above `russian_typography_issues`. The short version: every rule here is one whose violation is certain, because flagging correct Russian is worse than missing wrong Russian - a checker that cries wolf gets switched off, and then it catches nothing at all. The other half of writing quality - AI-flavoured vocabulary, verbal nouns that should be verbs, empty evaluative adjectives, claims with nothing behind them - cannot be decided mechanically and is not attempted here. It lives in the `deliverable-writing` skill, which points back at this tool for this half. DIAGRAMS -------- An HTML document may carry inline SVG diagrams, and those fail in four ways that are decidable from the bytes and invisible to everything else here: fill="#0a0a0f" instead of fill="var(--ink)" -> right in one theme, invisible in the other with no accessible name and no -> announced as a picture, aria-hidden="true" contents unavailable two diagrams, two id="arrow" markers -> the second borrows the first one's definitions -> not one file any more The theme one is the reason this exists. Plank's house style is a single file whose light/dark follows its host, and it does that with CSS custom properties - so a diagram's colours have to come from tokens too. Almost every general-purpose diagram library bakes literal hexes into the SVG and ships a separate file for dark mode, which this house style has nowhere to put; paste one in and the diagram is the same colour as the paper on half of everybody's screens. Nothing at render time notices, because the diagram IS there. These run on any `.html` document, English included, so the report no longer stops at "no Russian text found". The contract they check is written out for a human at https://plank.md/help/diagrams, together with the part no checker can do: whether the diagram was worth drawing at all. What is NOT checked is the drawing. Whether the connectors are traceable, whether the layout reads, whether nine nodes should have been two diagrams - those need eyes, and the guidance for them is on that page. TEXT OVER IMAGERY ----------------- One check is neither pure geometry nor a matter of taste, and it is the reason the art-directed house style can put type on photographs at all. A title over a picture is legible or it is not, and nothing in the .pptx knows: the file records that the text is #FAFAF7 and that a picture is behind it, and says nothing about what colour the picture happens to be at that spot. The raster does know. This tool already rasterises every page, so for each word that renders over a picture it reads the pixels the renderer actually put behind that word, throws away the ones that are the glyph itself, and computes the WCAG contrast ratio between the text colour and what is left. Below 3:1 is an error - display type at that size needs 3:1 and nothing survives less - and between 3:1 and 4.5:1 is a warning, because deck type is large but a lead line or a caption is not. That is a real measurement of the real render, and it is the only check here that could not have been written against the .pptx alone. THE CRITIQUE PASS ----------------- So the report does not stop at the mechanical half. Everything above is build-time and render-time checking; a deck can pass all of it and still be a bad deck. After the findings, this tool prints a CRITIQUE section, and the build order it belongs to is: build -> mechanical QA -> CRITIQUE -> revise -> deliver Two different things live in that section, and they are labelled so nobody confuses them: * The TITLE LADDER, and a handful of mechanical prompts. The ladder is not a check - it is the titles-alone test made unavoidable, because this tool already knows every title in order and printing them costs nothing. The prompts (a title that names a topic, an opening slide that promises the answer instead of stating it, two slides making one point, numbers with no source in sight) are cheap heuristics with real false-positive rates. They are `info`, they are never errors, and they are meant to be overruled out loud when they are wrong. * The JUDGEMENT questions. Nothing mechanical can answer these. They are printed every run, including a clean one, because a clean run is exactly when a deck is most likely to be shipped unread. WHY THE OUTPUT IS BLUNT ABOUT REPORTING --------------------------------------- A workspace run of an earlier version printed "12 slides, 3 clean, 9 with findings" and the assistant told the user "проверка: ошибок рендеринга нет" - no rendering errors. That was true (all nine were warnings, none an error) and it was worthless: nine defects went unmentioned and unrepaired, and the user was told the deck had been checked. The severity split is right and is kept - a substituted font is not a broken slide, and escalating every warning to an error would only teach people to ignore errors. What was wrong was a summary line that made "9 with findings" easy to compress into "no errors". So the header now leads with what is unresolved rather than with what is clean, and the report ends with an explicit statement of what has to be said to the user before the deck is handed over. A gate that can be summarised away is not a gate. One thing about the renderer that the numbers depend on: LibreOffice is not PowerPoint - line breaking differs slightly, so treat a finding that is within a point or two of tolerance as a near miss rather than a fact. The HTML path has no equivalent caveat and this is the one place it is strictly better: Chromium IS the renderer the deck was designed against, so a width measured here is the width the reader gets. It used to have a caveat of its own going the other way, and that caveat is now a third format-specific step rather than a sentence. An HTML slide is a fixed-size box with `overflow:hidden`, because without that, slide 3's overrun paints onto slide 4's page - so text that runs off the SLIDE EDGE is clipped and never reaches the PDF, where a .pptx would have let it render into the void and be reported. Text that overflowed a little was caught; text that left the slide completely was invisible, which is the worst way for a checker to fail, because "no findings" is what a reader takes away. * so, for HTML only, a MEASUREMENT. `measure_html_layout` asks Chromium where every declared block's words really are - before the clip, which is the one moment anything can know - and the words that fall entirely outside the slide become ordinary `Word` objects appended to the ones poppler read. No new finding kind, no second reporting path: the same detectors report them as the overflow they are. Everything short of that - a word outside its own box, a word past the safe margin but still on the page - is seen exactly as it is in a .pptx, and the builder's pre-render budgets (which are the same budgets, imported, not copied) still stand in front of all of it. The other thing they depend on is that the deck's fonts are actually installed. A substituted face has different advance widths, so text measured against a substitute is text measured against the wrong font, and every width in this report would then be a claim about a deck nobody built. That is asked of fontconfig and reported as a `font-substituted` finding rather than left to a reader - see detect_missing_fonts. REQUIREMENTS ------------ pdftoppm and pdftotext (poppler-utils), and Pillow for the raster checks. For a .pptx: soffice (LibreOffice) and python-pptx. For an .html deck: headless Chromium, found on PATH as `chrome-headless-shell` (or named by $PLANK_CHROME). All are preinstalled in every Plank workspace, and only the ones your deck's format needs are ever touched. Pillow is imported lazily: without it everything except the blank slide and failed-image checks still runs. fontconfig (`fc-list`) is used for the font check; without it that one check reports itself as unknown. """ from __future__ import annotations import argparse import glob import html import json import os import re import shutil import subprocess import sys import tempfile import uuid import xml.etree.ElementTree as ET from collections import Counter from dataclasses import dataclass, field, asdict from html.parser import HTMLParser # --- units ---------------------------------------------------------------- # # Everything in here is in PostScript points, 72 to the inch, origin top-left. # That is the coordinate space poppler reports and the space a PDF page is # measured in, so it is the space the comparison happens in. python-pptx speaks # EMU (914400 to the inch); the only conversion is at the boundary. EMU_PER_PT = 12700.0 def pt(emu): """EMU to points.""" return emu / EMU_PER_PT def inches(points): return points / 72.0 # --- the house geometry --------------------------------------------------- # # Defaults matching plank_deck.py. A deck built by something else gets the same # checks against whatever page size it declares; only the margins are assumed, # and --margins overrides them. DEFAULT_MARGINS_IN = (0.75, 0.6, 0.75, 0.7) # left, top, right, bottom # Rectangles are compared in floating point after two unit conversions, so # quantities below this are noise, not geometry. Without it a box whose left # edge is exactly the left margin reports itself "0.0pt past the left". EPSILON_PT = 0.01 # How far a rendered word may sit outside a box before it counts. A text frame # is laid out against the font's ascender and descender, not the ink, so a line # of type always renders a little taller than its nominal size and a box sized # exactly to its text reports a hairline overhang that no reader can see. # # Measured, not guessed. Against a clean 11-slide Cyrillic deck exercising # every primitive the builder had at the time of the calibration - eleven, the # two image primitives came later - plus a five-slide deck exercising all five # chart goals (including a 12-category axis and a doughnut), the worst # legitimate overhang of a rendered word past its own shape is 0.27pt - the # descender of a source note. The worst past a safe margin is the same 0.27pt, # the same word. The count is the calibration deck's, not a current inventory: # it is here so the measurement can be reproduced, not to be kept up to date. # # That number used to be 1.19pt, and the shape it belonged to was a slide # title: plank_deck.py ended the title box exactly where the type ended, so # every titled slide in every deck overhung its own box, and this tolerance # was the reason nobody noticed. The constant had been calibrated against a # deck carrying the fault, which quietly made it the fault's alibi. The # builder now reserves a bottom inset under the title and the overhang is # gone, so the calibration no longer certifies a bug as legitimate. # # 2pt stays anyway, and not out of inertia. The measurement above is one # renderer on one machine; the figure that exposed the title bug was 4.5pt, # reported by a workspace whose LibreOffice disagreed with this one by more # than the 0.8pt of headroom the old number had left. Tightening to just above # 0.27pt would make this check a detector of renderer versions. It is # deliberately no larger either: 3pt is a quarter of a body line, which a # reader can see. # # Charts are held to the same figure on the same evidence. An earlier draft # gave them 6pt on the assumption that LibreOffice's chart engine needed room; # measurement said otherwise - legitimate chart labels escaped their frame by # exactly 0.00pt across all five goals - and the extra slack only hid real # overflow, such as axis labels spilling out of a frame too short to hold them. SHAPE_SLACK_PT = 2.0 # Two words count as colliding when they overlap by this fraction of the # smaller one's area. It has to be well clear of zero: consecutive lines of # tightly-led display type overlap by a few tenths of a point, because a line # box runs from ascender to descender and 1.0 line spacing packs those boxes # edge to edge. Measured on a real render, adjacent 44pt cover-title lines # overlap by 0.41pt out of a 53pt line height - under 1%. Across the two clean # calibration decks the worst legitimate word-pair overlap was 0.0000. 0.30 # leaves two orders of magnitude of headroom and still fires on any real # overlap: the collided doughnut it was tuned against scores 1.00. COLLISION_AREA_RATIO = 0.30 # A slide with less ink than this is blank in every way that matters. A section # divider - a two-digit number and one word on a full-bleed surface - is the # sparsest legitimate slide the house style produces; measured at 1.02% of # non-background pixels, which is 20x this. BLANK_INK_RATIO = 0.0005 # How far a rendered pixel may be from the modal background colour before it # counts as ink. Anti-aliased edges of real glyphs land well above this; # JPEG-ish gradients in a flat-filled surface land below. INK_DISTANCE = 24 # The outermost sliver of the page is not content. pdftoppm leaves a one-pixel # seam along the page edge where the background fill stops, and on an ivory # deck that seam is near-white and invisible to this measurement. On an # obsidian one it is a bright line: 825 pixels of it on a 1467x825 render, or # 0.07% of the page - which is more than BLANK_INK_RATIO all by itself, and it # made the raster half of the blank-slide check unable to see an invisible # slide in the dark theme. Trimming half a percent from each edge removes the # seam and cannot remove content, because the house style's own safe margins # start eight times further in. BORDER_INSET = 0.005 # Two shapes overlapping by less than this are touching, not colliding. Grid # columns computed in EMU land on neighbouring integers. OVERLAP_SLACK_PT = 1.0 # How much of the smaller shape has to sit inside a PICTURE before the pair # reads as "this is on that" rather than "these two collided". # # Only against a picture, and that restriction is the whole correctness of it. # Things sit on surfaces: a caption on a photograph, a logo on a full-bleed # image, a title over a still. Nothing legitimately sits 90% on top of another # TEXT block - that is two blocks colliding, and it is what this check exists # to find. A threshold applied to every pair would excuse the defect along # with the design. MOSTLY_INSIDE = 0.8 # Below this, two boxes are touching, not colliding. A logo lockup whose # bounding box grazes the metadata row beside it by 1.4pt is not a defect, and # on the deck this house style came from that single pattern produced fourteen # identical warnings. The real check on whether two things landed on each # other is `text-collision`, which measures RENDERED WORDS rather than the # boxes they were declared in; this one is the cheap pre-filter, so its # tolerance should be loose enough that the expensive one is what speaks. GRAZE_PT = 3.0 # --- text over imagery ---------------------------------------------------- # # WCAG contrast ratios. 3:1 is the floor for large text - everything a deck # puts over a picture is large - and 4.5:1 is the floor for ordinary body # text, which a lead line or a caption over an image effectively is. So below # 3:1 is an error and the band between is a warning. CONTRAST_ERROR = 3.0 CONTRAST_WARN = 4.5 # A word's own glyphs are inside its box, so they have to come out of the # measurement or every word would read as 1:1 against itself. They are # identified by contrast, NOT by colour distance, and that distinction is the # whole correctness of this check. # # The first draft classified a pixel as ink if it was within a fixed colour # distance of the text colour. That inverts on the one input the check exists # for: white type on a near-white photograph has a BACKGROUND within that # distance too, so every pixel was thrown away as ink, there was nothing left # to measure, and the function returned "cannot tell" - silently passing the # most illegible slide it will ever see. A check that fails open on its own # worst case is worse than no check. # # A contrast threshold cannot invert like that: a pixel at 1.10:1 against the # text is indistinguishable from the text whether it is ink or backdrop, and # either way the reader cannot see the letter. So pixels below this are set # aside as ink, and if that leaves nothing at all, the answer is not "unknown" # - it is that the word is sitting on its own colour. INK_CONTRAST = 1.15 # Which backdrop pixel to judge by. Not the worst one - antialiasing always # leaves a ring of blended pixels between the ink and the ground, and the # single worst of those would fail every word ever set. Not the mean either, # which lets a bright patch under half a word average away against a dark # patch under the other half. The 25th percentile is the hardest quarter of # what is genuinely behind the word, which is the part a reader struggles with. BACKDROP_PERCENTILE = 0.25 # Below this many backdrop pixels there is nothing behind the word that is not # the word: see INK_CONTRAST. Reported as illegible, not as unknown. BACKDROP_MIN_PIXELS = 24 # --- geometry primitives -------------------------------------------------- @dataclass(frozen=True) class Rect: """An axis-aligned rectangle in points, top-left origin.""" x0: float y0: float x1: float y1: float @property def width(self): return self.x1 - self.x0 @property def height(self): return self.y1 - self.y0 @property def area(self): return max(0.0, self.width) * max(0.0, self.height) @property def cx(self): return (self.x0 + self.x1) / 2.0 @property def cy(self): return (self.y0 + self.y1) / 2.0 def grow(self, slack): return Rect(self.x0 - slack, self.y0 - slack, self.x1 + slack, self.y1 + slack) def intersection(self, other): return Rect( max(self.x0, other.x0), max(self.y0, other.y0), min(self.x1, other.x1), min(self.y1, other.y1), ) def intersects(self, other): inter = self.intersection(other) return inter.width > 0 and inter.height > 0 def contains(self, other, slack=0.0): grown = self.grow(slack) return (grown.x0 <= other.x0 and grown.y0 <= other.y0 and grown.x1 >= other.x1 and grown.y1 >= other.y1) def contains_point(self, x, y, slack=0.0): grown = self.grow(slack) return grown.x0 <= x <= grown.x1 and grown.y0 <= y <= grown.y1 def escape(self, container, slack=0.0): """How far this rect sticks out of `container`, per side, in points. Returns a dict of only the sides that are exceeded by more than `slack`. An empty dict means it fits. """ floor = max(slack, EPSILON_PT) out = {} if container.x0 - self.x0 > floor: out["left"] = container.x0 - self.x0 if container.y0 - self.y0 > floor: out["top"] = container.y0 - self.y0 if self.x1 - container.x1 > floor: out["right"] = self.x1 - container.x1 if self.y1 - container.y1 > floor: out["bottom"] = self.y1 - container.y1 return out def distance_to(self, other): """Shortest distance between two rects; 0 if they touch or overlap.""" dx = max(other.x0 - self.x1, self.x0 - other.x1, 0.0) dy = max(other.y0 - self.y1, self.y0 - other.y1, 0.0) return (dx * dx + dy * dy) ** 0.5 def _sides(escape): """'12.4pt past the bottom and 3.1pt past the right' from an escape dict.""" parts = ["%.1fpt past the %s" % (v, k) for k, v in sorted(escape.items(), key=lambda kv: -kv[1])] if len(parts) == 1: return parts[0] return ", ".join(parts[:-1]) + " and " + parts[-1] # --- what we read out of the two files ------------------------------------ @dataclass class Shape: """One shape from the .pptx, in points.""" index: int name: str kind: str # text | table | chart | picture | graphic | group rect: Rect text: str = "" headline: str = "" # first paragraph only - see _first_paragraph fonts: tuple = () image_ok: bool = True image_note: str = "" # The colour of this shape's first run, "RRGGBB", or "" when the file does # not state one (a theme colour, or an inherited default). Only the # contrast check reads it, and it skips a shape that has none rather than # assuming black - assuming would produce a confident finding about a # colour nobody wrote down. color: str = "" @property def can_own_text(self): return self.kind in ("text", "table", "chart", "graphic") @property def slack(self): return SHAPE_SLACK_PT def label(self): if self.text: body = self.text if len(self.text) <= 40 else self.text[:37] + "..." return '%s "%s"' % (self.kind, body) return "%s %r" % (self.kind, self.name) @dataclass class Word: """One rendered word, as poppler reports it.""" text: str rect: Rect block: int line: int @dataclass class SlideRender: """Everything known about one slide.""" number: int page: Rect # the slide box, 0,0 to width,height shapes: list words: list title: str = "" headline: str = "" # the title alone, for the critique ladder ink_ratio: float | None = None png: str | None = None image_regions: dict = field(default_factory=dict) # shape index -> (contrast ratio, the word it was measured on). Filled in # by the raster pass for text shapes that render over a picture, so the # detector that reads it stays a pure function like every other one. backdrops: dict = field(default_factory=dict) @dataclass class Finding: slide: int kind: str severity: str # error | warning | info what: str fix: str where: str = "" def to_dict(self): return asdict(self) SEVERITY_ORDER = {"error": 0, "warning": 1, "info": 2} # --- reading the .pptx ---------------------------------------------------- def _runs(text_frame): for para in text_frame.paragraphs: for run in para.runs: yield run def _shape_text_and_fonts(shape): text, fonts = [], set() if shape.has_text_frame: for run in _runs(shape.text_frame): if run.text.strip(): text.append(run.text) if run.font.name: fonts.add(run.font.name) if getattr(shape, "has_table", False): for row in shape.table.rows: for cell in row.cells: for run in _runs(cell.text_frame): if run.text.strip(): text.append(run.text) if run.font.name: fonts.add(run.font.name) return " ".join(text).strip(), tuple(sorted(fonts)) def _first_run_color(shape): """The RGB of the shape's first run that states one, as "RRGGBB". Empty string when nothing does. python-pptx raises rather than returns None when a colour is a theme reference or is simply absent, so every read is guarded; a shape whose colour the file does not state is not one this tool is willing to make a contrast claim about. """ if not shape.has_text_frame: return "" for run in _runs(shape.text_frame): if not run.text.strip(): continue try: rgb = run.font.color.rgb except (AttributeError, TypeError, ValueError): continue if rgb is not None: return str(rgb).upper() return "" def _first_paragraph(shape): """The shape's first non-empty paragraph, on its own. The title ladder needs the headline and nothing else. A cover, a quote and a closing each put two things in one text frame - title and subtitle, quotation and attribution, headline and call to action - so the joined text of the frame reads as one run-on line in the ladder and buries the line that is actually the deck's argument. Titled slides are unaffected: _slide_title() gives the title a text frame of its own. """ if not shape.has_text_frame: return "" for para in shape.text_frame.paragraphs: text = "".join(run.text for run in para.runs).strip() if text: return text return "" def _picture_health(shape): """Does this picture's payload actually decode? A picture whose bytes are truncated, whose part is missing, or that points at an external file the deck does not carry renders as an empty frame - and an empty frame is indistinguishable from a design choice until you know the bytes are bad. python-pptx raises on a missing part, so the check is a guarded read plus a decode. """ try: image = shape.image except (AttributeError, KeyError, ValueError) as exc: return False, "the image part is not embedded in the file (%s)" % exc try: blob = image.blob except Exception as exc: # noqa: BLE001 - report, don't die return False, "the image bytes could not be read (%s)" % exc if not blob: return False, "the image is zero bytes" try: from PIL import Image # noqa: PLC0415 - optional except ImportError: return True, "" import io try: with Image.open(io.BytesIO(blob)) as probe: probe.verify() except Exception: # noqa: BLE001 - report, don't die # Pillow's own message quotes the repr of the BytesIO it was handed, # which tells the reader nothing and changes every run. Report the size # and the format the bytes claim to be instead. # # Sniffed here rather than read from python-pptx's `image.ext`, which # asks Pillow to identify the format and therefore raises this same # exception from inside the handler for it - killing the whole run on # the one input this function exists to diagnose. return False, ("the %d bytes embedded for it are not a readable %s" % (len(blob), _sniff_format(blob))) return True, "" _MAGIC = [ (b"\x89PNG\r\n\x1a\n", "PNG"), (b"\xff\xd8\xff", "JPEG"), (b"GIF87a", "GIF"), (b"GIF89a", "GIF"), (b"BM", "BMP"), (b"II*\x00", "TIFF"), (b"MM\x00*", "TIFF"), (b".slides.json` next to `.html`: the # authored geometry of every block, in points, which is the same thing # read_pptx digs out of OOXML. It is not a convenience - it is the half of # every check that says what the deck MEANT, and without it the tool can only # report that two words overlap, never that a word left the box it belongs to. # # Read from a sidecar rather than from the document, and that is the choice # worth defending. The geometry could have been recovered by driving Chromium # over a debugging protocol and asking the DOM for every element's box - which # is more code, a websocket dependency, a second browser launch, and a report # that describes the DOM rather than what the builder intended. A .pptx is # trusted to state its own shape rects; an HTML deck built by this house style # states them the same way. HTML_SUFFIXES = (".html", ".htm") MANIFEST_SUFFIX = ".slides.json" def manifest_path(html_path): stem = os.path.splitext(html_path)[0] return stem + MANIFEST_SUFFIX def read_slides_manifest(html_path): """Every slide's blocks, in points. Returns (slides, page_rect).""" path = manifest_path(html_path) if not os.path.exists(path): raise RenderError( "no %s beside %s. plank_slides.py writes that manifest every time " "it saves, and it is where the authored geometry lives - without " "it nothing here can tell you a word left its box. Re-run the " "build script rather than checking the .html on its own." % (os.path.basename(path), os.path.basename(html_path)) ) with open(path, encoding="utf-8") as handle: data = json.load(handle) page_spec = data.get("page") or {} page = Rect(0.0, 0.0, float(page_spec.get("width", 960.0)), float(page_spec.get("height", 540.0))) slides = [] for entry in data.get("slides", []): shapes = [] for index, raw in enumerate(entry.get("shapes", [])): rect = raw.get("rect") or [0, 0, 0, 0] shapes.append(Shape( index=int(raw.get("index", index)), name=str(raw.get("name", "")), kind=str(raw.get("kind", "graphic")), rect=Rect(float(rect[0]), float(rect[1]), float(rect[2]), float(rect[3])), text=str(raw.get("text", "")), headline=str(raw.get("headline", "")), fonts=tuple(raw.get("fonts") or ()), image_ok=bool(raw.get("image_ok", True)), image_note=str(raw.get("image_note", "")), # The builder writes "RRGGBB" for the block's first paragraph, # same convention as _first_run_color, and writes nothing when # the block states no colour - so the contrast check skips it # rather than assuming one, exactly as on the .pptx path. color=str(raw.get("color", "")).lstrip("#").upper(), )) title, headline = "", "" for shape in shapes: if shape.kind == "text": title, headline = shape.text, shape.headline break slides.append(SlideRender(number=int(entry.get("number", len(slides) + 1)), page=page, shapes=shapes, words=[], title=title, headline=headline)) if not slides: raise RenderError("%s declares no slides" % os.path.basename(path)) return slides, page # --- rendering ------------------------------------------------------------ class RenderError(RuntimeError): """The deck could not be rendered. Nothing downstream can run.""" def _require(tool, package): if shutil.which(tool) is None: raise RenderError( "%s is not on PATH. It ships in every Plank workspace as part of " "%s; if you are somewhere else, install that package." % (tool, package) ) def renderer_version(): """Which LibreOffice is doing the laying out, for the record. Worth carrying in the report and in the test fixtures: every measurement here is a claim about what one renderer did, and a different build can break a line differently. """ if shutil.which("soffice") is None: return "soffice not installed" try: proc = subprocess.run(["soffice", "--version"], capture_output=True, text=True, timeout=60, check=False) except (subprocess.TimeoutExpired, OSError): return "soffice version unavailable" return (proc.stdout or "").strip().splitlines()[0] if proc.stdout else "unknown" # --- fonts ---------------------------------------------------------------- # # Every finding in here is a comparison between where a word landed and where # the .pptx says its box is. If the renderer could not find the font the deck # asked for, it substituted another face with different advance widths - and # the comparison is then between a real render and the wrong expectation. # # This used to be printed as prose and left to whoever read the line. It is a # mechanical question with a mechanical answer - fontconfig knows - so it is a # finding. The whole design is an exit-code-driven build-check-repair loop; a # caveat only a human can act on is a caveat nobody acts on. # # The other font question is how MANY. Two is the house rule and the house # style ships exactly two faces, so hierarchy has to come from size, weight # and placement rather than from a third typeface. That is a design rule, and # it is the rare one a program can settle: the deck states every face it asks # for. A brand override supplies a heading font and a mono font, which is # still two, so a third face is not a brand decision - it is a slide that # reached past the house style. MAX_FONT_FAMILIES = 2 def parse_fc_list_families(stdout): """Family names out of `fc-list : family`, casefolded. One line per font file, comma-separated aliases on it (`Inter,Inter SemiBold`), and fontconfig backslash-escapes punctuation in a family name (`Unifont\\-JP`). Split out from the subprocess call so the parsing is testable on a box with no fontconfig - the same reason parse_bbox_xml is split out from the pdftotext call. """ families = set() for line in stdout.splitlines(): for name in line.split(","): name = re.sub(r"\\(.)", r"\1", name).strip() if name: families.add(name.casefold()) return families def installed_font_families(timeout=20): """What fontconfig can see, or None if it cannot be asked. None and the empty set mean different things and the caller has to be able to tell them apart: no fontconfig means "unknown", and reporting every font in the deck as missing because fc-list is absent would be a lie. """ if shutil.which("fc-list") is None: return None try: proc = subprocess.run(["fc-list", ":", "family"], capture_output=True, text=True, timeout=timeout, check=False) except (subprocess.TimeoutExpired, OSError): return None if proc.returncode != 0 or not proc.stdout: return None return parse_fc_list_families(proc.stdout) def missing_fonts(fonts, installed): """The requested fonts fontconfig cannot find. None if it could not be asked.""" if installed is None: return None return [f for f in fonts if f.casefold() not in installed] def detect_missing_fonts(slides, installed): """Fonts the deck asks for that are not installed. A warning rather than an error: the deck still renders, and the substitute may even be metrically identical - Fira Code and DejaVu Sans Mono are both 600/1000-em like JetBrains Mono, so mono text survives that swap. What it is not is *measured*. Every other number in this report rests on the deck being laid out in the font it asked for, and when it was not, that has to be visible to the loop rather than inferred by a reader. Attributed to the first slide that asks for the font, so the finding points at something instead of floating above the deck. Reported once per font, not once per run - a 15-slide deck asks for the same two faces 66 times. """ if not installed: return [] findings, seen = [], set() for slide in slides: for shape in slide.shapes: for font in shape.fonts: key = font.casefold() if key in installed or key in seen: continue seen.add(key) findings.append(Finding( slide=slide.number, kind="font-substituted", severity="warning", what=("The deck asks for %r, which is not installed here. " "The renderer drew a substitute face, so every width " "and line count in this report - including whether " "anything overflowed - was measured against the " "wrong font." % font), fix=("Install %r and re-run this check before trusting it, " "or change the deck to a font that is installed." % font), where=shape.label(), )) return findings # A function-based shading: /ShadingType 1, whose colour comes from a # /FunctionType 4 PostScript program evaluated once per pixel. Matched in the # raw bytes rather than through a PDF library because Chromium writes PDF 1.4 # with no object streams, so every object dictionary is plain text in the file # - and because the alternative is a parser dependency this tool does not have. _PDF_SLOW_SHADING = re.compile(rb"/ShadingType\s+1\b") def detect_slow_pdf_shadings(pdf_path): """Will this PDF take seconds a page to open? The source check in `authored_deck_issues` reads the CSS and is the one that can name the fix. This reads the rendered PDF, so it also catches the ways a slow shading arrives without a `repeating-` or `conic-` anywhere in the deck's own stylesheet: inside an embedded SVG, inside a chart library's output, or out of a .pptx that LibreOffice converted. Counts objects PRESENT, not objects REACHED. For the file this runs on - one Chromium has just printed - those are the same thing, and the cheap version is the one that cannot be wrong about a deck that was never post-processed. A PDF somebody has already flattened can keep the orphaned shading dictionaries and read as a false positive here; that is the safe direction for a check whose whole job is to refuse a file that opens slowly. """ try: with open(pdf_path, "rb") as handle: blob = handle.read() except OSError: return [] count = len(_PDF_SLOW_SHADING.findall(blob)) if not count: return [] return [Finding( slide=0, kind="pdf-opens-slowly", severity="error", what=("The PDF carries %d function-based shading%s (/ShadingType 1). " "Every one is a PostScript program the reader runs once per " "pixel, and Preview on macOS and iOS rasterises the whole " "pattern cell before it clips - so the cost is set by the size " "of the element the gradient sits on, not by how much of it is " "visible. This is what makes a deck sit blank for most of a " "minute before the slide appears." % (count, "" if count == 1 else "s")), fix=("These come from repeating-linear-gradient, " "repeating-radial-gradient and conic-gradient - the two CSS " "gradients that are not PDF gradients. Replace a grid or " "scanline overlay with a repeating SVG background-image, and a " "sweep with linear-gradient at an angle. plain linear-gradient " "and radial-gradient are fine and are not counted here."), )] def detect_too_many_fonts(slides, limit=MAX_FONT_FAMILIES): """More typefaces than the house style allows. A warning, not an error: a third face renders perfectly well. What it does is make the deck look assembled rather than designed, which is a defect a reader sees and a geometry check cannot - so it is exactly the kind of thing that used to be left to "judgement" while sitting in plain sight in the file. ONE finding for the whole deck, and deliberately no culprit named. The obvious version - blame every face past the first two seen - was written first and was wrong on the first deck it met: a stray Comic Sans on slide 1 is "seen" before the mono face that first appears in a table on slide 7, so the finding accused JetBrains Mono and told the reader to restyle their own table. Frequency does no better, since the interloper can be the most used face on a deck that is mostly one bad slide. Nothing here knows which two faces the deck MEANT, so it says how many there are and lets the reader pick - which is the decision anyway. Anchored to the first slide on which the count goes over, so the finding points somewhere to start looking rather than floating above the deck. Names are compared casefolded and are not normalised beyond that. Nothing here can know whether "Sohne" and "Sohne Mono" are one family in two weights or two families, and guessing at font taxonomy would trade a check nobody can argue with for one nobody can trust. """ seen, over_at, over_shape = {}, 0, None for slide in slides: for shape in slide.shapes: for font in shape.fonts: if font.casefold() in seen: continue seen[font.casefold()] = font if len(seen) == limit + 1: over_at, over_shape = slide.number, shape if len(seen) <= limit: return [] return [Finding( slide=over_at, kind="too-many-fonts", severity="warning", what=("The deck sets %d typefaces: %s. The house style is %d." % (len(seen), ", ".join(seen.values()), limit)), fix=("Decide which %d the deck is in and set everything else in " "those. plank_deck.py's own pair is Inter and JetBrains Mono, " "and a brand `fonts:` override replaces both rather than adding " "a third. Hierarchy comes from size, weight and placement; " "another typeface adds noise, not emphasis." % limit), where=over_shape.label() if over_shape else "", )] def to_pdf(pptx_path, out_dir, timeout=240): """Convert with headless LibreOffice. Returns the PDF path. soffice is checked on its artifacts, not its exit status: a half-installed filter set makes it exit 0 having written nothing, which is the failure mode the sandbox image's own smoke test was built around. Every call gets a private UserInstallation profile. Two soffice processes sharing the default one deadlock on its lock file, and agents run things in parallel. """ _require("soffice", "libreoffice-impress") os.makedirs(out_dir, exist_ok=True) profile = os.path.join(tempfile.gettempdir(), "plank-deck-qa-%s" % uuid.uuid4().hex) try: proc = subprocess.run( ["soffice", "-env:UserInstallation=file://%s" % profile, "--headless", "--norestore", "--convert-to", "pdf", "--outdir", out_dir, pptx_path], capture_output=True, text=True, timeout=timeout, check=False, ) except subprocess.TimeoutExpired: raise RenderError( "soffice did not finish within %ds. A deck this slow to convert is " "usually carrying a very large embedded image." % timeout ) finally: shutil.rmtree(profile, ignore_errors=True) stem = os.path.splitext(os.path.basename(pptx_path))[0] pdf = os.path.join(out_dir, stem + ".pdf") if not os.path.exists(pdf) or os.path.getsize(pdf) == 0: raise RenderError( "soffice exited %d but produced no PDF.\n%s" % (proc.returncode, (proc.stderr or proc.stdout or "").strip()) ) return pdf # Headless Chromium, in the order a Plank workspace is most likely to have it. # # chrome-headless-shell first and deliberately: it is the build the sandbox # image installs, it is a third of the size of full Chrome, and it has no GTK, # no X11 and no dbus - so it neither needs a desktop stack nor prints one # screenful of dbus errors into the middle of a QA report, which full Chrome # does on every run in a container. CHROME_NAMES = ( "chrome-headless-shell", "chromium", "chromium-browser", "google-chrome-stable", "google-chrome", "chrome", ) CHROME_ENV = "PLANK_CHROME" # Where the sandbox image unpacks it. Checked after PATH, so a workspace that # installed its own browser wins over the baked one. CHROME_FALLBACK_PATHS = ( "/opt/chrome-headless-shell/chrome-headless-shell", ) def find_chrome(): """The headless browser to print with, or None.""" named = os.environ.get(CHROME_ENV) if named: # An explicit override is not second-guessed: if it is wrong, the # caller needs to hear that it is wrong, not watch the tool quietly # use a different browser than the one they named. return named for name in CHROME_NAMES: found = shutil.which(name) if found: return found for path in CHROME_FALLBACK_PATHS: if os.path.exists(path): return path return None def chrome_version(binary=None): binary = binary or find_chrome() if binary is None: return "chromium not installed" try: proc = subprocess.run([binary, "--version"], capture_output=True, text=True, timeout=60, check=False) except (subprocess.TimeoutExpired, OSError): return "chromium version unavailable" return (proc.stdout or "").strip() or "unknown" def html_to_pdf(html_path, out_dir, timeout=240): """Print an HTML deck with headless Chromium. Returns the PDF path. One page per slide at the exact slide size, and both halves of that are the document's doing rather than a flag's: `@page { size: 1280px 720px; margin: 0 }` fixes the page box, and a section that is exactly one page tall with `break-after: page` fixes the pagination. There is no CLI flag for page size, so a deck that omits the @page rule prints on US Letter and every measurement in this report would be against the wrong box - which is why the page size is checked against the manifest downstream rather than assumed. Checked on its artifacts, not its exit status, for the same reason `to_pdf` is: Chromium exits 0 after failing to load a file, and the only evidence is a PDF that is not there. """ binary = find_chrome() if binary is None: raise RenderError( "no headless Chromium on PATH (looked for %s). It ships in every " "Plank workspace as chrome-headless-shell; if you are somewhere " "else, install it or point %s at the binary." % (", ".join(CHROME_NAMES), CHROME_ENV) ) os.makedirs(out_dir, exist_ok=True) stem = os.path.splitext(os.path.basename(html_path))[0] pdf = os.path.abspath(os.path.join(out_dir, stem + ".pdf")) profile = os.path.join(tempfile.gettempdir(), "plank-deck-qa-chrome-%s" % uuid.uuid4().hex) argv = [ binary, "--headless", # A private profile per call. Two Chromiums sharing one lock the # profile and the second exits having drawn nothing - the same trap # soffice has, and agents run things in parallel. "--user-data-dir=%s" % profile, # Chromium's own sandbox needs user namespaces, which a container # usually does not grant, and without this the browser exits before it # loads anything. What it is being asked to open is a local file the # agent just wrote, inside a container that is already the isolation # boundary - so the sandbox is protecting the deck from itself. Do not # copy this flag to a browser that visits the open internet. "--no-sandbox", "--disable-gpu", "--disable-dev-shm-usage", "--hide-scrollbars", "--no-pdf-header-footer", "--print-to-pdf=%s" % pdf, os.path.abspath(html_path), ] try: proc = subprocess.run(argv, capture_output=True, text=True, timeout=timeout, check=False) except subprocess.TimeoutExpired: raise RenderError( "chromium did not finish within %ds. A deck this slow to print is " "usually carrying very large embedded images." % timeout ) finally: shutil.rmtree(profile, ignore_errors=True) if not os.path.exists(pdf) or os.path.getsize(pdf) == 0: raise RenderError( "chromium exited %d but produced no PDF.\n%s" % (proc.returncode, (proc.stderr or proc.stdout or "").strip()) ) return pdf # --- the HTML path's blind spot, measured out of it ------------------------ # # An HTML slide is `overflow:hidden`, so anything that leaves it is CLIPPED. # Text that overflows a little is still on the page and reads back out of the # PDF exactly as a .pptx would; text that leaves the slide entirely is painted # nowhere, reaches no PDF, and produces no word for any detector to find. The # report said so on every run - and a caveat is not a check. "No findings" is # what a reader takes away, and it was the one sentence this tool could not # honestly say about a deck whose third slide had run off the edge. # # The browser knows. Before it paints, Chromium has laid every box out and can # report where each word really is, clipped or not - so the measurement is # taken there, in a second headless pass over a copy of the deck with a script # appended, and the answer comes back through --dump-dom. What comes back is # not a new kind of finding: the words that fall entirely outside the slide are # turned into the same `Word` objects poppler produces, appended to the same # list, and every detector downstream sees them as it sees any other word. # detect_outside_margins has been able to say "It is off the slide edge # entirely" since the day it was written; it had simply never been given a word # that was. # # Only fully-off-slide words are synthesised. A word that straddles the edge is # painted, is in the PDF, and is already reported - adding it here would report # it twice, in two coordinate systems, and the second copy would be the one a # reader could not reconcile against the PNG. # # This is a READER-and-RENDERER-layer step, like read_slides_manifest and # html_to_pdf, and deliberately not a detector. The detectors stay a pure # function of the geometry that has already been read, they stay shared with # the .pptx path, and there is no second reporting path to keep in step. MEASURE_ID = "plank-qa-measure" # Per element: [text, left, top, width, height] in CSS px relative to the # slide's own top-left. One entry per client rect, so a word broken across a # soft line break contributes both halves as themselves. MEASURE_JS = r""" (function () { var out = []; var slides = document.querySelectorAll(".slide"); for (var i = 0; i < slides.length; i++) { var box = slides[i].getBoundingClientRect(); var blocks = []; var tagged = slides[i].querySelectorAll("[data-qa]"); for (var b = 0; b < tagged.length; b++) { var el = tagged[b]; var words = []; var walker = document.createTreeWalker(el, NodeFilter.SHOW_TEXT, null, false); var node; while ((node = walker.nextNode())) { var re = /\S+/g; var m; while ((m = re.exec(node.nodeValue)) !== null) { var range = document.createRange(); range.setStart(node, m.index); range.setEnd(node, m.index + m[0].length); var rects = range.getClientRects(); for (var k = 0; k < rects.length; k++) { var r = rects[k]; if (r.width <= 0 || r.height <= 0) { continue; } words.push([m[0], round(r.left - box.left), round(r.top - box.top), round(r.width), round(r.height)]); } } } if (words.length) { blocks.push({i: parseInt(el.getAttribute("data-qa"), 10), w: words}); } } out.push({n: i + 1, w: round(box.width), h: round(box.height), blocks: blocks}); } function round(v) { return Math.round(v * 100) / 100; } var tag = document.createElement("script"); tag.type = "application/json"; tag.id = "%s"; // A JSON payload inside a ' % MEASURE_ID, dom, re.DOTALL) if not match: return None try: return json.loads(match.group(1)) except ValueError: return None def measure_html_layout(html_path, timeout=240): """Where every declared word really is, measured in the browser. Returns the payload MEASURE_JS builds: one entry per slide, each with the slide's own pixel size and the words of every block that carries a `data-qa` index. Raises RenderError if the browser could not be asked. """ binary = find_chrome() if binary is None: raise RenderError( "no headless Chromium on PATH (looked for %s). It ships in every " "Plank workspace as chrome-headless-shell; if you are somewhere " "else, install it or point %s at the binary." % (", ".join(CHROME_NAMES), CHROME_ENV) ) with open(html_path, encoding="utf-8") as handle: source = handle.read() # The copy sits beside the original so a deck that references anything by # relative path still resolves it. The builder inlines its images, but a # hand-written deck need not. directory = os.path.dirname(os.path.abspath(html_path)) or "." copy = os.path.join(directory, ".plank-qa-measure-%s.html" % uuid.uuid4().hex) profile = os.path.join(tempfile.gettempdir(), "plank-deck-qa-chrome-%s" % uuid.uuid4().hex) try: with open(copy, "w", encoding="utf-8") as handle: handle.write(inject_measure_script(source)) argv = [ binary, "--headless", "--user-data-dir=%s" % profile, # Same three flags, for the same reasons, as html_to_pdf. "--no-sandbox", "--disable-gpu", "--disable-dev-shm-usage", "--dump-dom", os.path.abspath(copy), ] try: proc = subprocess.run(argv, capture_output=True, text=True, timeout=timeout, check=False) except subprocess.TimeoutExpired: raise RenderError( "chromium did not finish laying the deck out within %ds." % timeout ) finally: shutil.rmtree(profile, ignore_errors=True) if os.path.exists(copy): os.remove(copy) measured = parse_measured_dom(proc.stdout or "") if measured is None: raise RenderError( "chromium exited %d without returning a layout measurement. " "Nothing downstream can tell whether this deck has text off the " "slide edge, and a clipped render looks identical to a clean " "one.\n%s" % (proc.returncode, (proc.stderr or proc.stdout or "").strip()[:2000]) ) return measured # One runaway block can produce hundreds of clipped words, and every one of # them is the same finding. Enough of them to be sure, then stop. MAX_CLIPPED_WORDS = 60 # Far outside anything poppler numbers, so a synthesised word can never share a # (block, line) pair with a rendered one and silently suppress a collision. CLIPPED_BLOCK_BASE = 100000 def clipped_words(measured, page): """The measured words the slide clipped away, as Word objects per slide. Only words with no overlap at all with the slide box: those are the ones that reached no PDF, so nothing downstream has seen them. Everything else is already in the render and already reported. """ out = {} for entry in measured or []: width, height = float(entry.get("w") or 0), float(entry.get("h") or 0) if width <= 0 or height <= 0: continue # px to points, off the slide's own measured size rather than a # constant: the manifest states the page box, and a deck whose CSS # disagrees with it is a deck whose measurements must still line up. sx, sy = page.width / width, page.height / height number = int(entry.get("n") or 0) for block in entry.get("blocks") or []: index = int(block.get("i", -1)) for line, word in enumerate(block.get("w") or []): text, x, y, w, h = word[0], float(word[1]), float(word[2]), \ float(word[3]), float(word[4]) rect = Rect(x * sx, y * sy, (x + w) * sx, (y + h) * sy) if rect.intersects(page): continue found = out.setdefault(number, []) if len(found) >= MAX_CLIPPED_WORDS: break found.append(Word(text=text, rect=rect, block=CLIPPED_BLOCK_BASE + index, line=line)) return out def to_pngs(pdf_path, out_dir, dpi=110): """Rasterise every page. Returns the PNG paths in page order.""" _require("pdftoppm", "poppler-utils") os.makedirs(out_dir, exist_ok=True) prefix = os.path.join(out_dir, "slide") # Clear previous output first. pdftoppm zero-pads the page number to the # width of the page count, so a 7-page deck writes slide-1.png and an # 11-page deck writes slide-01.png - different names for the same page. # Rasterising a short deck into a directory that still holds a long one's # pages leaves both behind, and the glob below then returns a mixture of # two decks in numeric order. That is not hypothetical: it silently # attached one deck's pages to another deck's slides while this tool's own # test fixtures were being recorded, and every blank-slide measurement # taken from them was wrong. for stale in glob.glob(prefix + "-*.png"): os.remove(stale) subprocess.run(["pdftoppm", "-png", "-r", str(dpi), pdf_path, prefix], capture_output=True, text=True, check=True) # Sort on the parsed number, never on the string: "slide-10" sorts before # "slide-2" lexically. found = [] for path in glob.glob(prefix + "-*.png"): match = re.search(r"-(\d+)\.png$", path) if match: found.append((int(match.group(1)), path)) return [path for _, path in sorted(found)] def read_pdf_words(pdf_path): """Word boxes per page, from `pdftotext -bbox-layout`. poppler reports exactly what the renderer laid down, in PDF points with a top-left origin - the same space and orientation as a .pptx, so the two compare directly. This is the whole reason the tool can talk about where text *actually* went rather than where it was asked to go. """ _require("pdftotext", "poppler-utils") with tempfile.TemporaryDirectory() as tmp: xml_path = os.path.join(tmp, "bbox.xml") subprocess.run(["pdftotext", "-bbox-layout", pdf_path, xml_path], capture_output=True, text=True, check=True) with open(xml_path, "rb") as handle: raw = handle.read() return parse_bbox_xml(raw) def parse_bbox_xml(raw): """Parse poppler's bbox-layout XHTML into per-page (page_rect, words). Split out from the subprocess call so the parsing can be tested against captured renderer output without a renderer present. """ text = raw.decode("utf-8", "replace") # poppler emits an XHTML doctype with an external DTD reference. Drop it: # ElementTree will not fetch it and errors on the undefined entities it # would have defined. text = re.sub(r"]*>", "", text, count=1) root = ET.fromstring(text) ns = "{http://www.w3.org/1999/xhtml}" pages = [] for page_el in root.iter(ns + "page"): page = Rect(0.0, 0.0, float(page_el.get("width", 0)), float(page_el.get("height", 0))) words, block_no = [], 0 for block_el in page_el.iter(ns + "block"): line_no = 0 for line_el in block_el.iter(ns + "line"): for word_el in line_el.iter(ns + "word"): body = (word_el.text or "").strip() if not body: continue words.append(Word( text=body, rect=Rect(float(word_el.get("xMin")), float(word_el.get("yMin")), float(word_el.get("xMax")), float(word_el.get("yMax"))), block=block_no, line=line_no, )) line_no += 1 block_no += 1 pages.append((page, words)) return pages def rescale_words(words, pdf_page, slide_page): """Put poppler's coordinates in the .pptx's space. LibreOffice rounds the page box (a 13.333in slide comes back as 959.98pt, not 960), so a comparison in raw PDF points drifts by a fifth of a point at the right margin. Small, but it is free to remove and the whole tool is about sub-point claims. """ if pdf_page.width <= 0 or pdf_page.height <= 0: return words sx = slide_page.width / pdf_page.width sy = slide_page.height / pdf_page.height if abs(sx - 1.0) < 1e-9 and abs(sy - 1.0) < 1e-9: return words return [Word(text=w.text, rect=Rect(w.rect.x0 * sx, w.rect.y0 * sy, w.rect.x1 * sx, w.rect.y1 * sy), block=w.block, line=w.line) for w in words] # How tall poppler says a word is, and why the two renderers disagree. # # This is a units problem wearing a layout problem's clothes, and it cost an # afternoon, so the measurement is written down. # # A word's height, as poppler reports it, is not ink. It is the font's # declared vertical extent - and poppler can only read that when the PDF # embeds a font it can interrogate. LibreOffice embeds Inter (an OTF, so CFF # outlines) as a Type 1 font and poppler reads Inter's real hhea extent: # 1.211em, measured identically at 9pt, 30pt and 40pt on this deck, and # matching what fontTools reports for the installed file. # # Chromium's PDF backend embeds any CFF outline as a **Type 3** font - glyphs # as drawing procedures - and a Type 3 font declares no vertical metrics at # all. Poppler then substitutes a fixed fallback: 1.706 x the font size, at # every size, for every Type 3 face. Measured on the same deck and on a # purpose-built calibration page. # # So the same Inter word, same size, same ink, arrives 41% taller on the HTML # path than on the .pptx path. Every threshold in this file - SHAPE_SLACK_PT, # COLLISION_AREA_RATIO, the one-line error grade - was calibrated in the first # convention. Two sets of thresholds would be two sets of thresholds to keep # true, so the boxes are converted at the boundary instead, exactly as # rescale_words converts the page box, and every rule downstream stays one # rule. Without it, adjacent lines of 44pt display type overlap by 29.7% of a # word box and sit a fifth of a percentage point under the collision # threshold, which is not a margin anybody should rely on. # # It is NOT fixed by installing a TrueType Inter, and that was checked before # this was written: it would change which file LibreOffice lays the .pptx path # out with, and every number in that path's calibration is a claim about the # file it has today. Correcting the reader is reversible; swapping the house # font under a calibrated renderer is not. # # The one place this is approximate: a face that really is TrueType survives # Chromium as a CID TrueType and poppler reads its true metrics, so correcting # it is over-correction. That is JetBrains Mono, extent 1.32em, in a table # body - over-corrected by 0.11em, which is 1.3pt at the 12pt the house style # sets a table in, against a one-line error grade of 15.8pt. TYPE3_FALLBACK_EM = 1.706 HOUSE_FONT_EXTENT_EM = 1.211 HTML_WORD_BOX_SCALE = HOUSE_FONT_EXTENT_EM / TYPE3_FALLBACK_EM def rescale_word_heights(words, scale): """Shrink every word box vertically about its own centre.""" if abs(scale - 1.0) < 1e-9: return words out = [] for word in words: half = word.rect.height * scale / 2.0 out.append(Word(text=word.text, rect=Rect(word.rect.x0, word.rect.cy - half, word.rect.x1, word.rect.cy + half), block=word.block, line=word.line)) return out # --- raster inspection ---------------------------------------------------- def ink_ratio(png_path, box=None, sample=600): """Fraction of pixels that differ from the image's own modal colour. The modal colour is the background whatever the theme is, so this works on an ivory deck, a dark one, or somebody else's brand without being told. A slide reduced to a single colour returns 0.0. """ from PIL import Image # noqa: PLC0415 - optional with Image.open(png_path) as img: img = img.convert("RGB") if box is not None: img = img.crop(box) else: # Whole-page measurement: trim the renderer's edge seam. Not done # for an explicit box, which is a shape somewhere inside the page # and carries no seam. See BORDER_INSET. dx = max(1, int(img.width * BORDER_INSET)) dy = max(1, int(img.height * BORDER_INSET)) img = img.crop((dx, dy, img.width - dx, img.height - dy)) if img.width == 0 or img.height == 0: return 0.0 if max(img.size) > sample: scale = sample / float(max(img.size)) img = img.resize((max(1, int(img.width * scale)), max(1, int(img.height * scale)))) # tobytes(), not getdata(): getdata() is deprecated in Pillow 12 and # emits a DeprecationWarning into the middle of the report. raw = img.tobytes() if not raw: return 0.0 pixels = [raw[i:i + 3] for i in range(0, len(raw) - 2, 3)] counts = {} for pixel in pixels: counts[pixel] = counts.get(pixel, 0) + 1 bg = max(counts, key=counts.get) off = sum( 1 for pixel in pixels if abs(pixel[0] - bg[0]) + abs(pixel[1] - bg[1]) + abs(pixel[2] - bg[2]) > INK_DISTANCE ) return off / float(len(pixels)) def _srgb_to_linear(channel): c = channel / 255.0 return c / 12.92 if c <= 0.04045 else ((c + 0.055) / 1.055) ** 2.4 def relative_luminance(rgb): """WCAG 2.x relative luminance of an (r, g, b) 0-255 triple.""" r, g, b = (_srgb_to_linear(v) for v in rgb) return 0.2126 * r + 0.7152 * g + 0.0722 * b def contrast_ratio(a, b): """WCAG contrast ratio between two (r, g, b) triples. 1.0 to 21.0.""" la, lb = relative_luminance(a), relative_luminance(b) hi, lo = (la, lb) if la >= lb else (lb, la) return (hi + 0.05) / (lo + 0.05) def parse_hex(value): """"RRGGBB" (or "#RRGGBB") to an (r, g, b) triple, or None.""" if not value: return None text = str(value).strip().lstrip("#") if len(text) != 6: return None try: return tuple(int(text[i:i + 2], 16) for i in (0, 2, 4)) except ValueError: return None def backdrop_contrast(png_path, page, rect, text_rgb): """Contrast between `text_rgb` and what actually rendered behind `rect`. The crop covers the glyphs as well as the ground they sit on, so the pixels that are effectively the ink (contrast under INK_CONTRAST) are set aside first. The figure returned is the BACKDROP_PERCENTILE-worst of what is left - see the constants for why by contrast and not by colour, and for why not the single worst pixel. When almost nothing is left, the word is on ground the same colour as itself: that is the answer, not a reason to give up, so the median of everything is returned and it will be close to 1.0. Returns None only when there is no raster to read at all. """ from PIL import Image # noqa: PLC0415 - optional box = _crop_box(rect, page, png_path) if box[2] <= box[0] or box[3] <= box[1]: return None with Image.open(png_path) as img: crop = img.convert("RGB").crop(box) raw = crop.tobytes() ratios = [] for i in range(0, len(raw) - 2, 3): ratios.append(contrast_ratio(text_rgb, (raw[i], raw[i + 1], raw[i + 2]))) if not ratios: return None ratios.sort() backdrop = [r for r in ratios if r >= INK_CONTRAST] if len(backdrop) < BACKDROP_MIN_PIXELS: return ratios[len(ratios) // 2] index = min(len(backdrop) - 1, max(0, int(len(backdrop) * BACKDROP_PERCENTILE))) return backdrop[index] def measure_backdrops(slide, page, png_path): """Worst backdrop contrast per text shape that renders over a picture. Restricted to text over a PICTURE on purpose. Everywhere else the surface behind the type is a flat fill this deck chose, and the blank-slide detector already catches ink the same colour as its own background; a contrast check there would only re-litigate the theme. Over a photograph nothing knows the colour behind the words until the page is rasterised, which is exactly the gap. """ pictures = [s for s in slide.shapes if s.kind == "picture" and s.image_ok] if not pictures: return {} out = {} owners = assign_owners(slide.shapes, slide.words) for word, shape in owners: if shape is None or shape.kind != "text": continue if not any(p.rect.intersects(shape.rect) for p in pictures): continue if not any(p.rect.contains_point(word.rect.cx, word.rect.cy) for p in pictures): continue text_rgb = parse_hex(shape.color) if text_rgb is None: continue ratio = backdrop_contrast(png_path, page, word.rect, text_rgb) if ratio is None: continue current = out.get(shape.index) if current is None or ratio < current[0]: out[shape.index] = (ratio, word.text) return out def _crop_box(rect, page, png_path): """A shape's rect as pixel coordinates in its page's PNG.""" from PIL import Image # noqa: PLC0415 - optional with Image.open(png_path) as img: w, h = img.size sx = w / page.width sy = h / page.height return (max(0, int(rect.x0 * sx)), max(0, int(rect.y0 * sy)), min(w, int(rect.x1 * sx)), min(h, int(rect.y1 * sy))) # --- detectors ------------------------------------------------------------ # # Every detector is a pure function of the geometry that has already been read. # Nothing below shells out, opens a file, or needs a renderer to be installed - # which is what makes the rules testable against captured render output rather # than only against a live LibreOffice. # Beyond this there is no honest claim that a stray word came out of a given # box, and naming the wrong box sends the repair to the wrong slide element. STRAY_DISTANCE_PT = 72.0 _PUNCT = " \t.,;:!?()[]{}\"'«»„“”‘’—–-…" def _normalize(text): return text.strip(_PUNCT).casefold() def _claims(shape, key): """Did this shape author the word that rendered? The .pptx carries the exact authored string for every shape, so provenance does not have to be inferred from position alone - and position alone gets it wrong precisely when it matters. A table whose rows grew renders its last row *inside the box of the source note underneath it*; by containment that word belongs to the source note, and the report then quotes a table cell as if the note had overflowed. Matching the word against the authored text sends it back to the table it actually came from. Words of one character are not allowed to claim anything: a stray Cyrillic "и" or "в" appears in almost every shape on the slide, so a match carries no information. """ if not shape.text or len(key) < 2: return False return key in _normalize(shape.text) def assign_owners(shapes, words): """Which shape does each rendered word belong to? Returns a list of (word, shape or None) pairs, resolved in this order: 1. a shape that both contains the word and authored it; 2. a shape that contains it and whose text cannot be read at all - a chart, whose labels live in chart XML rather than in a text frame; 3. the nearest shape that authored it, when nothing containing it did - this is the signature of text that escaped its box; 4. a shape that contains it but did not author it; 5. the nearest shape of any kind, if it is close enough to be plausible. Within a tier the smallest box wins, then the earliest. That tiebreak is quantized on purpose: two boxes built to the same width differ in the twelfth decimal place after the EMU-to-point conversion, so an unquantized `min` on area chose between identical boxes on floating-point noise and named the wrong one in the report. """ candidates = [s for s in shapes if s.can_own_text] return [(word, _owner_for(candidates, word)) for word in words] def _rank(shape): return (round(shape.rect.area, 3), shape.index) def _nearest(shapes, word): return min(shapes, key=lambda s: (round(s.rect.distance_to(word.rect), 3),) + _rank(s)) def _owner_for(candidates, word): if not candidates: return None key = _normalize(word.text) inside = [s for s in candidates if s.rect.contains_point(word.rect.cx, word.rect.cy)] claimed = [s for s in inside if _claims(s, key)] if claimed: whole = [s for s in claimed if s.rect.contains(word.rect, s.slack)] return min(whole or claimed, key=_rank) opaque = [s for s in inside if not s.text] if opaque: return min(opaque, key=_rank) elsewhere = [s for s in candidates if _claims(s, key)] if elsewhere: best = _nearest(elsewhere, word) if best.rect.distance_to(word.rect) <= STRAY_DISTANCE_PT: return best if inside: whole = [s for s in inside if s.rect.contains(word.rect, s.slack)] return min(whole or inside, key=_rank) nearest = _nearest(candidates, word) if nearest.rect.distance_to(word.rect) > STRAY_DISTANCE_PT: return None return nearest def detect_text_overflow(slide, owners): """Text that rendered outside the shape it belongs to. Reported per shape, not per word, and graded against the escaping word's own height - which is one rendered line. Less than a line poking out of a box is untidy and usually invisible; a whole line or more outside the box is content the layout is no longer holding. Grading against the line height rather than a fixed number of points keeps the rule the same for a 44pt cover title and an 11pt source note. Whether the escaped text also left the slide's safe margins is deliberately NOT mentioned here - detect_outside_margins owns that claim, and having two detectors argue about the same measurement produced a report that said "0.0pt past the left of the safe margin". """ worst = {} for word, shape in owners: if shape is None: continue # Only a shape that actually authored the word may be named as the box # it escaped from. Assignment falls back to containment for words it # cannot attribute - a one-character Cyrillic preposition claims # nothing, because it appears in nearly every shape on the slide - and # without this guard those words land on whatever box they drifted # into, producing findings that quote a table cell as the overflow of # the source note beneath it. A shape with no readable text at all is # exempt: that is a chart, whose labels are never in a text frame. if shape.text and not _claims(shape, _normalize(word.text)): continue escape = word.rect.escape(shape.rect, shape.slack) if not escape: continue depth = max(escape.values()) current = worst.get(shape.index) if current is None or depth > current[0]: worst[shape.index] = (depth, shape, word, escape) findings = [] for depth, shape, word, escape in sorted(worst.values(), key=lambda v: -v[0]): line_height = max(word.rect.height, 1.0) severity = "error" if depth >= line_height else "warning" findings.append(Finding( slide=slide.number, kind="text-overflow", severity=severity, what=("Text renders outside %s - %s (worst word: %r, on a %.0fpt " "line). %s" % (shape.label(), _sides(escape), word.text, line_height, "That is %.1f lines of text outside the box." % (depth / line_height) if severity == "error" else "That is under one line, so it may only be the " "ascender or descender.")), fix=("Shorten the text or split the slide. If this is a table, " "drop rows or shorten cells so no cell wraps to two lines - a " "wrapped cell grows its row and pushes the whole table down. " "Do not reduce the type size."), where=shape.label(), )) return findings def safe_rect(page, margins): left, top, right, bottom = margins return Rect(left * 72.0, top * 72.0, page.width - right * 72.0, page.height - bottom * 72.0) def detect_outside_margins(slide, margins, owners): """Rendered text outside the safe area, whatever box it came from. Independent of the overflow check on purpose: a shape can be placed outside the margins in the first place, in which case its text never escapes anything and the overflow detector has nothing to say. """ safe = safe_rect(slide.page, margins) offenders = [] for word, _shape in owners: escape = word.rect.escape(safe, SHAPE_SLACK_PT) if escape: offenders.append((max(escape.values()), word, escape)) if not offenders: return [] offenders.sort(key=lambda v: -v[0]) depth, word, escape = offenders[0] off_slide = word.rect.escape(slide.page) return [Finding( slide=slide.number, kind="outside-margins", severity="error", what=("%d word(s) render outside the safe margins; the worst is %r, %s." % (len(offenders), word.text, _sides(escape)) + (" It is off the slide edge entirely." if off_slide else "")), fix=("Move the shape back onto the 12-column grid, or shorten the " "text so it stops before the margin. Content in the margin is " "cropped by some projectors and printers."), where="%.2fin from the left, %.2fin from the top" % (inches(word.rect.x0), inches(word.rect.y0)), )] def detect_collisions(slide, owners): """Words rendered on top of other words. Words on the same line of the same block are skipped: those touch by kerning, never by collision, and a renderer will not overlap them. Anything else that overlaps by more than COLLISION_AREA_RATIO of the smaller word is ink on ink, which is always a defect. """ entries = list(owners) seen_charts = {} generic = [] for i in range(len(entries)): word_a, shape_a = entries[i] for j in range(i + 1, len(entries)): word_b, shape_b = entries[j] if word_a.block == word_b.block and word_a.line == word_b.line: continue inter = word_a.rect.intersection(word_b.rect) if inter.width <= 0 or inter.height <= 0: continue smaller = min(word_a.rect.area, word_b.rect.area) if smaller <= 0 or inter.area / smaller < COLLISION_AREA_RATIO: continue chart = None if (shape_a is not None and shape_a is shape_b and shape_a.kind == "chart"): chart = shape_a if chart is not None: seen_charts.setdefault(chart.index, (chart, []))[1].append( (word_a, word_b, inter.area / smaller)) else: generic.append((word_a, word_b, inter.area / smaller)) findings = [] for chart, pairs in seen_charts.values(): word_a, word_b, ratio = max(pairs, key=lambda p: p[2]) findings.append(Finding( slide=slide.number, kind="chart-label-collision", severity="error", what=("%d pair(s) of labels overlap inside %s; the worst is %r on " "%r, sharing %.0f%% of the smaller label's area." % (len(pairs), chart.label(), word_a.text, word_b.text, ratio * 100)), fix=("The category axis is crowded. Group categories, shorten the " "labels, or switch a ranked comparison to the 'progress' goal " "so the labels run down the side instead of along the bottom."), where=chart.label(), )) if generic: word_a, word_b, ratio = max(generic, key=lambda p: p[2]) findings.append(Finding( slide=slide.number, kind="text-collision", severity="error", what=("%d pair(s) of words render on top of each other; the worst " "is %r on %r, sharing %.0f%% of the smaller word's area." % (len(generic), word_a.text, word_b.text, ratio * 100)), fix=("Two blocks are landing in the same place. Shorten whichever " "one grew, or move it onto its own slide."), where="%.2fin from the left, %.2fin from the top" % (inches(word_a.rect.x0), inches(word_a.rect.y0)), )) return findings def detect_shape_overlap(slide): """Shapes that partially overlap in the .pptx itself. Containment is not overlap: the house style deliberately puts a text box inside a card and a card on a full-bleed surface, and every one of those is a containment. What is never intentional is two boxes that each stick out of the other - that is arithmetic gone wrong in a column width. """ findings = [] shapes = [s for s in slide.shapes if s.kind != "group"] for i in range(len(shapes)): for j in range(i + 1, len(shapes)): a, b = shapes[i], shapes[j] inter = a.rect.intersection(b.rect) if inter.width <= GRAZE_PT or inter.height <= GRAZE_PT: continue if a.rect.contains(b.rect, OVERLAP_SLACK_PT) or \ b.rect.contains(a.rect, OVERLAP_SLACK_PT): continue # Containment in practice, not in arithmetic. Exact containment # missed the case that matters on an art-directed deck: a logo, a # caption or a title sitting ON a full-bleed photograph, poking a # point or two outside it. Measured on the deck this house style # came from, that produced 38 of its 76 overlap warnings - every # one of them the design working as intended, and every one of # them noise on top of the `text-over-image` check that already # judges whether the type is legible there. big, small = (a, b) if a.rect.area >= b.rect.area else (b, a) if (big.kind == "picture" and small.rect.area and inter.area >= small.rect.area * MOSTLY_INSIDE): continue findings.append(Finding( slide=slide.number, kind="shape-overlap", severity="warning", what=("%s and %s overlap by %.2fin x %.2fin without either " "containing the other." % (a.label(), b.label(), inches(inter.width), inches(inter.height))), fix=("Place both on the 12-column grid with col() and span() " "rather than at hand-computed coordinates, or give the " "one that grew fewer columns."), where=a.label(), )) return findings PLACEHOLDER_PATTERNS = [ # Only patterns that cannot plausibly be real content in a finished deck. # A word like "example" or "draft" is a real word in a real sentence and is # deliberately absent: a detector that cries wolf gets switched off. (r"lorem\s+ipsum", "Lorem ipsum"), (r"\bTBD\b", "TBD"), (r"\bTODO\b", "TODO"), (r"\bFIXME\b", "FIXME"), (r"\bXXX\b", "XXX"), (r"click to (add|edit)", "a PowerPoint prompt"), (r"\bsample text\b", "sample text"), (r"\byour (text|title) here\b", "your text here"), (r"образец (заголовка|текста|подзаголовка)", "a PowerPoint prompt"), (r"текст слайда", "a PowerPoint prompt"), (r"заглушк", "a placeholder"), (r"\[[^\]]*(fill|insert|todo|tbd|name|заполн|вставь|вставить)[^\]]*\]", "a bracketed fill-in"), ] def detect_placeholders(slide): """Template text nobody replaced. Read from the .pptx, not the render. The .pptx is the exact authored string; the render is the same string after line breaking, hyphenation and font substitution have had a go at it. For a literal match the source is strictly better. """ findings = [] for shape in slide.shapes: if not shape.text: continue for pattern, label in PLACEHOLDER_PATTERNS: match = re.search(pattern, shape.text, re.IGNORECASE) if match: findings.append(Finding( slide=slide.number, kind="placeholder-text", severity="error", what=("%s still contains %s: %r." % (shape.label(), label, match.group(0))), fix="Replace it with the real content, or delete the slide.", where=shape.label(), )) break return findings def detect_blank(slide): """Slides with nothing on them. Two independent signals, because each has a blind spot. The .pptx knows whether anything was authored, and misses a slide whose content is all empty strings; the raster knows whether anything was drawn, and misses a slide whose only content is white-on-white. """ findings = [] has_content = any( s.text or s.kind in ("picture", "chart", "table") for s in slide.shapes ) if not has_content: return [Finding( slide=slide.number, kind="blank-slide", severity="error", what="No text, picture, chart or table on this slide.", fix="Put something on it or delete it.", )] if slide.ink_ratio is not None and slide.ink_ratio < BLANK_INK_RATIO: findings.append(Finding( slide=slide.number, kind="blank-slide", severity="error", what=("The slide has content in the file but rendered almost " "empty (%.3f%% of pixels are not background). Text the same " "colour as the surface behind it looks like this." % (slide.ink_ratio * 100)), fix=("Check the text colour against the slide background, and " "check the shape is not behind a filled rectangle."), )) return findings def detect_bad_images(slide): """Pictures that will not render, and pictures that rendered as nothing.""" findings = [] for shape in slide.shapes: if shape.kind != "picture": continue if not shape.image_ok: findings.append(Finding( slide=slide.number, kind="image-failed", severity="error", what="Picture %r will not load: %s." % (shape.name, shape.image_note), fix=("Re-export the image and embed it again. Confirm the " "source path exists before referencing it - a deck must " "never ship a broken image frame."), where=shape.label(), )) continue region = slide.image_regions.get(shape.index) if region is not None and region < BLANK_INK_RATIO: findings.append(Finding( slide=slide.number, kind="image-failed", severity="error", what=("Picture %r decodes, but the area it occupies rendered " "as one flat colour - an empty frame." % shape.name), fix=("The renderer could not draw it. Convert it to PNG or " "JPEG and embed it again."), where=shape.label(), )) return findings def detect_illegible_over_image(slide): """Text over a picture that is not legible against it. The measurement is in `slide.backdrops`, put there by the raster pass - this is the same split every other detector has, so the rule itself stays pure and testable against recorded output. The fix is never "make the text bigger" and never "pick a darker photo". It is a scrim: an opaque-enough wash between the picture and the type, which is what the builder's image primitives put there and what a hand-built slide leaves out. """ findings = [] for shape in slide.shapes: measured = slide.backdrops.get(shape.index) if measured is None: continue ratio, word = measured if ratio >= CONTRAST_WARN: continue severity = "error" if ratio < CONTRAST_ERROR else "warning" findings.append(Finding( slide=slide.number, kind="text-over-image", severity=severity, what=("%s renders over a picture at a contrast of %.1f:1 against " "the pixels actually behind it (worst word: %r). %s" % (shape.label(), ratio, word, "Below 3:1 nothing at this size is readable." if severity == "error" else "Large display type clears 3:1, but a lead line or a " "caption at this contrast does not.")), fix=("Put a scrim between the picture and the text - a dark " "gradient wash across the image, heaviest where the type " "sits. The builder's hero() and image_content() do this for " "you and cannot be told not to; a hand-placed text box over " "a picture has to do it explicitly. Do not fix this by " "enlarging the type or by choosing a different photograph - " "the next photograph is a coin toss."), where=shape.label(), )) return findings # --- Russian typography --------------------------------------------------- # # Everything above this line is geometry. This is orthography, and it lives in # this file rather than in a script of its own on purpose: one QA entry point, # one report format, one exit status. An agent that has been told to check its # deliverable should not have to know which of two tools to reach for, and a # second tool is a second thing to forget. # # It reads three kinds of input: # # * the text of a .pptx, taken from the shapes (detect_russian_typography, # wired into inspect_slide below); # * an HTML document or dashboard, and a .md or .txt file # (inspect_text_document, which is what main() reaches for when it is # handed something that is not a deck). # # So `python3 plank_deck_qa.py отчёт.html` and `python3 plank_deck_qa.py # дашборд.html` work the same way `python3 plank_deck_qa.py колода.pptx` does. # # WHY MECHANICAL. Prose quality needs a reader; typography does not. Whether a # Russian sentence uses «ёлочки» or "straight quotes", whether a dash is an em # dash or a hyphen somebody typed instead, whether ё is written - all three are # decidable from the bytes. A rule that is decidable should be decided, not # suggested, because an instruction catches whatever the model happens to # notice and a check catches every instance every time. The half that does need # a reader - AI-flavoured vocabulary, nominalisation, empty adjectives - lives # in the `deliverable-writing` skill, which points here for this half. # # WHY IT IS DELIBERATELY INCOMPLETE. Flagging correct Russian is worse than # missing wrong Russian. A checker that cries wolf gets switched off, and a # switched-off checker catches nothing at all. So every rule below is one whose # violation is certain, and several rules that are genuinely part of Russian # typographic convention are left out for being undecidable in isolation: # # * ё in any word whose е-spelling is also a word: все/всё, чем/чём, # узнаем/узнаём, берет/берёт. The list below holds only words with one # reading, and it is short on purpose. # * a non-breaking space after a one-letter preposition («в 2026 году»). # Correct, and it would fire on nearly every honest paragraph. # * a missing space before % («40 %» vs «40%»). Business Russian writes both # and neither is wrong, so only a space that IS there and should have been # non-breaking is reported. # * straight quotes around a Latin-only string inside a Russian sentence # ("Excel"). Telling that from a quoted identifier needs a reader. # * anything inside or 
, a fenced code block,
#     a URL, a path, a filename or an e-mail address. A dashboard that builds
#     its chart labels in JavaScript therefore goes unchecked - a real gap,
#     and the alternative is proofreading source code.

RU_ALPHA = "А-Яа-яЁёӘәҒғҚқҢңӨөҰұҮүҺһІі"
_RU_WORD = re.compile("[%s]{3,}" % RU_ALPHA)
_RU_ANY = re.compile("[%s]" % RU_ALPHA)

# Below this, a line cannot wrap, so a space inside it cannot break in the
# wrong place and a non-breaking space would be decoration. This is what keeps
# the rule off "4 мин" in a KPI tile while keeping it on a paragraph.
RU_WRAPPABLE_LINE = 45

# How many times one rule is reported against one unit before the rest are
# summarised. A hundred identical findings is a wall, not a report.
RU_MAX_PER_KIND = 4


def has_russian(text, minimum=2):
    """True when there is enough Cyrillic here to be worth proofreading."""
    return len(_RU_WORD.findall(text or "")) >= minimum


def uses_yo(text):
    """Has the author decided to write ё at all?

    Two occurrences, not one, so a single «ё» in a proper noun somebody pasted
    does not turn on a rule the document was not written to.
    """
    return (text or "").count("ё") + (text or "").count("Ё") >= 2


# Spans no typographic rule may read. Masked to spaces rather than deleted so
# every offset below still points at the real text.
_PROTECTED = (
    re.compile(r"```.*?```", re.S),
    re.compile(r"`[^`\n]*`"),
    re.compile(r"", re.S),
    re.compile(r"https?://\S+"),
    re.compile(r"\bwww\.[^\s<>\"']+"),
    re.compile(r"[\w.+-]+@[\w-]+\.[\w.-]+"),
    re.compile(
        r"[\w./\\-]*\.(?:py|ts|tsx|js|jsx|json|html?|css|md|csv|xlsx?|pptx?"
        r"|docx?|pdf|png|jpe?g|svg|sql|sh|ya?ml|txt|zip)\b", re.I),
    re.compile(r"\b\d{4}-\d{2}-\d{2}\b"),                 # ISO dates
    re.compile(r"\bv?\d+\.\d+(?:\.\d+)+\b"),              # version numbers
    re.compile(r"\+\d[\d\-()  ]{7,}\d"),             # +7 700 000 00 00
    re.compile(r"\b\d{2,4}-\d{2,3}-\d{2,3}\b"),           # 300-11-22
)


def mask_protected(text):
    """Blank out code, links, paths and filenames, preserving offsets."""
    out = list(text)
    for pattern in _PROTECTED:
        for match in pattern.finditer(text):
            for i in range(match.start(), match.end()):
                if out[i] != "\n":
                    out[i] = " "
    return "".join(out)


def _excerpt(text, start, end, pad=26):
    left = max(0, start - pad)
    right = min(len(text), end + pad)
    snippet = re.sub(r"\s+", " ", text[left:right]).strip()
    return "%s%s%s" % ("..." if left else "", snippet,
                       "..." if right < len(text) else "")


# Words whose е-spelling has exactly one reading, weighted towards the
# vocabulary Plank's readers actually write: отчёт, учёт, расчёт, счёт, объём.
# Every entry here was checked for a second reading before it went in.
_YO_WORDS = (
    (r"еще", "ещё"),
    (r"ее", "её"),
    (r"мое", "моё"),
    (r"твое", "твоё"),
    (r"свое", "своё"),
    (r"причем", "причём"),
    (r"о[  ]чем", "о чём"),
    (r"ни[  ]при[  ]чем", "ни при чём"),
    (r"(?:в|о|на|при)[  ]нем", "нём"),
    (r"отчет\w*", "отчёт…"),
    (r"учет\w*", "учёт…"),
    (r"расчет\w*", "расчёт…"),
    (r"подсчет\w*", "подсчёт…"),
    (r"просчет\w*", "просчёт…"),
    (r"перерасчет\w*", "перерасчёт…"),
    (r"зачет\w*", "зачёт…"),
    (r"счет", "счёт"),
    (r"объем(?!лющ|лем)\w*", "объём…"),
    (r"прием(?!лем)\w*", "приём…"),
    (r"подъем\w*", "подъём…"),
    (r"четк\w*", "чётк…"),
    (r"серьезн\w*", "серьёзн…"),
    (r"надежн\w*", "надёжн…"),
    (r"партнер\w*", "партнёр…"),
    (r"трех\w*", "трёх…"),
    (r"четырех\w*", "четырёх…"),
    (r"(?:при|у|на|подо|пере|произо|вы|про|разо|ото|за|обо|до|со|изо)?шел",
     "шёл"),
    (r"(?:вы|до|за|из|на|от|пере|по|при|про|раз|со|у|воз|пред|препод)?"
     r"да(?:ет|ем)", "даёт"),
    (r"(?:жив|вед|раст|нес|цвет)(?:ет|ем)", "…ёт"),
    (r"(?:и|при|вый|вы|най|пой|зай|вой|подой|перей|произой|обой|отой|разой"
     r"|сой|дой|взой|уй)д(?:ет|ем)", "…йдёт"),
    (r"остает(?:ся)?", "остаётся"),
    (r"остаемся", "остаёмся"),
)
_YO_PATTERNS = tuple(
    # IGNORECASE, because «Отчет» at the start of a title and «СЧЕТ» in a
    # table header are the same mistake as the lower-case one, and a slide
    # title is exactly where the capital lives.
    (re.compile("(?= 3 and cyrillic * 2 >= len(letters)


def _sentence_is_russian(text, start, end, span=90):
    """Is the sentence AROUND this quote Russian?

    An English sentence quoting a Russian string - `A row reading "Меновазин
    40 мл" is one bottle` - is correctly punctuated English, and its quotes
    are English quotes. Only the text outside the quotation marks decides
    which language's rules apply, so the other quoted spans in the context are
    stripped before the question is asked: two adjacent Russian quotes must
    not vouch for each other.
    """
    context = "%s %s" % (text[max(0, start - span):start], text[end:end + span])
    return has_russian(_QUOTED_SPAN.sub(" ", context), minimum=2)


def _nested_quote_spans(text, reach=600):
    """Positions of a « opened while another « was still open.

    Counted per paragraph and within `reach` characters of the outer «: over a
    whole document a single unbalanced » leaves the depth wrong for everything
    after it, and every later quote is then reported as nested.
    """
    offset = 0
    for paragraph in text.split("\n\n"):
        open_at = []
        for match in re.finditer("[«»]", paragraph):
            if match.group() == "«":
                if open_at and match.start() - open_at[-1] <= reach:
                    yield offset + match.start(), offset + match.end()
                open_at.append(match.start())
            elif open_at:
                open_at.pop()
        offset += len(paragraph) + 2


def russian_typography_issues(text, check_yo=False, limit=RU_MAX_PER_KIND):
    """Every mechanical Russian typography defect in one block of text.

    `check_yo` is decided by the caller from the WHOLE deliverable, not from
    this fragment: a deck that writes ё on nine slides and forgets it on the
    tenth has a mistake, while a deck that never writes ё has a house style.
    """
    if not has_russian(text):
        return []
    masked = mask_protected(text)
    issues = []
    counts = {}
    extra = {}

    def add(kind, severity, start, end, what, fix):
        counts[kind] = counts.get(kind, 0) + 1
        if counts[kind] > limit:
            extra[kind] = extra.get(kind, 0) + 1
            return
        issues.append(RuIssue(
            kind=kind, severity=severity,
            what="%s: %s" % (what, _excerpt(text, start, end)), fix=fix))

    for pattern, label in ((_STRAIGHT_QUOTED, 'straight quotes "..."'),
                           (_CURLY_QUOTED, "English quotes “...”")):
        for match in pattern.finditer(masked):
            if not _mostly_russian(match.group(1)):
                continue
            if not _sentence_is_russian(masked, match.start(), match.end()):
                continue
            add("ru-quotes", "error", match.start(), match.end(),
                "Russian text in %s" % label,
                "Russian takes «ёлочки»: «Итоги квартала». "
                "A quote inside a quote takes „лапки“.")

    for start, end in _nested_quote_spans(masked):
        add("ru-quotes-nested", "warning", start, end,
            "A second « opens while one is still open",
            "The inner pair takes „лапки“: «договор „Отчёт“».")

    for match in _HYPHEN_FOR_DASH.finditer(masked):
        add("ru-dash", "error", match.start(), match.end(),
            "A hyphen is standing in for a dash",
            "A dash between clauses is an em dash with a space on each "
            "side: «Выручка — рекорд». The hyphen belongs to compounds "
            "only (по-русски, из-за), and those never take spaces.")

    for match in _EN_DASH_FOR_EM.finditer(masked):
        add("ru-dash", "warning", match.start(), match.end(),
            "An en dash – is being used between clauses",
            "Russian uses the em dash — between clauses and reserves the "
            "en dash – for ranges (2024–2026).")

    for match in _GLUED_EM_DASH.finditer(masked):
        add("ru-dash", "warning", match.start(), match.end(),
            "An em dash with no space beside it",
            "An em dash between words takes a space on each side.")

    for match in _HYPHEN_RANGE.finditer(masked):
        add("ru-dash-range", "warning", match.start(), match.end(),
            "A numeric range written with a hyphen",
            "A range takes an en dash and no spaces: 5–10, "
            "2024–2026, 10–12 %.")

    for match in _THOUSANDS_COMMA.finditer(masked):
        add("ru-number-format", "error", match.start(), match.end(),
            "Thousands grouped with commas, which is English",
            "Russian groups thousands with a non-breaking space and marks the "
            "decimal with a comma: 1 234 567, not 1,234,567.")

    for match in _DECIMAL_POINT.finditer(masked):
        add("ru-number-format", "warning", match.start(), match.end(),
            "A decimal point in front of a unit",
            "Russian writes the decimal with a comma: 12,5 %, not 12.5%.")

    offset = 0
    for line in text.split("\n"):
        if len(line) >= RU_WRAPPABLE_LINE:
            masked_line = masked[offset:offset + len(line)]
            for match in _NUMBER_UNIT_SPACE.finditer(masked_line):
                # `111 % 100 == 11` is a modulo, not eleven per cent. Caught in
                # a Russian help page explaining plural agreement; a percentage
                # is never followed by another number.
                if _MODULO.match(masked_line, match.end()):
                    continue
                add("ru-nbsp", "warning",
                    offset + match.start(), offset + match.end(),
                    "An ordinary space between a number and its unit",
                    "Use a non-breaking space (U+00A0) so the value "
                    "cannot be split across two lines: 12 %, 5 млн, 2026 г.")
        offset += len(line) + 1

    if check_yo:
        # Not inside «...». What is in quotation marks is usually a name
        # copied out of another system - «Счет на оплату покупателю» and
        # «Бухгалтерский учет для Казахстана» are how 1C spells them - and an
        # accountant does not want a document type silently renamed to be
        # more orthographically correct than the software it came from. This
        # loses real findings inside real quotations; that is the trade.
        yo_text = _INSIDE_GUILLEMETS.sub(lambda m: " " * len(m.group()), masked)
        for pattern, correct in _YO_PATTERNS:
            for match in pattern.finditer(yo_text):
                add("ru-yo", "warning", match.start(), match.end(),
                    "ё is missing where the word takes it (%s)" % correct,
                    "The rest of this deliverable writes ё, so this is an "
                    "inconsistency rather than a house style. Write it.")

    for kind, more in extra.items():
        issues.append(RuIssue(
            kind=kind, severity="info",
            what="%d further %s finding(s) not listed." % (more, kind),
            fix="Fix the ones above and re-run; the rest are the same defect."))
    return issues


def detect_russian_typography(slide, check_yo=False):
    """The typography rules, per shape, so the report can name the box."""
    findings = []
    for shape in slide.shapes:
        if not shape.text:
            continue
        for issue in russian_typography_issues(shape.text, check_yo=check_yo):
            findings.append(Finding(
                slide=slide.number, kind=issue.kind, severity=issue.severity,
                what="%s %s" % (shape.label(), issue.what),
                fix=issue.fix, where=shape.label()))
    return findings


# --- HTML, markdown and plain text ----------------------------------------

_HTML_COMMENT = re.compile(r"", re.S)
_HTML_OPAQUE = re.compile(r"<(script|style|code|pre)\b.*?", re.S | re.I)
_HTML_BLOCK = re.compile(
    r"]*>", re.I)
_HTML_TAG = re.compile(r"<[^>]+>", re.S)

TEXT_SUFFIXES = (".html", ".htm", ".md", ".markdown", ".txt")


def html_to_text(source):
    """The visible text of an HTML document or dashboard.

    Entities are unescaped, which matters more than it looks: ` ` has to
    become U+00A0 here, or a document that got its non-breaking spaces right
    would be reported for not having them.
    """
    text = _HTML_COMMENT.sub(" ", source)
    text = _HTML_OPAQUE.sub(" ", text)
    text = _HTML_BLOCK.sub("\n", text)
    text = _HTML_TAG.sub(" ", text)
    return html.unescape(text)


def read_document_source(path):
    with open(path, encoding="utf-8", errors="replace") as handle:
        return handle.read()


# --- diagrams -------------------------------------------------------------
#
# A diagram in a Plank deliverable is inline SVG in the same file as everything
# else, and it fails in ways the Russian rules above cannot see. All of them
# are decidable from the bytes, which is why they are here rather than in the
# guidance at /help/diagrams:
#
#   * THEME. The house style is one file that follows its host's light/dark
#     theme, and the sheet does that with CSS custom properties. An SVG that
#     paints `fill="#0a0a0f"` instead of `fill="var(--ink)"` is correct in the
#     theme it was drawn in and invisible in the other one. This is the single
#     most common way a diagram copied out of a general-purpose diagram library
#     lands wrong here: almost every such library bakes literal hexes into the
#     SVG and ships a *separate file* for dark mode, which this house style has
#     no place to put. Nothing at render time notices, because the diagram is
#     still there - it is the same colour as the paper.
#
#   * ACCESSIBILITY. An  is announced to a screen reader as a graphic. If
#     it carries no accessible name the reader is told there is a picture and
#     nothing about what it shows, and a diagram is usually the part of the
#     document that carries the argument. So an SVG is either LABELLED - the
#     `role="img"` + `aria-labelledby` -> / contract - or
#     explicitly marked decorative with `aria-hidden="true"`. Anything in
#     between is a defect, and which of the two it should be is a judgement the
#     author has to make rather than one this tool can guess.
#
#   * IDs. Two diagrams pasted from the same template bring two `id="arrow"`
#     markers and two `id="title"` labels into one document. The duplicate is
#     not a style problem: `url(#arrow)` resolves to the first one, and the
#     second diagram's `aria-labelledby` can announce the first diagram's name.
#
#   * SELF-CONTAINMENT. A remote `` inside the SVG
#     means the deliverable stops being one file - it breaks the moment it is
#     forwarded, downloaded or opened offline, which is most of the time.
#
# The contract these check is written out for a human at /help/diagrams. The
# design rules that produced it - complexity budget, connector grammar, the
# accessible-SVG contract - were adapted from the MIT-licensed diagram-design
# project (https://github.com/cathrynlavery/diagram-design); see
# docs/technical/diagram-design-audit.md for what was taken and what was not.

# Paint values that are NOT a locked colour. Everything else in a paint slot is
# a literal the theme cannot move.
THEME_SAFE_PAINT = {
    "none", "transparent", "inherit", "initial", "unset", "currentcolor",
    "context-fill", "context-stroke",
}

# Attributes whose value is a paint. `color` is here because it is what
# `currentColor` resolves against, so a locked `color` locks everything that
# inherits from it.
PAINT_ATTRS = ("fill", "stroke", "stop-color", "flood-color",
               "lighting-color", "color")

# A colour written out, in any notation CSS accepts. The theme cannot move any
# of these, and that is the whole test - it does not matter whether the literal
# sits on its own, inside a `var()` FALLBACK (`var(--gone, #fff)`, which is the
# value that gets used exactly when the token is missing), or after a paint
# server (`url(#grad) #fff`). If a literal is reachable, the paint is locked.
# The `(?` is deliberately
# absent: a hyperlink inside a diagram is ordinary, correct markup and does not
# make the file depend on anything, so reporting it was the checker crying wolf
# on the one case it should stay quiet about.
RESOURCE_TAGS = {
    "image", "use", "feimage", "script", "img", "audio", "video", "source",
    "track", "iframe", "embed", "object",
}

# HTML tags that END an  when the parser meets them in foreign content
# outside a . This is the HTML spec's own breakout list, and a
# browser applies it: `
text
` renders the paragraph
# OUTSIDE the diagram. Honouring it here is what keeps the scanner's idea of
# where a diagram ends the same as the reader's.
HTML_BREAKOUT = {
    "b", "big", "blockquote", "body", "br", "center", "code", "dd", "div",
    "dl", "dt", "em", "embed", "h1", "h2", "h3", "h4", "h5", "h6", "head",
    "hr", "i", "img", "li", "listing", "menu", "meta", "nobr", "ol", "p",
    "pre", "ruby", "s", "small", "span", "strong", "strike", "sub", "sup",
    "table", "tt", "u", "ul", "var",
}


class _DiagramScan(HTMLParser):
    """Every inline  in a document, and what is inside it.

    HTMLParser rather than an XML parser on purpose. The input is a whole HTML
    deliverable, which is not well-formed XML, and an XML parser would also
    expand entity declarations - a 400-byte file can be written to expand into
    hundreds of megabytes that way, and a QA tool is exactly the thing an
    untrusted file gets pointed at first.

    FINDING WHERE A DIAGRAM ENDS
    ----------------------------
    That is the whole difficulty, and getting it wrong in the permissive
    direction is the failure that matters most: a scanner still "inside" the
    diagram reports the document's ordinary prose as diagram defects, and a
    checker that flags things that are fine gets switched off. So three
    separate rules decide the boundary, each matching what a browser does:

      * a STACK of open tag names, not a counter, because HTML lets you leave
        tags open - `
one
two`, `
  • a
  • b`, `x` are all valid and all
        appear inside a real , and a counter never returns to
        zero after them;
      * the HTML BREAKOUT list, because `
    x
    ` puts that
        paragraph outside the diagram in every browser;
      * an UNTERMINATED  is reported as exactly that, and its other
        findings are dropped, because everything collected after the missing
        `` is the rest of the document rather than the diagram.
    """

    # Popping these on a new start tag is what makes implied end tags safe.
    _IMPLIED_CLOSE = {
        "p": {"p", "div", "ul", "ol", "table", "h1", "h2", "h3", "h4", "h5",
              "h6", "blockquote", "pre", "figure", "section", "article"},
        "li": {"li"},
        "dt": {"dt", "dd"},
        "dd": {"dt", "dd"},
        "td": {"td", "th", "tr"},
        "th": {"td", "th", "tr"},
        "tr": {"tr"},
        "option": {"option"},
        "thead": {"tbody", "tfoot"},
        "tbody": {"tbody", "tfoot"},
    }

    def __init__(self):
        super().__init__(convert_charrefs=True)
        self.svgs = []
        self.page_css = []          #