diff --git a/CHANGELOG.md b/CHANGELOG.md index 82562c1..e18833d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,21 @@ All notable changes to **export_1c_help** are documented in this file. ## [0.3.8] - 2026-07-29 +## [0.3.9] - 2026-07-29 + +### Added + +- New CLI command `export_docs.py clean` to remove export_docs-managed wiki pages by `--folder` manifest and remove export_docs nav block from `Home`/`_Sidebar`. +- Auto-generated per-folder index pages with links to articles using full page titles. + +### Changed + +- `export_docs.py` defaults to flat root wiki page naming (`docs__path__name.md`) for Gitea compatibility where nested `/wiki/docs/...` routes return 404. +- Markdown links are URL-encoded in generated indexes and rewritten content to keep links with spaces/parentheses/Cyrillic valid. +- Wiki clone reuse now validates `origin` URL before fetch/reset; clone is recreated if target wiki differs. + +## [0.3.8] - 2026-07-29 + ### Changed - Wiki page filenames switched to short ASCII transliteration slugs (with stable hash suffix on truncation), while visible page titles remain full Russian. diff --git a/README.md b/README.md index b547008..c682067 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,12 @@ # export_1c_help -**Версия:** см. [`VERSION`](VERSION) (текущая: **0.3.4**) · [CHANGELOG](CHANGELOG.md) +**Версия:** см. [`VERSION`](VERSION) (текущая: **0.3.9**) · [CHANGELOG](CHANGELOG.md) **Лицензия:** [MIT](LICENSE) Выгрузка встроенной справки **любой** конфигурации 1С (`**/Ext/Help/ru.html`) в Markdown и публикация в wiki git-репозитория (Gitea / GitLab: `*.wiki.git`). +Дополнительно: публикация произвольных Markdown-статей в подпапки wiki через `export_docs.py` (спецификация: [`SPEC_EXPORT_DOCS.md`](SPEC_EXPORT_DOCS.md)). + Зависимости: **Python 3.10+**, только стандартная библиотека. --- @@ -143,6 +145,37 @@ python3 export_1c_help.py push -c /path/to/config | `convert` | только Markdown на диск | | `push` | convert + commit/push в wiki | +### Публикация произвольных Markdown-документов + +```bash +python3 export_docs.py push \ + --files ./docs/*.md \ + --folder docs +``` + +Если `-s/ -c` не указаны, wiki URL берётся из git remote текущей папки. + +Выгружаются только `.md`, но если внутри них есть локальные ссылки на файлы (картинки/документы), эти файлы тоже копируются в wiki и относительные ссылки сохраняются. Поддерживаются и reference-style ссылки через `[id]: target` (для `[text][id]` переопределения переписываются). + +С сохранением структуры подкаталогов: + +```bash +python3 export_docs.py push \ + --files ./docs/**/*.md \ + -s /path/to/config/src \ + --folder migration/docs +``` + +Очистка опубликованных `export_docs`-страниц из wiki: + +```bash +python3 export_docs.py clean \ + --folder docs \ + --wiki-url https://git.example/org/repo.wiki.git +``` + +`clean` не принимает `--files`: удаление идёт по сохранённому `manifest.export_docs.*.json` для указанного `--folder`. + Общие параметры источника (взаимоисключающие): | Параметр | Описание | diff --git a/SPEC_EXPORT_DOCS.md b/SPEC_EXPORT_DOCS.md new file mode 100644 index 0000000..fc53435 --- /dev/null +++ b/SPEC_EXPORT_DOCS.md @@ -0,0 +1,77 @@ +# Спецификация `export_docs.py` + +## Цель + +Публикация аналитических Markdown-статей из рабочих каталогов проекта (`docs/` и др.) в wiki-репозиторий (`*.wiki.git`) с: + +- размещением в указанной папке wiki (`--folder`); +- сохранением структуры подкаталогов по маске; +- конвертацией локальных ссылок между выгружаемыми `.md`-файлами. + +## CLI + +```bash +python export_docs.py push \ + --files [ ...] \ + -s \ + --folder \ + [--wiki-url ] [--remote origin] \ + [--work-dir ] [-o ] [--branch main] \ + [-m "commit message"] [--dry-run] +``` + +## Параметры + +- `--files` (обяз.): одна или несколько glob-масок. + - Экспортируются только `.md`. + - Если маска указывает на каталог — берутся `**/*.md`. + - Пример: `./docs/*.md`, `./docs/**/*.md`, `./reports`. +- `-s/--src` или `-c/--config` (опц.): путь к конфигурации/`src` только для определения wiki URL по remote (аналогично `export_1c_help.py`). + - Если не указаны, анализируется текущая папка: берётся git remote текущей папки и из неё выводится `*.wiki.git`. +- `--folder` (опц., по умолчанию `docs`): папка назначения в wiki, например `docs` или `migration/docs`. +- `--wiki-url` (опц.): явный wiki URL; если не указан, выводится из `git remote` источника. +- `--dry-run`: сборка + подготовка clone, без `git push`. + +## Правила размещения файлов + +1. Для каждой маски определяется статический префикс (часть до wildcard). +2. Для каждого совпавшего `.md` считается относительный путь от этого префикса. +3. В wiki файл размещается как: + - `/`. + +Примеры: + +- `--files ./docs/*.md --folder docs` + - `docs/a.md` → `docs/a.md` +- `--files ./docs/**/*.md --folder migration/docs` + - `docs/plan.md` → `migration/docs/plan.md` + - `docs/phases/p1.md` → `migration/docs/phases/p1.md` + +## Конвертация ссылок + +Для Markdown-ссылок вида `[text](target)` и `![alt](target)`: + +- внешние URL (`http(s)`, `mailto`, `ftp`), якоря `#...`, абсолютные `/...` — без изменений; +- относительные ссылки на выгружаемые `.md` пересчитываются на новый относительный путь между целевыми файлами в wiki; +- ссылки на невыгружаемые файлы остаются как есть. + +Цель: сохранить ссылочность внутри набора опубликованных статей. + +## Управление удалениями (без потери чужих wiki-страниц) + +Используется отдельный manifest для `export_docs` в корне wiki clone: + +- `manifest.export_docs..json`. + +При следующем запуске для того же `--folder`: + +- файлы из предыдущего manifest, которых нет в новой выгрузке, удаляются; +- статьи/файлы, не управляемые этим manifest, не трогаются. + +## Ограничения текущей версии + +- Автоматически обрабатываются только `.md` как источники выгрузки. +- Вложения (картинки/файлы), на которые есть локальные ссылки в `[](...)` / `![](...)`, **копируются** в wiki вместе со статьями (с сохранением структуры каталогов относительно статического префикса маски). +- Rewrite поддерживает как: + - inline-ссылки/картинки: `[](...)` и `![](...)`; + - reference-style определения: `[id]: target` (то есть ссылки вида `[text][id]` остаются рабочими, т.к. переопределения переписываются). diff --git a/VERSION b/VERSION index 6678432..940ac09 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.3.8 +0.3.9 diff --git a/export1c_help/docs_export.py b/export1c_help/docs_export.py new file mode 100644 index 0000000..a3effb7 --- /dev/null +++ b/export1c_help/docs_export.py @@ -0,0 +1,680 @@ +"""Export arbitrary markdown docs to wiki subfolders.""" + +from __future__ import annotations + +import glob +import hashlib +import json +import os +import re +import subprocess +from dataclasses import dataclass +from pathlib import Path +from typing import Iterable +from urllib.parse import quote + +from export1c_help import __version__ +from export1c_help.fscompat import copy_file, ensure_dir, remove_file, write_text +from export1c_help.wiki import WikiPushError, prepare_wiki_clone + +_MD_LINK_RE = re.compile(r"(!?\[[^\]]*\]\()([^)]+)(\))") +_MD_REF_DEF_RE = re.compile( + r"^(?P\s*\[[^\]]+\]:\s*)(?P<[^>]+>|\S+)(?P.*)$", + re.MULTILINE, +) +_URL_SCHEMES = ("http://", "https://", "mailto:", "ftp://") + +_EXPORT_DOCS_MARKER_BEGIN = "" +_EXPORT_DOCS_MARKER_END = "" +_RESERVED_ROOT_PAGES = {"Home.md", "History.md", "_Sidebar.md"} + + +@dataclass(frozen=True) +class DocItem: + src: Path + src_rel: Path + dst_rel: Path + pattern_root: Path + + +@dataclass(frozen=True) +class DocsStats: + files_total: int + files_written: int + files_deleted: int + markdown_files: int + attachment_files: int + folders: dict[str, tuple[int, int]] + + +def _has_glob(s: str) -> bool: + return any(ch in s for ch in ("*", "?", "[")) + + +def _static_prefix(pattern: str) -> Path: + path = Path(pattern) + parts = list(path.parts) + keep: list[str] = [] + for part in parts: + if _has_glob(part): + break + keep.append(part) + if not keep: + return Path.cwd() + return Path(*keep) + + +def _iter_markdown_files(raw_pattern: str) -> list[Path]: + p = Path(raw_pattern) + if p.is_file(): + return [p.resolve()] if p.suffix.lower() == ".md" else [] + if p.is_dir(): + return sorted(x.resolve() for x in p.rglob("*.md") if x.is_file()) + matched = [Path(x).resolve() for x in glob.glob(raw_pattern, recursive=True)] + return sorted(x for x in matched if x.is_file() and x.suffix.lower() == ".md") + + +def collect_docs(files_patterns: list[str], *, folder: str) -> list[DocItem]: + if not files_patterns: + raise ValueError("no --files patterns supplied") + folder_clean = folder.strip().strip("/").strip("\\") + dst_root = Path(folder_clean) if folder_clean else Path(".") + + explicit_files = [ + Path(raw).resolve() + for raw in files_patterns + if not _has_glob(raw) and Path(raw).is_file() + ] + explicit_root: Path | None = None + if explicit_files: + common = os.path.commonpath([str(p) for p in explicit_files]) + common_path = Path(common) + explicit_root = common_path if common_path.is_dir() else common_path.parent + + items: dict[Path, DocItem] = {} + for raw in files_patterns: + raw_path = Path(raw) + if explicit_root is not None and not _has_glob(raw) and raw_path.is_file(): + root = explicit_root + else: + root = _static_prefix(raw).resolve() + for src in _iter_markdown_files(raw): + try: + src_rel = src.relative_to(root) + except ValueError: + src_rel = Path(src.name) + dst_rel = dst_root / src_rel + items[src] = DocItem( + src=src, + src_rel=src_rel, + dst_rel=dst_rel, + pattern_root=root, + ) + return sorted(items.values(), key=lambda x: x.src.as_posix()) + + +def _split_target(value: str) -> tuple[str, str]: + s = value.strip() + if not s: + return "", "" + if s.startswith("<"): + idx = s.find(">") + if idx > 0: + return s[1:idx], s[idx + 1 :] + parts = s.split(maxsplit=1) + if len(parts) == 1: + return parts[0], "" + return parts[0], " " + parts[1] + + +def _rewrite_links( + text: str, + *, + src: Path, + dst_rel: Path, + src_to_md_dst: dict[Path, Path], + map_src_abs_to_dst_rel, + attachments: set[Path], +) -> str: + def repl(m: re.Match[str]) -> str: + left, raw_target, right = m.group(1), m.group(2), m.group(3) + target, tail = _split_target(raw_target) + if not target: + return m.group(0) + if target.startswith(_URL_SCHEMES) or target.startswith(("#", "/")): + return m.group(0) + + anchor = "" + base = target + if "#" in target: + base, anchor = target.split("#", 1) + anchor = "#" + anchor + if not base: + return m.group(0) + + cand = (src.parent / base).resolve() + dst_md_target = src_to_md_dst.get(cand) + if dst_md_target is None and Path(base).suffix == "": + dst_md_target = src_to_md_dst.get(cand.with_suffix(".md")) + + if dst_md_target is not None: + dst_target_rel = dst_md_target + else: + if not cand.is_file(): + return m.group(0) + dst_target_rel = map_src_abs_to_dst_rel(cand) + if dst_target_rel is None: + return m.group(0) + attachments.add(cand) + + rel = os.path.relpath( + dst_target_rel.as_posix(), start=dst_rel.parent.as_posix() + ) + rel = rel.replace("\\", "/") + return f"{left}{_encode_md_target(rel)}{anchor}{tail}{right}" + + out = _MD_LINK_RE.sub(repl, text) + + def repl_def(m: re.Match[str]) -> str: + prefix = m.group("prefix") + raw_target = m.group("target") + suffix = m.group("suffix") or "" + target = raw_target.strip() + if target.startswith("<") and target.endswith(">"): + target = target[1:-1].strip() + + if not target: + return m.group(0) + if target.startswith(_URL_SCHEMES) or target.startswith(("#", "/")): + return m.group(0) + if "#" in target: + base, anchor = target.split("#", 1) + anchor = "#" + anchor + else: + base, anchor = target, "" + + if not base: + return m.group(0) + + cand = (src.parent / base).resolve() + dst_md_target = src_to_md_dst.get(cand) + if dst_md_target is None and Path(base).suffix == "": + dst_md_target = src_to_md_dst.get(cand.with_suffix(".md")) + + if dst_md_target is not None: + dst_target_rel = dst_md_target + else: + if not cand.is_file(): + return m.group(0) + dst_target_rel = map_src_abs_to_dst_rel(cand) + if dst_target_rel is None: + return m.group(0) + attachments.add(cand) + + rel = os.path.relpath(dst_target_rel.as_posix(), start=dst_rel.parent.as_posix()) + rel = rel.replace("\\", "/") + return f"{prefix}{_encode_md_target(rel)}{anchor}{suffix}" + + return _MD_REF_DEF_RE.sub(repl_def, out) + + +def _manifest_name(folder: str) -> str: + key = folder.strip().strip("/").strip("\\") or "root" + digest = hashlib.sha1(key.encode("utf-8")).hexdigest()[:8] + return f"manifest.export_docs.{digest}.json" + + +def _flatten_rel_path(rel: Path) -> Path: + parts = [p for p in rel.as_posix().split("/") if p and p != "."] + if not parts: + return Path("page.md") + name = "__".join(parts) + return Path(name) + + +def _extract_doc_title(text: str, fallback_stem: str) -> str: + m = re.search(r"^\s*#\s+(.+?)\s*$", text, flags=re.MULTILINE) + if m: + return m.group(1).strip() + fallback = fallback_stem.replace("_", " ").strip() + return fallback or "Untitled" + + +def _target_rel_for(rel: Path, *, flat_root: bool) -> Path: + return _flatten_rel_path(rel) if flat_root else rel + + +def _encode_md_target(target: str) -> str: + # Keep slash + fragment markers, encode spaces/parentheses/unicode safely. + return quote(target, safe="/#._-~") + + +def _merge_marker_block(existing: str, block: str) -> str: + import re + + # Be tolerant: allow whitespace differences around markers. + marker_re = re.compile( + r".*?", + flags=re.DOTALL, + ) + new_block = f"{_EXPORT_DOCS_MARKER_BEGIN}\n{block}\n{_EXPORT_DOCS_MARKER_END}" + if marker_re.search(existing): + return marker_re.sub(new_block, existing) + if existing and not existing.endswith("\n"): + existing += "\n" + if existing and not existing.endswith("\n\n"): + existing += "\n" + return existing + "\n" + new_block + "\n" + + +def _docs_index_rel(folder: str, managed_files: set[str]) -> str | None: + folder_clean = folder.strip().strip("/").strip("\\") + if folder_clean: + cand = f"{folder_clean}/INDEX.md" + else: + cand = "INDEX.md" + if cand in managed_files: + return cand + flat = _flatten_rel_path(Path(cand)).as_posix() + if flat in managed_files: + return flat + return None + + +def ensure_root_navigation( + *, + work_dir: Path, + folder: str, + managed_files: set[str], +) -> None: + index_rel = _docs_index_rel(folder, managed_files) + + home_path = work_dir / "Home.md" + sidebar_path = work_dir / "_Sidebar.md" + + home_text = home_path.read_text(encoding="utf-8", errors="ignore") if home_path.is_file() else "" + sidebar_text = ( + sidebar_path.read_text(encoding="utf-8", errors="ignore") if sidebar_path.is_file() else "" + ) + + if index_rel: + # Gitea wiki page URLs typically omit the `.md` extension: + # file `docs/INDEX.md` → page `/wiki/docs/INDEX`. + link_target = index_rel[:-3] if index_rel.endswith(".md") else index_rel + # For nested wiki paths, INDEX is often addressed as lowercase `index`. + if "/" in link_target: + parts = link_target.split("/") + parts[-1] = parts[-1].lower() + link_target = "/".join(parts) + display = link_target + home_block = "\n".join( + [ + "## Документация (docs)", + "", + f"- [{display}]({_encode_md_target(link_target)})", + ] + ) + sidebar_block = "\n".join( + [ + "### Документация (docs)", + "", + f"- [{display}]({_encode_md_target(link_target)})", + "", + ] + ) + else: + home_block = "\n".join(["## Документация (docs)", "", "_INDEX.md в этой выгрузке не найден._"]) + sidebar_block = "\n".join(["### Документация (docs)", "", "_INDEX.md в этой выгрузке не найден._", ""]) + + home_text = _merge_marker_block(home_text, home_block) + sidebar_text = _merge_marker_block(sidebar_text, sidebar_block) + + write_text(home_path, home_text) + write_text(sidebar_path, sidebar_text) + + +def build_docs_output( + items: list[DocItem], + *, + out_dir: Path, + folder: str, + flat_root: bool = True, + clean: bool = True, +) -> tuple[DocsStats, str]: + out_dir = out_dir.resolve() + if clean and out_dir.exists(): + for child in out_dir.iterdir(): + if child.name == ".git": + continue + if child.is_dir(): + import shutil + + shutil.rmtree(child) + else: + remove_file(child) + ensure_dir(out_dir) + + folder_clean = folder.strip().strip("/").strip("\\") + dst_root = Path(folder_clean) if folder_clean else Path(".") + pattern_roots = sorted({x.pattern_root.resolve() for x in items}) + + # Avoid clobbering export_1c_help navigation pages when publishing into wiki root. + items_ok: list[DocItem] = [] + for item in items: + if ( + not folder_clean + and item.dst_rel.parent == Path(".") + and item.dst_rel.name in _RESERVED_ROOT_PAGES + ): + continue + items_ok.append(item) + + def map_src_abs_to_dst_rel(abs_src: Path) -> Path | None: + abs_src = abs_src.resolve() + best_root: Path | None = None + best_rel: Path | None = None + for r in pattern_roots: + try: + rel = abs_src.relative_to(r) + except ValueError: + continue + if best_root is None or len(r.as_posix()) > len(best_root.as_posix()): + best_root = r + best_rel = rel + if best_root is None or best_rel is None: + return None + return dst_root / best_rel + + md_dst_by_src: dict[Path, Path] = {} + title_by_src: dict[Path, str] = {} + for x in items_ok: + rel = x.dst_rel + rel = _target_rel_for(rel, flat_root=flat_root) + md_dst_by_src[x.src.resolve()] = rel + + src_to_md_dst = md_dst_by_src + attachments: set[Path] = set() + managed_rels: set[Path] = set(src_to_md_dst.values()) + folder_docs: dict[Path, list[tuple[str, Path]]] = {} + for item in items_ok: + raw = item.src.read_text(encoding="utf-8-sig", errors="ignore") + title_by_src[item.src.resolve()] = _extract_doc_title(raw, item.src.stem) + dst_rel = src_to_md_dst[item.src.resolve()] + rendered = _rewrite_links( + raw, + src=item.src, + dst_rel=dst_rel, + src_to_md_dst=src_to_md_dst, + map_src_abs_to_dst_rel=map_src_abs_to_dst_rel, + attachments=attachments, + ) + write_text(out_dir / dst_rel, rendered) + src_folder = item.dst_rel.parent + folder_docs.setdefault(src_folder, []).append( + (title_by_src[item.src.resolve()], dst_rel) + ) + + for abs_src in sorted(attachments, key=lambda p: p.as_posix()): + dst_rel = map_src_abs_to_dst_rel(abs_src) + if dst_rel is None: + continue + dst_rel = _target_rel_for(dst_rel, flat_root=flat_root) + if dst_rel not in managed_rels: + managed_rels.add(dst_rel) + copy_file(abs_src, out_dir / dst_rel) + + # Build per-folder index pages with links to articles and child folders. + all_folders = sorted(folder_docs.keys(), key=lambda p: p.as_posix()) + child_folders: dict[Path, set[Path]] = {} + folder_set = set(all_folders) + for f in all_folders: + parent = f.parent + while parent != Path(".") and parent not in folder_set: + parent = parent.parent + if parent in folder_set and parent != f: + child_folders.setdefault(parent, set()).add(f) + if f.parent == Path(".") and Path(".") in folder_set and f != Path("."): + child_folders.setdefault(Path("."), set()).add(f) + + index_target_by_folder: dict[Path, Path] = {} + for f in all_folders: + idx_rel = (f / "INDEX.md") if f != Path(".") else Path("INDEX.md") + index_target_by_folder[f] = _target_rel_for(idx_rel, flat_root=flat_root) + + for f in all_folders: + lines: list[str] = [] + heading = f.as_posix() if f != Path(".") else (folder_clean or "docs") + lines.append(f"# Индекс: {heading}") + lines.append("") + + docs = sorted(folder_docs.get(f, []), key=lambda x: x[0].casefold()) + if docs: + lines.append("## Статьи") + lines.append("") + for title, target_rel in docs: + rel_link = os.path.relpath( + target_rel.as_posix(), + start=index_target_by_folder[f].parent.as_posix(), + ).replace("\\", "/") + lines.append(f"- [{title}]({_encode_md_target(rel_link)})") + lines.append("") + + children = sorted(child_folders.get(f, set()), key=lambda p: p.as_posix().casefold()) + if children: + lines.append("## Папки") + lines.append("") + for ch in children: + target_rel = index_target_by_folder[ch] + rel_link = os.path.relpath( + target_rel.as_posix(), + start=index_target_by_folder[f].parent.as_posix(), + ).replace("\\", "/") + lines.append(f"- [{ch.as_posix()}]({_encode_md_target(rel_link)})") + lines.append("") + + idx_target = index_target_by_folder[f] + write_text(out_dir / idx_target, "\n".join(lines).rstrip() + "\n") + managed_rels.add(idx_target) + + manifest_name = _manifest_name(folder) + manifest = { + "tool": "export_docs", + "version": __version__, + "folder": folder, + "files": [p.as_posix() for p in sorted(managed_rels, key=lambda x: x.as_posix())], + "sources": [x.src.as_posix() for x in items_ok], + } + write_text(out_dir / manifest_name, json.dumps(manifest, ensure_ascii=False, indent=2) + "\n") + folders: dict[str, list[int]] = {} + markdown_rels = set(src_to_md_dst.values()) | set(index_target_by_folder.values()) + attachment_rels = managed_rels - markdown_rels + for rel in markdown_rels: + key = rel.parent.as_posix() if rel.parent.as_posix() != "." else "/" + folders.setdefault(key, [0, 0])[0] += 1 + for rel in attachment_rels: + key = rel.parent.as_posix() if rel.parent.as_posix() != "." else "/" + folders.setdefault(key, [0, 0])[1] += 1 + return DocsStats( + files_total=len(items_ok) + len(attachments) + len(index_target_by_folder), + files_written=len(managed_rels), + files_deleted=0, + markdown_files=len(markdown_rels), + attachment_files=len(attachment_rels), + folders={k: (v[0], v[1]) for k, v in sorted(folders.items())}, + ), manifest_name + + +def _run_git(cmd: list[str], *, cwd: Path) -> str: + proc = subprocess.run(cmd, cwd=str(cwd), capture_output=True, text=True, check=False) + if proc.returncode != 0: + raise WikiPushError(f"$ {' '.join(cmd)}\n{proc.stdout}\n{proc.stderr}".strip()) + return proc.stdout + + +def _load_prev_managed(work_dir: Path, manifest_name: str) -> set[str]: + p = work_dir / manifest_name + if not p.is_file(): + return set() + try: + data = json.loads(p.read_text(encoding="utf-8")) + except (OSError, json.JSONDecodeError): + return set() + files = data.get("files") or [] + return {str(x) for x in files if isinstance(x, str)} + + +def push_docs_wiki( + *, + wiki_url: str, + out_dir: Path, + work_dir: Path, + message: str, + folder: str, + branch: str = "main", + dry_run: bool = False, +) -> DocsStats: + out_dir = out_dir.resolve() + work_dir = work_dir.resolve() + prepare_wiki_clone(wiki_url, work_dir, branch=branch) + + manifest_name = _manifest_name(folder) + new_manifest = out_dir / manifest_name + if not new_manifest.is_file(): + raise WikiPushError(f"missing manifest in output: {new_manifest}") + new_data = json.loads(new_manifest.read_text(encoding="utf-8")) + new_files = {str(x) for x in (new_data.get("files") or [])} + prev_files = _load_prev_managed(work_dir, manifest_name) + + deleted = 0 + for rel in sorted(prev_files - new_files): + p = work_dir / rel + if p.is_file(): + remove_file(p) + deleted += 1 + + for rel in sorted(new_files): + src = out_dir / rel + if src.is_file(): + copy_file(src, work_dir / rel) + copy_file(new_manifest, work_dir / manifest_name) + + # Update root navigation pages to expose published docs. + ensure_root_navigation( + work_dir=work_dir, + folder=folder, + managed_files=new_files, + ) + + _run_git(["git", "add", "-A"], cwd=work_dir) + status = _run_git(["git", "status", "--porcelain"], cwd=work_dir) + if not status.strip() or dry_run: + return DocsStats( + files_total=len(new_files), + files_written=len(new_files), + files_deleted=deleted, + markdown_files=0, + attachment_files=0, + folders={}, + ) + _run_git(["git", "commit", "-m", message], cwd=work_dir) + _run_git(["git", "push", "origin", f"HEAD:{branch}"], cwd=work_dir) + return DocsStats( + files_total=len(new_files), + files_written=len(new_files), + files_deleted=deleted, + markdown_files=0, + attachment_files=0, + folders={}, + ) + + +def clean_docs_wiki( + *, + wiki_url: str, + work_dir: Path, + folder: str, + message: str, + branch: str = "main", + dry_run: bool = False, +) -> DocsStats: + """ + Remove docs pages previously managed by export_docs for one folder. + + Also removes export_docs marker blocks from Home/_Sidebar. + """ + work_dir = work_dir.resolve() + prepare_wiki_clone(wiki_url, work_dir, branch=branch) + + manifest_name = _manifest_name(folder) + prev_files = _load_prev_managed(work_dir, manifest_name) + deleted = 0 + for rel in sorted(prev_files): + p = work_dir / rel + if p.is_file(): + remove_file(p) + deleted += 1 + + manifest_path = work_dir / manifest_name + if manifest_path.is_file(): + remove_file(manifest_path) + + # Drop export_docs nav block if present. + for name in ("Home.md", "_Sidebar.md"): + p = work_dir / name + if not p.is_file(): + continue + raw = p.read_text(encoding="utf-8", errors="ignore") + cleaned = re.sub( + r".*?\n?", + "", + raw, + flags=re.DOTALL, + ).rstrip() + cleaned = cleaned + "\n" if cleaned else "" + if cleaned != raw: + write_text(p, cleaned) + + _run_git(["git", "add", "-A"], cwd=work_dir) + status = _run_git(["git", "status", "--porcelain"], cwd=work_dir) + if not status.strip() or dry_run: + return DocsStats( + files_total=deleted, + files_written=0, + files_deleted=deleted, + markdown_files=0, + attachment_files=0, + folders={}, + ) + _run_git(["git", "commit", "-m", message], cwd=work_dir) + _run_git(["git", "push", "origin", f"HEAD:{branch}"], cwd=work_dir) + return DocsStats( + files_total=deleted, + files_written=0, + files_deleted=deleted, + markdown_files=0, + attachment_files=0, + folders={}, + ) + + +def validate_folder(folder: str) -> str: + f = folder.strip().strip("\\").strip("/") + if not f: + return "" + p = Path(f) + if any(part in ("..", "") for part in p.parts): + raise ValueError(f"invalid folder path: {folder}") + return p.as_posix() + + +def format_patterns(patterns: Iterable[str]) -> str: + return ", ".join(patterns) + + +def format_folder_summary(stats: DocsStats) -> str: + if not stats.folders: + return " /: 0 md, 0 attachments" + lines: list[str] = [] + for folder, (md_count, attachment_count) in stats.folders.items(): + lines.append(f" {folder}: {md_count} md, {attachment_count} attachments") + return "\n".join(lines) diff --git a/export1c_help/wiki.py b/export1c_help/wiki.py index f4ed440..479e404 100644 --- a/export1c_help/wiki.py +++ b/export1c_help/wiki.py @@ -79,25 +79,42 @@ def prepare_wiki_clone( """Clone or reset wiki working tree.""" work_dir = work_dir.resolve() if work_dir.exists() and (work_dir / ".git").exists(): - _run(["git", "fetch", "origin"], cwd=work_dir) - _run(["git", "checkout", branch], cwd=work_dir) - _run(["git", "reset", "--hard", f"origin/{branch}"], cwd=work_dir) - else: - if work_dir.exists(): + def _norm_git_url(u: str) -> str: + return u.strip().rstrip("/") + + expected = _norm_git_url(wiki_url) + try: + actual = _run( + ["git", "remote", "get-url", "origin"], + cwd=work_dir, + ).strip() + except WikiPushError: + actual = "" + + if _norm_git_url(actual) != expected: + # Existing clone points to another wiki repo; don't reuse it. shutil.rmtree(work_dir) - ensure_dir(work_dir.parent) - _run( - [ - "git", - "clone", - "--branch", - branch, - "--single-branch", - wiki_url, - str(work_dir), - ], - cwd=work_dir.parent, - ) + else: + _run(["git", "fetch", "origin"], cwd=work_dir) + _run(["git", "checkout", branch], cwd=work_dir) + _run(["git", "reset", "--hard", f"origin/{branch}"], cwd=work_dir) + return work_dir + + if work_dir.exists(): + shutil.rmtree(work_dir) + ensure_dir(work_dir.parent) + _run( + [ + "git", + "clone", + "--branch", + branch, + "--single-branch", + wiki_url, + str(work_dir), + ], + cwd=work_dir.parent, + ) return work_dir diff --git a/export_docs.py b/export_docs.py new file mode 100644 index 0000000..2d0ae41 --- /dev/null +++ b/export_docs.py @@ -0,0 +1,253 @@ +#!/usr/bin/env python3 +"""CLI: publish arbitrary markdown docs to repository wiki.""" + +from __future__ import annotations + +import argparse +import re +import sys +from pathlib import Path + +TOOL_ROOT = Path(__file__).resolve().parent +if str(TOOL_ROOT) not in sys.path: + sys.path.insert(0, str(TOOL_ROOT)) + +from export1c_help import __status__, __version__ # noqa: E402 +from export1c_help.docs_export import ( # noqa: E402 + build_docs_output, + clean_docs_wiki, + collect_docs, + format_folder_summary, + format_patterns, + push_docs_wiki, + validate_folder, +) +from export1c_help.resolve import ( # noqa: E402 + ResolveError, + git_remote_url, + git_toplevel, + resolve_src_root, + resolve_wiki_url, + slug_for_path, + to_wiki_clone_url, +) +from export1c_help.wiki import WikiPushError # noqa: E402 + + +def build_parser() -> argparse.ArgumentParser: + p = argparse.ArgumentParser( + prog="export_docs", + description=f"Публикация Markdown-документов в wiki-папки. v{__version__} ({__status__}).", + ) + p.add_argument("--version", action="version", version=f"%(prog)s {__version__}") + sub = p.add_subparsers(dest="cmd", required=True) + + push = sub.add_parser("push", help="Собрать Markdown по маскам и выгрузить в wiki") + _add_source_args(push) + push.add_argument( + "--files", + nargs="+", + required=True, + help="Глоб-маски исходных файлов (экспортируются только .md)", + ) + push.add_argument( + "--folder", + default="", + help="Папка назначения в wiki. Пусто = корень wiki (например docs или migration/docs)", + ) + push.add_argument("--wiki-url", default=None, help="Явный URL wiki git (иначе из remote источника)") + push.add_argument("--remote", default="origin", help="Имя git remote (по умолчанию origin)") + push.add_argument("--work-dir", type=Path, default=None, help="Локальный clone wiki") + push.add_argument("-o", "--out", type=Path, default=None, help="Промежуточный каталог сборки") + push.add_argument("--branch", default="main") + push.add_argument("-m", "--message", default=None, help="Сообщение коммита") + push.add_argument("--dry-run", action="store_true", help="Без git push") + push.add_argument("--verbose", action="store_true", help="Печатать полный список входных файлов") + push.add_argument( + "--no-flat-root", + action="store_true", + help="Не преобразовывать пути в root-файлы (оставлять иерархию каталогов wiki)", + ) + push.set_defaults(func=cmd_push) + + clean = sub.add_parser( + "clean", + help="Удалить из wiki статьи, опубликованные export_docs (по --folder/manifest, без --files)", + ) + _add_source_args(clean) + clean.add_argument( + "--folder", + default="", + help="Папка назначения в wiki, которую чистим. Пусто = root-набор export_docs", + ) + clean.add_argument("--wiki-url", default=None, help="Явный URL wiki git (иначе из remote источника)") + clean.add_argument("--remote", default="origin", help="Имя git remote (по умолчанию origin)") + clean.add_argument("--work-dir", type=Path, default=None, help="Локальный clone wiki") + clean.add_argument("--branch", default="main") + clean.add_argument("-m", "--message", default=None, help="Сообщение коммита") + clean.add_argument("--dry-run", action="store_true", help="Без git push") + clean.set_defaults(func=cmd_clean) + return p + + +def _add_source_args(p: argparse.ArgumentParser) -> None: + # Если -s/-c не указаны, wiki URL выводим из git remote текущей папки. + g = p.add_mutually_exclusive_group(required=False) + g.add_argument("-c", "--config", type=Path, help="Корень выгрузки конфигурации или src/") + g.add_argument("-s", "--src", type=Path, help="Путь к src/") + + +def _src_input(args: argparse.Namespace) -> Path: + return args.config if args.config is not None else args.src + + +def cmd_push(args: argparse.Namespace) -> int: + try: + src_input = _src_input(args) + if src_input is not None: + src = resolve_src_root(src_input) + wiki_url = resolve_wiki_url(src, wiki_url=args.wiki_url, remote=args.remote) + slug = slug_for_path(src) + else: + top = git_toplevel(Path.cwd()) + if top is None: + raise ResolveError("cannot detect git repository for current directory") + repo_url = git_remote_url(top, remote=args.remote) + if not repo_url: + raise ResolveError(f"cannot detect git remote '{args.remote}' for {top}") + wiki_url = to_wiki_clone_url(args.wiki_url or repo_url) + src = top + slug = re.sub(r"[^\w.\-]+", "-", top.name, flags=re.UNICODE).strip("-") or "repo" + folder = validate_folder(args.folder) + except (ResolveError, ValueError) as exc: + print(f"error: {exc}", file=sys.stderr) + return 1 + out = (args.out or (TOOL_ROOT / "out" / f"wiki-docs-md-{slug}")).resolve() + work = (args.work_dir or (TOOL_ROOT / "out" / f"wiki_clone-{slug}")).resolve() + + print(f"wiki_url: {wiki_url}") + print(f"src: {src}") + print(f"folder: {folder or '.'}") + if args.verbose: + print(f"files: {format_patterns(args.files)}") + else: + print(f"inputs: {len(args.files)} path args") + print(f"out: {out}") + print(f"work: {work}") + + try: + items = collect_docs(args.files, folder=folder) + except ValueError as exc: + print(f"error: {exc}", file=sys.stderr) + return 1 + if not items: + print("error: no markdown files matched --files patterns", file=sys.stderr) + return 1 + + build_stats, _manifest_name = build_docs_output( + items, + out_dir=out, + folder=folder, + flat_root=not args.no_flat_root, + clean=True, + ) + print( + f"collect: md={build_stats.markdown_files}, attachments={build_stats.attachment_files}, " + f"prepared={build_stats.files_written} → {out}" + ) + print("folders:") + print(format_folder_summary(build_stats)) + + msg = args.message or ( + f"export_docs v{__version__}: sync docs to {folder or '/'} " + f"({build_stats.files_written} files)" + ) + try: + push_stats = push_docs_wiki( + wiki_url=wiki_url, + out_dir=out, + work_dir=work, + message=msg, + folder=folder, + branch=args.branch, + dry_run=args.dry_run, + ) + except WikiPushError as exc: + print(f"push failed:\n{exc}", file=sys.stderr) + return 1 + + if args.dry_run: + print( + f"dry-run: files={push_stats.files_written}, deleted={push_stats.files_deleted}; push skipped" + ) + return 0 + print( + f"OK: pushed docs files={push_stats.files_written}, deleted={push_stats.files_deleted} → {wiki_url}" + ) + return 0 + + +def _resolve_wiki_context(args: argparse.Namespace) -> tuple[Path, str, str]: + src_input = _src_input(args) + if src_input is not None: + src = resolve_src_root(src_input) + wiki_url = resolve_wiki_url(src, wiki_url=args.wiki_url, remote=args.remote) + slug = slug_for_path(src) + else: + top = git_toplevel(Path.cwd()) + if top is None: + raise ResolveError("cannot detect git repository for current directory") + repo_url = git_remote_url(top, remote=args.remote) + if not repo_url: + raise ResolveError(f"cannot detect git remote '{args.remote}' for {top}") + wiki_url = to_wiki_clone_url(args.wiki_url or repo_url) + src = top + slug = re.sub(r"[^\w.\-]+", "-", top.name, flags=re.UNICODE).strip("-") or "repo" + return src, wiki_url, slug + + +def cmd_clean(args: argparse.Namespace) -> int: + try: + src, wiki_url, slug = _resolve_wiki_context(args) + folder = validate_folder(args.folder) + except (ResolveError, ValueError) as exc: + print(f"error: {exc}", file=sys.stderr) + return 1 + + work = (args.work_dir or (TOOL_ROOT / "out" / f"wiki_clone-{slug}")).resolve() + print(f"wiki_url: {wiki_url}") + print(f"src: {src}") + print(f"folder: {folder or '.'}") + print(f"work: {work}") + + msg = args.message or ( + f"export_docs v{__version__}: cleanup docs in {folder or '/'}" + ) + try: + stats = clean_docs_wiki( + wiki_url=wiki_url, + work_dir=work, + folder=folder, + message=msg, + branch=args.branch, + dry_run=args.dry_run, + ) + except WikiPushError as exc: + print(f"clean failed:\n{exc}", file=sys.stderr) + return 1 + + if args.dry_run: + print(f"dry-run: would delete={stats.files_deleted}; push skipped") + return 0 + print(f"OK: cleaned docs files deleted={stats.files_deleted} → {wiki_url}") + return 0 + + +def main(argv: list[str] | None = None) -> int: + parser = build_parser() + args = parser.parse_args(argv) + return args.func(args) + + +if __name__ == "__main__": + raise SystemExit(main())