Track configuration version and page lifecycle in wiki export.

Record created/edited/deleted-in versions per page; add History.md and update workflow docs.
This commit is contained in:
mihailkudravcev
2026-07-23 15:16:06 +03:00
parent fa6a2a3799
commit ecddcef586
8 changed files with 672 additions and 168 deletions
+18
View File
@@ -2,6 +2,23 @@
All notable changes to **export_1c_help** are documented in this file. All notable changes to **export_1c_help** are documented in this file.
## [0.2.0] - 2026-07-23
### Added
- Configuration version on each export (`VERSION` / `Configuration.xml`)
- Per-page lifecycle in wiki footer and `manifest.json`:
- `created_in` — версия конфигурации, в которой страница впервые попала в wiki
- `edited_in` — версия, в которой содержимое справки изменилось
- `deleted_in` — версия, в которой страница исчезла из `src`
- `History.md` — сводка выгрузки и накопительный список удалённых страниц
- `push` читает предыдущий `manifest.json` из wiki перед пересборкой
- CLI: `--prev-manifest`, `--config-version`
### Changed
- Home / Sidebar показывают версию конфигурации и ссылку на History
## [0.1.0] - 2026-07-23 ## [0.1.0] - 2026-07-23
### Status ### Status
@@ -25,4 +42,5 @@ All notable changes to **export_1c_help** are documented in this file.
- Nested subsystem help paths (`Subsystems/A/Subsystems/B/…`) no longer collapse to the parent meta key - Nested subsystem help paths (`Subsystems/A/Subsystems/B/…`) no longer collapse to the parent meta key
- Unicode wiki filenames replaced with percent-encoded slugs (Gitea UI returned HTTP 500) - Unicode wiki filenames replaced with percent-encoded slugs (Gitea UI returned HTTP 500)
[0.2.0]: https://git.p7net.ru/tools/export_1c_help/releases/tag/v0.2.0
[0.1.0]: https://git.p7net.ru/tools/export_1c_help/releases/tag/v0.1.0 [0.1.0]: https://git.p7net.ru/tools/export_1c_help/releases/tag/v0.1.0
+56 -48
View File
@@ -1,6 +1,6 @@
# export_1c_help # export_1c_help
**Версия:** см. [`VERSION`](VERSION) (текущая: **0.1.0**) · [CHANGELOG](CHANGELOG.md) **Версия:** см. [`VERSION`](VERSION) (текущая: **0.2.0**) · [CHANGELOG](CHANGELOG.md)
**Лицензия:** [MIT](LICENSE) **Лицензия:** [MIT](LICENSE)
Выгрузка встроенной справки конфигурации 1С (`**/Ext/Help/ru.html`) в Markdown и публикация в [Gitea Wiki](https://docs.gitea.com/usage/wiki). Выгрузка встроенной справки конфигурации 1С (`**/Ext/Help/ru.html`) в Markdown и публикация в [Gitea Wiki](https://docs.gitea.com/usage/wiki).
@@ -9,26 +9,66 @@
--- ---
## Порядок обновления wiki (после доработки справки в конфигурации)
Инструмент **всегда** пересобирает wiki из всего `src/` (инкрементального патча нет). Частичная правка Help в конфигураторе → полная перевыгрузка `src` → полный `push`.
1. Доработать справку в конфигураторе (нужные объекты/формы).
2. Выгрузить конфигурацию в git (`crm3-26/`) и закоммитить/запушить, как обычно.
3. В монорепо: `git pull` (актуальный `crm3-26/src`, в т.ч. `VERSION` / `Configuration.xml`).
4. Синхронизировать wiki:
```bash
cd tools/export_1c_help
python3 export_1c_help.py push \
-s ../../crm3-26/src \
--wiki-url https://git.p7net.ru/1c/crm3_26.wiki.git \
-m "sync help after config update"
```
Перед записью можно проверить без push:
```bash
python3 export_1c_help.py push \
-s ../../crm3-26/src \
--wiki-url https://git.p7net.ru/1c/crm3_26.wiki.git \
--dry-run
```
**Канон текста** — HTML в конфигурации. Ручные правки страниц в wiki сотрутся при следующем `push` (если не указан `--keep-unmanaged`).
---
## Версии конфигурации на страницах
| Где | Что фиксируется |
|-----|-----------------|
| `Home.md` | версия конфигурации **этой** выгрузки |
| подвал каждой страницы | версия выгрузки; **создана в** / **изменена в** |
| `History.md` | сводка +/−/~ и накопительный список **удалённых** (`deleted_in`) |
| `manifest.json` | те же поля + `content_hash`, история `exports[]` |
Версия берётся из `<config>/VERSION` или `<Version>` в `Configuration.xml` (можно переопределить `--config-version`).
Жизненный цикл считается относительно предыдущего `manifest.json` в wiki (при `push` подтягивается автоматически).
---
## Быстрый старт ## Быстрый старт
```bash ```bash
cd tools/export_1c_help cd tools/export_1c_help
# конвертация в каталог
python3 export_1c_help.py convert \ python3 export_1c_help.py convert \
-s ../../crm3-26/src \ -s ../../crm3-26/src \
-o ./out/wiki-md \ -o ./out/wiki-md \
--clean --clean
# конвертация + push в wiki-репозиторий Gitea
python3 export_1c_help.py push \ python3 export_1c_help.py push \
-s ../../crm3-26/src \ -s ../../crm3-26/src \
--wiki-url https://git.p7net.ru/1c/crm3_26.wiki.git \ --wiki-url https://git.p7net.ru/1c/crm3_26.wiki.git
-o ./out/wiki-md
``` ```
Проверка версии:
```bash ```bash
python3 export_1c_help.py --version python3 export_1c_help.py --version
``` ```
@@ -39,62 +79,30 @@ python3 export_1c_help.py --version
### `convert` ### `convert`
Читает `src/`, пишет Markdown-дерево wiki:
| Файл | Назначение |
|------|------------|
| `Home.md` | оглавление по типам метаданных |
| `_Sidebar.md` | боковая навигация (объекты без форм) |
| `«Заголовок».md` / `%D0%….md` | страницы справки (кириллица в git — **percent-encoded**, как хранит Gitea) |
| `manifest.json` | карта `meta_key` → title / filename |
| Параметр | Описание | | Параметр | Описание |
|----------|----------| |----------|----------|
| `-s` / `--src` | каталог `src` выгрузки конфигурации | | `-s` / `--src` | каталог `src` выгрузки конфигурации |
| `-o` / `--out` | каталог результата | | `-o` / `--out` | каталог результата |
| `--objects-only` | без справки форм (`Forms/*/Ext/Help`) | | `--objects-only` | без справки форм |
| `--keep-empty` | не пропускать почти пустые страницы | | `--keep-empty` | не пропускать почти пустые страницы |
| `--clean` | очистить `--out` перед записью (кроме `.git`) | | `--clean` | очистить `--out` перед записью |
| `--prev-manifest` | предыдущий `manifest.json` (lifecycle) |
| `--config-version` | явная версия конфигурации |
### `push` ### `push`
Выполняет `convert`, затем `git clone` / обновление wiki-репозитория (обычно `*.wiki.git`), коммит и `git push`. `prepare wiki``convert` (с `manifest.json` из wiki) → `git commit` / `push`.
| Параметр | Описание | | Параметр | Описание |
|----------|----------| |----------|----------|
| `--wiki-url` | URL wiki git, напр. `https://git.p7net.ru/1c/crm3_26.wiki.git` | | `--wiki-url` | URL `*.wiki.git` |
| `--work-dir` | локальный clone wiki |
| `--dry-run` | только convert | | `--dry-run` | только convert |
| `--keep-unmanaged` | не удалять чужие `.md` в wiki | | `--keep-unmanaged` | не удалять чужие `.md` |
| `-m` / `--message` | сообщение коммита | | `-m` / `--message` | сообщение коммита |
Для `push` нужны права записи в wiki (HTTPS + credential helper / token).
---
## Как устроены ссылки
В HTML справки 1С:
```text
Catalog.ВидыНоменклатуры/Help
Document.ЗаказКлиента.Form.ФормаДокумента/Help
```
В wiki:
```markdown
[[Виды номенклатуры]]
[[текст|Виды номенклатуры]]
```
Ключ метаданных (`Catalog.ВидыНоменклатуры`) стабилен и пишется в подвал страницы и в `manifest.json`.
--- ---
## Репозиторий ## Репозиторий
Отдельный git-репозиторий (подмодуль монорепозитория миграции): - путь: `tools/export_1c_help/`
- remote: https://git.p7net.ru/tools/export_1c_help.git
- путь в дереве проекта: `tools/export_1c_help/`
- remote (ожидаемый): `https://git.p7net.ru/tools/export_1c_help.git`
+1 -1
View File
@@ -1 +1 @@
0.1.0 0.2.0
+64
View File
@@ -0,0 +1,64 @@
"""Read 1C configuration version from dump tree."""
from __future__ import annotations
import re
from pathlib import Path
_VERSION_TAG = re.compile(r"<Version>([^<]+)</Version>")
_NAME_TAG = re.compile(
r"<Name>([^<]+)</Name>",
)
def config_root_from_src(src_root: Path) -> Path:
"""src/ → configuration root (parent that usually has VERSION + Configuration.xml)."""
src_root = Path(src_root).resolve()
if src_root.name == "src":
return src_root.parent
return src_root
def read_config_version(src_root: Path) -> str:
"""
Resolve configuration version.
Priority:
1. <config_root>/VERSION
2. <config_root>/src/Configuration.xml → <Version>
3. <src_root>/Configuration.xml → <Version>
"""
src_root = Path(src_root)
root = config_root_from_src(src_root)
version_file = root / "VERSION"
if version_file.is_file():
text = version_file.read_text(encoding="utf-8-sig", errors="ignore").strip()
if text:
return text.splitlines()[0].strip()
for xml_path in (
root / "src" / "Configuration.xml",
src_root / "Configuration.xml",
root / "Configuration.xml",
):
if not xml_path.is_file():
continue
raw = xml_path.read_text(encoding="utf-8-sig", errors="ignore")
m = _VERSION_TAG.search(raw)
if m:
return m.group(1).strip()
return "unknown"
def read_config_name(src_root: Path) -> str | None:
src_root = Path(src_root)
root = config_root_from_src(src_root)
xml_path = root / "src" / "Configuration.xml"
if not xml_path.is_file():
xml_path = src_root / "Configuration.xml"
if not xml_path.is_file():
return None
raw = xml_path.read_text(encoding="utf-8-sig", errors="ignore")
m = _NAME_TAG.search(raw)
return m.group(1).strip() if m else None
+228 -36
View File
@@ -3,16 +3,26 @@
from __future__ import annotations from __future__ import annotations
import json import json
import re
import shutil import shutil
from dataclasses import dataclass from dataclasses import dataclass
from datetime import datetime, timezone from datetime import datetime, timezone
from pathlib import Path from pathlib import Path
from export1c_help.__version__ import __version__
from export1c_help.config_version import read_config_name, read_config_version
from export1c_help.convert import default_link_rewrite, extract_h1, html_to_markdown from export1c_help.convert import default_link_rewrite, extract_h1, html_to_markdown
from export1c_help.discover import HelpPage, build_link_title_map, discover_help_pages from export1c_help.discover import HelpPage, build_link_title_map, discover_help_pages
from export1c_help.lifecycle import (
LifecycleResult,
PageRecord,
content_hash,
load_prev_manifest,
merge_lifecycle,
prev_records,
)
from export1c_help.naming import assign_titles from export1c_help.naming import assign_titles
from export1c_help.types_map import DIR_TO_RU from export1c_help.types_map import DIR_TO_RU
from export1c_help.__version__ import __version__
@dataclass @dataclass
@@ -21,6 +31,11 @@ class ExportStats:
pages_written: int = 0 pages_written: int = 0
pages_skipped_empty: int = 0 pages_skipped_empty: int = 0
images_copied: int = 0 images_copied: int = 0
created_count: int = 0
edited_count: int = 0
unchanged_count: int = 0
deleted_count: int = 0
config_version: str = "unknown"
def _is_effectively_empty(md: str) -> bool: def _is_effectively_empty(md: str) -> bool:
@@ -29,8 +44,6 @@ def _is_effectively_empty(md: str) -> bool:
def re_sub_md(md: str) -> str: def re_sub_md(md: str) -> str:
import re
t = re.sub(r"\[\[([^\]|]+)(?:\|[^\]]+)?\]\]", r"\1", md) t = re.sub(r"\[\[([^\]|]+)(?:\|[^\]]+)?\]\]", r"\1", md)
t = re.sub(r"\[([^\]]+)\]\([^)]+\)", r"\1", t) t = re.sub(r"\[([^\]]+)\]\([^)]+\)", r"\1", t)
t = re.sub(r"[#*_`|-]", " ", t) t = re.sub(r"[#*_`|-]", " ", t)
@@ -38,18 +51,43 @@ def re_sub_md(md: str) -> str:
return t return t
def build_home(pages: list[HelpPage], titles: dict[str, str], *, source_label: str) -> str: def _page_footer(rec: PageRecord, *, config_version: str) -> str:
lines = [
"",
"---",
"",
f"*Конфигурация выгрузки: **{config_version}*** ",
f"*Страница создана в: **{rec.created_in}** · изменена в: **{rec.edited_in}*** ",
f"*Мета: `{rec.meta_key}` · файл: `{rec.source}`*",
"",
]
return "\n".join(lines)
def build_home(
pages: list[HelpPage],
titles: dict[str, str],
*,
source_label: str,
config_version: str,
config_name: str | None,
life: LifecycleResult,
) -> str:
by_type: dict[str, list[HelpPage]] = {} by_type: dict[str, list[HelpPage]] = {}
for p in pages: for p in pages:
by_type.setdefault(p.type_dir, []).append(p) by_type.setdefault(p.type_dir, []).append(p)
name_line = f" (`{config_name}`)" if config_name else ""
lines = [ lines = [
f"# Справка конфигурации", "# Справка конфигурации",
"", "",
f"Автогенерация из встроенной справки 1С (`Ext/Help/ru.html`).", "Автогенерация из встроенной справки 1С (`Ext/Help/ru.html`).",
"", "",
f"- **Версия конфигурации (эта выгрузка):** `{config_version}`{name_line}",
f"- Источник: `{source_label}`", f"- Источник: `{source_label}`",
f"- Страниц: **{len(pages)}**", f"- Страниц: **{len(pages)}** "
f"(+{life.created_count} / ~{life.edited_count} / ={life.unchanged_count} / {life.deleted_count})",
f"- Удалённых (накопительно): **{len(life.deleted)}** — см. [[History]]",
f"- Инструмент: `export_1c_help` v{__version__}", f"- Инструмент: `export_1c_help` v{__version__}",
f"- Дата: {datetime.now(timezone.utc).strftime('%Y-%m-%d %H:%M UTC')}", f"- Дата: {datetime.now(timezone.utc).strftime('%Y-%m-%d %H:%M UTC')}",
"", "",
@@ -72,12 +110,17 @@ def build_sidebar(pages: list[HelpPage], titles: dict[str, str]) -> str:
by_type: dict[str, list[HelpPage]] = {} by_type: dict[str, list[HelpPage]] = {}
for p in pages: for p in pages:
by_type.setdefault(p.type_dir, []).append(p) by_type.setdefault(p.type_dir, []).append(p)
lines = ["# Справка", "", "[[Home|Оглавление]]", ""] lines = [
"# Справка",
"",
"[[Home|Оглавление]]",
"[[History|История выгрузок]]",
"",
]
for type_dir in sorted(by_type, key=lambda d: DIR_TO_RU.get(d, d)): for type_dir in sorted(by_type, key=lambda d: DIR_TO_RU.get(d, d)):
label = DIR_TO_RU.get(type_dir, type_dir) label = DIR_TO_RU.get(type_dir, type_dir)
lines.append(f"### {label}") lines.append(f"### {label}")
lines.append("") lines.append("")
# only object-level in sidebar to keep it usable
objs = [p for p in by_type[type_dir] if not p.is_form] objs = [p for p in by_type[type_dir] if not p.is_form]
objs.sort(key=lambda p: titles[p.meta_key].casefold()) objs.sort(key=lambda p: titles[p.meta_key].casefold())
for p in objs[:80]: for p in objs[:80]:
@@ -88,6 +131,71 @@ def build_sidebar(pages: list[HelpPage], titles: dict[str, str]) -> str:
return "\n".join(lines).rstrip() + "\n" return "\n".join(lines).rstrip() + "\n"
def build_history(
*,
config_version: str,
config_name: str | None,
life: LifecycleResult,
prev_exports: list[dict],
) -> str:
name = f" `{config_name}`" if config_name else ""
lines = [
"# История выгрузок справки",
"",
f"Текущая выгрузка: конфигурация **{config_version}**{name}.",
"",
"## Эта выгрузка",
"",
f"- создано страниц: **{life.created_count}**",
f"- изменено: **{life.edited_count}**",
f"- без изменений: **{life.unchanged_count}**",
f"- удалено в этой версии: **{life.deleted_count}**",
f"- всего удалённых (накопительно): **{len(life.deleted)}**",
"",
]
if life.deleted_count:
lines.append("### Удалено в этой версии")
lines.append("")
for rec in life.deleted:
if rec.deleted_in != config_version:
continue
lines.append(
f"- `{rec.meta_key}` — «{rec.title}» "
f"(создана в {rec.created_in}, последняя правка {rec.edited_in})"
)
lines.append("")
if life.deleted:
lines.append("### Все удалённые страницы")
lines.append("")
lines.append("| Мета | Заголовок | Создана | Изменена | Удалена |")
lines.append("|------|-----------|---------|----------|---------|")
for rec in life.deleted:
lines.append(
f"| `{rec.meta_key}` | {rec.title} | {rec.created_in} | "
f"{rec.edited_in} | {rec.deleted_in or ''} |"
)
lines.append("")
lines.append("## Предыдущие выгрузки")
lines.append("")
if not prev_exports:
lines.append("_Нет сохранённой истории (первая выгрузка с учётом версий)._")
lines.append("")
else:
lines.append("| Версия конфигурации | Дата (UTC) | Страниц | + | ~ | − |")
lines.append("|---------------------|------------|---------|---|---|---|")
for row in prev_exports:
lines.append(
f"| {row.get('config_version', '?')} | {row.get('exported_at', '?')} | "
f"{row.get('pages_written', '?')} | {row.get('created', '?')} | "
f"{row.get('edited', '?')} | {row.get('deleted', '?')} |"
)
lines.append("")
return "\n".join(lines).rstrip() + "\n"
def export_help( def export_help(
src_root: Path, src_root: Path,
out_dir: Path, out_dir: Path,
@@ -96,12 +204,23 @@ def export_help(
skip_empty: bool = True, skip_empty: bool = True,
clean: bool = False, clean: bool = False,
source_label: str | None = None, source_label: str | None = None,
prev_manifest_path: Path | None = None,
config_version: str | None = None,
) -> ExportStats: ) -> ExportStats:
"""Convert all help pages into a wiki working tree at out_dir.""" """Convert all help pages into a wiki working tree at out_dir."""
src_root = src_root.resolve() src_root = src_root.resolve()
out_dir = out_dir.resolve() out_dir = out_dir.resolve()
stats = ExportStats() stats = ExportStats()
cfg_ver = (config_version or read_config_version(src_root)).strip() or "unknown"
cfg_name = read_config_name(src_root)
stats.config_version = cfg_ver
prev_manifest = load_prev_manifest(prev_manifest_path)
if prev_manifest is None and not clean:
prev_manifest = load_prev_manifest(out_dir / "manifest.json")
previous = prev_records(prev_manifest)
pages = discover_help_pages(src_root, include_forms=include_forms) pages = discover_help_pages(src_root, include_forms=include_forms)
stats.pages_total = len(pages) stats.pages_total = len(pages)
@@ -128,7 +247,8 @@ def export_help(
def rewrite(href: str, text: str) -> str: def rewrite(href: str, text: str) -> str:
return default_link_rewrite(href, text, link_map) return default_link_rewrite(href, text, link_map)
manifest_pages: list[dict] = [] draft_records: list[PageRecord] = []
body_by_meta: dict[str, str] = {}
for p in pages: for p in pages:
html = html_cache[p.meta_key] html = html_cache[p.meta_key]
@@ -137,20 +257,41 @@ def export_help(
if skip_empty and _is_effectively_empty(md): if skip_empty and _is_effectively_empty(md):
stats.pages_skipped_empty += 1 stats.pages_skipped_empty += 1
continue continue
body = md.rstrip() + "\n"
# footer body_by_meta[p.meta_key] = body
md = ( draft_records.append(
md.rstrip() PageRecord(
+ "\n\n---\n\n" meta_key=p.meta_key,
+ f"*Мета: `{p.meta_key}` · файл: `{p.rel_posix}`*\n" title=title,
filename=files[p.meta_key],
type_dir=p.type_dir,
object=p.object_name,
form=p.form_name,
source=p.rel_posix,
content_hash=content_hash(html),
created_in="",
edited_in="",
)
) )
fname = files[p.meta_key] life = merge_lifecycle(
(out_dir / fname).write_text(md, encoding="utf-8") current=draft_records,
previous=previous,
config_version=cfg_ver,
)
stats.created_count = life.created_count
stats.edited_count = life.edited_count
stats.unchanged_count = life.unchanged_count
stats.deleted_count = life.deleted_count
for rec in life.active:
body = body_by_meta[rec.meta_key]
md = body.rstrip() + _page_footer(rec, config_version=cfg_ver)
(out_dir / rec.filename).write_text(md, encoding="utf-8")
stats.pages_written += 1 stats.pages_written += 1
# copy sibling images help_dir = src_root / Path(rec.source).parent
help_dir = p.path.parent if help_dir.is_dir():
for img in help_dir.iterdir(): for img in help_dir.iterdir():
if img.suffix.lower() in {".png", ".jpg", ".jpeg", ".gif", ".bmp", ".webp"}: if img.suffix.lower() in {".png", ".jpg", ".jpeg", ".gif", ".bmp", ".webp"}:
target = out_dir / img.name target = out_dir / img.name
@@ -158,39 +299,90 @@ def export_help(
shutil.copy2(img, target) shutil.copy2(img, target)
stats.images_copied += 1 stats.images_copied += 1
manifest_pages.append(
{
"meta_key": p.meta_key,
"title": title,
"filename": fname,
"type_dir": p.type_dir,
"object": p.object_name,
"form": p.form_name,
"source": p.rel_posix,
}
)
label = source_label or str(src_root) label = source_label or str(src_root)
written_keys = {m["meta_key"] for m in manifest_pages} written_keys = {r.meta_key for r in life.active}
written_pages = [p for p in pages if p.meta_key in written_keys] written_pages = [p for p in pages if p.meta_key in written_keys]
prev_exports: list[dict] = []
if prev_manifest:
prev_exports = list(prev_manifest.get("exports") or [])
# append previous export summary if present at top level
if prev_manifest.get("config_version"):
prev_exports.insert(
0,
{
"config_version": prev_manifest.get("config_version"),
"exported_at": prev_manifest.get("exported_at"),
"pages_written": prev_manifest.get("pages_written"),
"created": prev_manifest.get("created_count"),
"edited": prev_manifest.get("edited_count"),
"deleted": prev_manifest.get("deleted_count"),
},
)
# dedupe consecutive identical
deduped: list[dict] = []
for row in prev_exports:
if deduped and deduped[-1].get("exported_at") == row.get("exported_at"):
continue
deduped.append(row)
prev_exports = deduped[:50]
(out_dir / "Home.md").write_text( (out_dir / "Home.md").write_text(
build_home(written_pages, titles, source_label=label), build_home(
written_pages,
titles,
source_label=label,
config_version=cfg_ver,
config_name=cfg_name,
life=life,
),
encoding="utf-8", encoding="utf-8",
) )
(out_dir / "_Sidebar.md").write_text( (out_dir / "_Sidebar.md").write_text(
build_sidebar(written_pages, titles), build_sidebar(written_pages, titles),
encoding="utf-8", encoding="utf-8",
) )
(out_dir / "History.md").write_text(
build_history(
config_version=cfg_ver,
config_name=cfg_name,
life=life,
prev_exports=prev_exports,
),
encoding="utf-8",
)
exported_at = datetime.now(timezone.utc).isoformat()
exports = [
{
"config_version": cfg_ver,
"exported_at": exported_at,
"pages_written": stats.pages_written,
"created": stats.created_count,
"edited": stats.edited_count,
"deleted": stats.deleted_count,
"tool_version": __version__,
},
*prev_exports,
][:50]
manifest = { manifest = {
"tool": "export_1c_help", "tool": "export_1c_help",
"version": __version__, "version": __version__,
"config_version": cfg_ver,
"config_name": cfg_name,
"source": label, "source": label,
"exported_at": datetime.now(timezone.utc).isoformat(), "exported_at": exported_at,
"pages_total": stats.pages_total, "pages_total": stats.pages_total,
"pages_written": stats.pages_written, "pages_written": stats.pages_written,
"pages_skipped_empty": stats.pages_skipped_empty, "pages_skipped_empty": stats.pages_skipped_empty,
"pages": manifest_pages, "created_count": stats.created_count,
"edited_count": stats.edited_count,
"unchanged_count": stats.unchanged_count,
"deleted_count": stats.deleted_count,
"exports": exports,
"pages": [r.to_dict() for r in life.active],
"deleted_pages": [r.to_dict() for r in life.deleted],
} }
(out_dir / "manifest.json").write_text( (out_dir / "manifest.json").write_text(
json.dumps(manifest, ensure_ascii=False, indent=2) + "\n", json.dumps(manifest, ensure_ascii=False, indent=2) + "\n",
+174
View File
@@ -0,0 +1,174 @@
"""Page lifecycle tracking across wiki exports."""
from __future__ import annotations
import hashlib
import json
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
@dataclass
class PageRecord:
meta_key: str
title: str
filename: str
type_dir: str
object: str
form: str | None
source: str
content_hash: str
created_in: str
edited_in: str
deleted_in: str | None = None
status: str = "active" # active | deleted
def to_dict(self) -> dict[str, Any]:
d: dict[str, Any] = {
"meta_key": self.meta_key,
"title": self.title,
"filename": self.filename,
"type_dir": self.type_dir,
"object": self.object,
"form": self.form,
"source": self.source,
"content_hash": self.content_hash,
"created_in": self.created_in,
"edited_in": self.edited_in,
"status": self.status,
}
if self.deleted_in:
d["deleted_in"] = self.deleted_in
return d
@classmethod
def from_dict(cls, d: dict[str, Any]) -> PageRecord:
return cls(
meta_key=d["meta_key"],
title=d.get("title") or d["meta_key"],
filename=d.get("filename") or "",
type_dir=d.get("type_dir") or "",
object=d.get("object") or "",
form=d.get("form"),
source=d.get("source") or "",
content_hash=d.get("content_hash") or "",
created_in=d.get("created_in") or d.get("config_version") or "unknown",
edited_in=d.get("edited_in") or d.get("created_in") or "unknown",
deleted_in=d.get("deleted_in"),
status=d.get("status") or ("deleted" if d.get("deleted_in") else "active"),
)
@dataclass
class LifecycleResult:
active: list[PageRecord] = field(default_factory=list)
deleted: list[PageRecord] = field(default_factory=list)
created_count: int = 0
edited_count: int = 0
unchanged_count: int = 0
deleted_count: int = 0
def content_hash(text: str) -> str:
return hashlib.sha256(text.encode("utf-8")).hexdigest()[:16]
def load_prev_manifest(path: Path | None) -> dict[str, Any] | None:
if path is None or not path.is_file():
return None
try:
return json.loads(path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
return None
def prev_records(manifest: dict[str, Any] | None) -> dict[str, PageRecord]:
if not manifest:
return {}
out: dict[str, PageRecord] = {}
for key in ("pages", "deleted_pages"):
for item in manifest.get(key) or []:
if not isinstance(item, dict) or "meta_key" not in item:
continue
rec = PageRecord.from_dict(item)
# prefer active over deleted if both present
if rec.meta_key in out and out[rec.meta_key].status == "active":
continue
out[rec.meta_key] = rec
return out
def merge_lifecycle(
*,
current: list[PageRecord],
previous: dict[str, PageRecord],
config_version: str,
) -> LifecycleResult:
"""
current — new active pages (created_in/edited_in placeholders ignored;
content_hash must be set).
"""
result = LifecycleResult()
current_keys = {r.meta_key for r in current}
for cur in current:
old = previous.get(cur.meta_key)
if old is None or old.status == "deleted":
cur.created_in = config_version
cur.edited_in = config_version
cur.deleted_in = None
cur.status = "active"
result.created_count += 1
elif not old.content_hash:
# legacy manifest without hashes — baseline this export
cur.created_in = (
old.created_in
if old.created_in and old.created_in != "unknown"
else config_version
)
cur.edited_in = config_version
cur.deleted_in = None
cur.status = "active"
result.unchanged_count += 1
elif old.content_hash == cur.content_hash:
cur.created_in = old.created_in
cur.edited_in = old.edited_in
cur.deleted_in = None
cur.status = "active"
result.unchanged_count += 1
else:
cur.created_in = old.created_in or config_version
cur.edited_in = config_version
cur.deleted_in = None
cur.status = "active"
result.edited_count += 1
result.active.append(cur)
for key, old in previous.items():
if key in current_keys:
continue
if old.status == "deleted" and old.deleted_in:
# already tombstoned — keep
result.deleted.append(old)
continue
tomb = PageRecord(
meta_key=old.meta_key,
title=old.title,
filename=old.filename,
type_dir=old.type_dir,
object=old.object,
form=old.form,
source=old.source,
content_hash=old.content_hash,
created_in=old.created_in,
edited_in=old.edited_in,
deleted_in=config_version,
status="deleted",
)
result.deleted.append(tomb)
result.deleted_count += 1
result.active.sort(key=lambda r: r.meta_key)
result.deleted.sort(key=lambda r: r.meta_key)
return result
+37 -15
View File
@@ -26,24 +26,14 @@ def _run(cmd: list[str], *, cwd: Path) -> str:
return proc.stdout return proc.stdout
def push_wiki( def prepare_wiki_clone(
content_dir: Path,
wiki_url: str, wiki_url: str,
*,
work_dir: Path, work_dir: Path,
message: str, *,
branch: str = "main", branch: str = "main",
keep_unmanaged: bool = False, ) -> Path:
) -> None: """Clone or reset wiki working tree; return path to existing manifest.json if any."""
"""
Clone/fetch wiki repo into work_dir, replace managed pages, commit, push.
Managed pages = all *.md from content_dir plus images and manifest.json.
If keep_unmanaged=False, removes other *.md (except maybe custom) before copy.
"""
content_dir = content_dir.resolve()
work_dir = work_dir.resolve() work_dir = work_dir.resolve()
if work_dir.exists() and (work_dir / ".git").exists(): if work_dir.exists() and (work_dir / ".git").exists():
_run(["git", "fetch", "origin"], cwd=work_dir) _run(["git", "fetch", "origin"], cwd=work_dir)
_run(["git", "checkout", branch], cwd=work_dir) _run(["git", "checkout", branch], cwd=work_dir)
@@ -53,9 +43,41 @@ def push_wiki(
shutil.rmtree(work_dir) shutil.rmtree(work_dir)
work_dir.parent.mkdir(parents=True, exist_ok=True) work_dir.parent.mkdir(parents=True, exist_ok=True)
_run( _run(
["git", "clone", "--branch", branch, "--single-branch", wiki_url, str(work_dir)], [
"git",
"clone",
"--branch",
branch,
"--single-branch",
wiki_url,
str(work_dir),
],
cwd=work_dir.parent, cwd=work_dir.parent,
) )
return work_dir
def push_wiki(
content_dir: Path,
wiki_url: str,
*,
work_dir: Path,
message: str,
branch: str = "main",
keep_unmanaged: bool = False,
already_prepared: bool = False,
) -> None:
"""
Clone/fetch wiki repo into work_dir, replace managed pages, commit, push.
Managed pages = all files from content_dir (md/json/images).
If keep_unmanaged=False, removes other .md/.json/images before copy.
"""
content_dir = content_dir.resolve()
work_dir = work_dir.resolve()
if not already_prepared:
prepare_wiki_clone(wiki_url, work_dir, branch=branch)
if not keep_unmanaged: if not keep_unmanaged:
for path in work_dir.iterdir(): for path in work_dir.iterdir():
+88 -62
View File
@@ -12,8 +12,9 @@ if str(TOOL_ROOT) not in sys.path:
sys.path.insert(0, str(TOOL_ROOT)) sys.path.insert(0, str(TOOL_ROOT))
from export1c_help import __status__, __version__ # noqa: E402 from export1c_help import __status__, __version__ # noqa: E402
from export1c_help.config_version import read_config_version # noqa: E402
from export1c_help.export import export_help # noqa: E402 from export1c_help.export import export_help # noqa: E402
from export1c_help.wiki import WikiPushError, push_wiki # noqa: E402 from export1c_help.wiki import WikiPushError, prepare_wiki_clone, push_wiki # noqa: E402
def build_parser() -> argparse.ArgumentParser: def build_parser() -> argparse.ArgumentParser:
@@ -29,44 +30,11 @@ def build_parser() -> argparse.ArgumentParser:
sub = p.add_subparsers(dest="cmd", required=True) sub = p.add_subparsers(dest="cmd", required=True)
c = sub.add_parser("convert", help="Конвертация Help → Markdown в каталог") c = sub.add_parser("convert", help="Конвертация Help → Markdown в каталог")
c.add_argument( _add_convert_args(c)
"-s",
"--src",
type=Path,
required=True,
help="Каталог src выгрузки конфигурации (…/crm3-26/src)",
)
c.add_argument(
"-o",
"--out",
type=Path,
required=True,
help="Каталог результата (wiki working tree)",
)
c.add_argument(
"--objects-only",
action="store_true",
help="Только справка объектов (без Forms/*/Ext/Help)",
)
c.add_argument(
"--keep-empty",
action="store_true",
help="Не пропускать почти пустые страницы",
)
c.add_argument(
"--clean",
action="store_true",
help="Очистить --out перед записью (кроме .git)",
)
c.add_argument(
"--source-label",
default=None,
help="Подпись источника в Home.md",
)
c.set_defaults(func=cmd_convert) c.set_defaults(func=cmd_convert)
w = sub.add_parser("push", help="convert + push в Gitea wiki (.wiki.git)") w = sub.add_parser("push", help="convert + push в Gitea wiki (.wiki.git)")
w.add_argument("-s", "--src", type=Path, required=True) _add_convert_args(w, require_out=False)
w.add_argument( w.add_argument(
"--wiki-url", "--wiki-url",
required=True, required=True,
@@ -76,7 +44,7 @@ def build_parser() -> argparse.ArgumentParser:
"--work-dir", "--work-dir",
type=Path, type=Path,
default=None, default=None,
help="Локальный clone wiki (по умолчанию: <out>/.wiki_clone)", help="Локальный clone wiki (по умолчанию: <out-parent>/wiki_clone)",
) )
w.add_argument( w.add_argument(
"-o", "-o",
@@ -85,30 +53,75 @@ def build_parser() -> argparse.ArgumentParser:
default=None, default=None,
help="Промежуточный каталог MD (по умолчанию: ./out/wiki-md)", help="Промежуточный каталог MD (по умолчанию: ./out/wiki-md)",
) )
w.add_argument("--objects-only", action="store_true")
w.add_argument("--keep-empty", action="store_true")
w.add_argument( w.add_argument(
"--keep-unmanaged", "--keep-unmanaged",
action="store_true", action="store_true",
help="Не удалять чужие .md в wiki перед копированием", help="Не удалять чужие .md в wiki перед копированием",
) )
w.add_argument( w.add_argument("-m", "--message", default=None, help="Сообщение коммита wiki")
"-m",
"--message",
default=None,
help="Сообщение коммита wiki",
)
w.add_argument("--branch", default="main") w.add_argument("--branch", default="main")
w.add_argument( w.add_argument("--dry-run", action="store_true", help="Только convert, без git push")
"--dry-run",
action="store_true",
help="Только convert, без git push",
)
w.set_defaults(func=cmd_push) w.set_defaults(func=cmd_push)
return p return p
def _add_convert_args(p: argparse.ArgumentParser, *, require_out: bool = True) -> None:
p.add_argument(
"-s",
"--src",
type=Path,
required=True,
help="Каталог src выгрузки конфигурации (…/crm3-26/src)",
)
if require_out:
p.add_argument(
"-o",
"--out",
type=Path,
required=True,
help="Каталог результата (wiki working tree)",
)
p.add_argument(
"--objects-only",
action="store_true",
help="Только справка объектов (без Forms/*/Ext/Help)",
)
p.add_argument(
"--keep-empty",
action="store_true",
help="Не пропускать почти пустые страницы",
)
p.add_argument(
"--clean",
action="store_true",
help="Очистить --out перед записью (кроме .git)",
)
p.add_argument("--source-label", default=None, help="Подпись источника в Home.md")
p.add_argument(
"--prev-manifest",
type=Path,
default=None,
help="Предыдущий manifest.json (жизненный цикл страниц)",
)
p.add_argument(
"--config-version",
default=None,
help="Версия конфигурации (иначе VERSION / Configuration.xml)",
)
def _print_stats(stats, dest: Path) -> None:
print(
f"OK: config={stats.config_version} "
f"written={stats.pages_written} "
f"+{stats.created_count}/~{stats.edited_count}/={stats.unchanged_count}/{stats.deleted_count} "
f"skipped_empty={stats.pages_skipped_empty} "
f"total={stats.pages_total} "
f"images={stats.images_copied}{dest}"
)
def cmd_convert(args: argparse.Namespace) -> int: def cmd_convert(args: argparse.Namespace) -> int:
stats = export_help( stats = export_help(
args.src, args.src,
@@ -117,38 +130,50 @@ def cmd_convert(args: argparse.Namespace) -> int:
skip_empty=not args.keep_empty, skip_empty=not args.keep_empty,
clean=args.clean, clean=args.clean,
source_label=args.source_label, source_label=args.source_label,
prev_manifest_path=args.prev_manifest,
config_version=args.config_version,
) )
print( _print_stats(stats, args.out)
f"OK: written={stats.pages_written} "
f"skipped_empty={stats.pages_skipped_empty} "
f"total={stats.pages_total} "
f"images={stats.images_copied}{args.out}"
)
return 0 return 0
def cmd_push(args: argparse.Namespace) -> int: def cmd_push(args: argparse.Namespace) -> int:
out = args.out or (TOOL_ROOT / "out" / "wiki-md") out = args.out or (TOOL_ROOT / "out" / "wiki-md")
work = args.work_dir or (out.parent / "wiki_clone") work = args.work_dir or (out.parent / "wiki_clone")
branch = args.branch
try:
prepare_wiki_clone(args.wiki_url, work, branch=branch)
except WikiPushError as exc:
print(f"wiki prepare failed:\n{exc}", file=sys.stderr)
return 1
prev = args.prev_manifest or (work / "manifest.json")
cfg = args.config_version or read_config_version(args.src)
stats = export_help( stats = export_help(
args.src, args.src,
out, out,
include_forms=not args.objects_only, include_forms=not args.objects_only,
skip_empty=not args.keep_empty, skip_empty=not args.keep_empty,
clean=True, clean=True,
source_label=str(args.src), source_label=args.source_label or str(args.src),
prev_manifest_path=prev if prev.is_file() else None,
config_version=cfg,
) )
print( print(
f"convert: written={stats.pages_written} " f"convert: config={stats.config_version} "
f"skipped_empty={stats.pages_skipped_empty} " f"written={stats.pages_written} "
f"total={stats.pages_total}{out}" f"+{stats.created_count}/~{stats.edited_count}/={stats.unchanged_count}/{stats.deleted_count} "
f"{out}"
) )
if args.dry_run: if args.dry_run:
print("dry-run: push skipped") print("dry-run: push skipped")
return 0 return 0
message = args.message or ( message = args.message or (
f"export_1c_help v{__version__}: sync help ({stats.pages_written} pages)" f"export_1c_help v{__version__}: sync help cfg {stats.config_version} "
f"({stats.pages_written} pages, +{stats.created_count}/~{stats.edited_count}/{stats.deleted_count})"
) )
try: try:
push_wiki( push_wiki(
@@ -156,8 +181,9 @@ def cmd_push(args: argparse.Namespace) -> int:
args.wiki_url, args.wiki_url,
work_dir=work, work_dir=work,
message=message, message=message,
branch=args.branch, branch=branch,
keep_unmanaged=args.keep_unmanaged, keep_unmanaged=args.keep_unmanaged,
already_prepared=True,
) )
except WikiPushError as exc: except WikiPushError as exc:
print(f"push failed:\n{exc}", file=sys.stderr) print(f"push failed:\n{exc}", file=sys.stderr)