重构回忆录为 story-first / markdown-first 架构并整合图片意图与前端 UI 修复

本次 squash merge 将 codex-story-first-image-intent 的整体改动合入 development,核心内容包括:

1. 后端数据与迁移:新增 stories、story_versions、story_image_intents、chapter_cover_intents、assets 等模型与 Alembic 迁移,建立 story-first、markdown-first、asset-first 的主数据链路。

2. 生成与任务链:引入 StoryBuilderOrchestrator、ChapterComposerOrchestrator、story_image_tasks、chapter_cover_tasks,图片生成从正文占位符改为结构化 intent -> asset -> markdown 回填。

3. 并发与一致性:为 story/chapter intent 增加 claim_token、claimed_at、attempt_count,采用数据库原子 claim 为主、Redis 锁为辅,避免重复生成、锁误删和 processing 卡死。

4. Memoir 读写路径:章节 canonical_markdown 成为正文真源,列表/详情接口补齐 markdown、cover_asset、word_count 等字段,PDF 与 asset 解析链路同步升级。

5. Memory / Retrieval:扩展 transcript ingest、chunking、evidence 检索与 story 聚合基础设施,为后续 story-first RAG 与多 agent 编排提供底座。

6. App 端体验:章节页继续走 MarkdownRenderer 阅读链,同时吸收 fix3-19 的跨平台 UI glitch 修复;更新对话页、首页、文案资源与章节列表映射逻辑。

7. 测试与文档:补充 asset resolver、story image task、章节封面派发、markdown 映射等回归测试,并加入图片占位符退役设计文档。
This commit is contained in:
Kevin
2026-03-20 10:30:07 +08:00
parent 13e3124b85
commit 7f57f96c25
67 changed files with 4751 additions and 832 deletions

View File

@@ -4,7 +4,7 @@ PDF 生成服务(从 services 迁入 memoir feature
from app.core.logging import get_logger
from io import BytesIO
from typing import List
from typing import List, Optional
import httpx
from PIL import Image
@@ -21,12 +21,20 @@ from reportlab.platypus import (
from reportlab.pdfbase import pdfmetrics
from reportlab.pdfbase.cidfonts import UnicodeCIDFont
from app.features.memoir.asset_resolver import (
collect_asset_ids_from_markdown,
split_markdown_by_asset_refs,
strip_legacy_image_placeholders,
)
from app.features.memoir.helpers import (
_chapter_markdown,
sections_to_content_and_images,
)
from app.features.memoir.memoir_images.parser import PLACEHOLDER_RE
from app.features.memoir.memoir_images.schema import (
IMAGE_STATUS_COMPLETED,
normalize_image_assets,
)
from app.features.memoir.memoir_images.serializers import memoir_image_to_dict
from app.features.memoir.memoir_images.storage import (
CosDownloadUrlError,
TencentCosStorageService,
@@ -60,24 +68,6 @@ def split_content_blocks(content: str, images: list[dict]) -> list[dict]:
return blocks
def sections_to_blocks(sections: list, prepare_fn=None) -> list[dict]:
if prepare_fn is None:
prepare_fn = _prepare_pdf_image_assets
blocks: list[dict] = []
for section in sorted(sections, key=lambda s: getattr(s, "order_index", 0)):
content = (getattr(section, "content", None) or "").strip()
if content:
blocks.append({"type": "text", "value": content})
img = None
if getattr(section, "image_record", None):
img = memoir_image_to_dict(section.image_record)
if img:
prepared = prepare_fn([img])
if prepared and prepared[0].get("url"):
blocks.append({"type": "image", "url": prepared[0]["url"]})
return blocks
def _prepare_pdf_image_assets(images: list[dict]) -> list[dict]:
storage = TencentCosStorageService.from_env()
prepared_assets: list[dict] = []
@@ -132,7 +122,12 @@ class PDFService:
logger.warning("PDF 图片下载失败: url=%s, error=%s", url, exc)
return None
async def generate_pdf(self, book, chapters: List) -> bytes:
async def generate_pdf(
self,
book,
chapters: List,
asset_url_map: Optional[dict[str, str]] = None,
) -> bytes:
buffer = BytesIO()
doc = SimpleDocTemplate(buffer, pagesize=A4)
styles = getSampleStyleSheet()
@@ -170,16 +165,27 @@ class PDFService:
for chapter in chapters:
story.append(Paragraph(chapter.title, heading_style))
story.append(Spacer(1, 0.2 * inch))
sections = getattr(chapter, "sections", None) or []
if sections:
blocks = sections_to_blocks(sections)
# 正文真源canonical_markdown与 API / 前端一致)
markdown = _chapter_markdown(chapter)
_, images_list = sections_to_content_and_images(chapter)
if not markdown:
markdown = getattr(chapter, "content", "") or ""
if not images_list:
images_list = list(getattr(chapter, "images", None) or [])
prepared_images = _prepare_pdf_image_assets(images_list)
blocks: list[dict]
if asset_url_map and collect_asset_ids_from_markdown(markdown):
blocks = split_markdown_by_asset_refs(
markdown,
lambda aid: asset_url_map.get(aid) if asset_url_map else None,
)
for b in blocks:
if b.get("type") == "text":
b["value"] = strip_legacy_image_placeholders(
b.get("value") or ""
)
else:
images = _prepare_pdf_image_assets(
getattr(chapter, "images", None) or []
)
blocks = split_content_blocks(
getattr(chapter, "content", "") or "", images
)
blocks = split_content_blocks(markdown, prepared_images)
for block in blocks:
if block["type"] == "text":
paragraphs = block["value"].split("\n\n")