refactor(api,expo): 多智能体与会话收敛、回忆录兼容层移除、后端测试集大幅删减
- 对齐「多智能体收敛」与「回忆录 stories-first / markdown-first」方向：收紧运行时契约、 删除过渡兼容路径与双轨逻辑，并同步更新客户端与文档。 - Chat：以 ChatOrchestrator 为实时编排入口；删除独立 conversation_agent，精简 prompts。 - Memoir：删除 memory_agent；MemoirOrchestrator、classification / story_route 与 prompts 收敛到 prepare_batches + run_story_pipeline_for_category_batch 主链路。 - 将 agents 侧 processor 迁入 feature 层为 background_runner，并移除 features 下重复/过时 processor 封装。 - 新增 history_store，强化「conversation_messages 为 DB 真源、Redis 为缓存」模型。 - 调整 models、repo、service、session_history；精简 WS message_types，重构 pipeline 与 router。 - 移除章节占位、整章再生等旧路径；章节列表与封面逻辑要求 story 关联；收紧 cover 资格与 enqueue。 - helpers、repo、service、router、reading_segment_materialize、story_pipeline_sync、pdf_service 等按 canonical markdown / cover_asset_id 收缩；删除 memoir_images/provider 等冗余。 - tasks：memoir_tasks、chapter_cover_tasks 等大幅瘦身；story_image_tasks 等与当前图片任务对齐。 - core：config、logging、redis、task_tracker 小幅调整。 - auth / user / payment / quota：路由或服务侧删减过时接口或逻辑（如 payment router 行数减少）。 - pyproject.toml、development.sh、.env.example / .env.production、README 等同步说明或变量。 - Alembic 0001_initial_schema 微调（与当前 schema 叙事一致的小改动）。 - 回忆录：types / mappers / api、章节页与 memoir 页与后端契约对齐；markdown-renderer 调整。 - 语音：删除 voice/player，voice-segment-store 相应精简。 - api/tests：删除 conftest 及绝大部分既有测试文件（websocket_baseline、conversation、memoir 图片、PDF、SMS 等），属有意收缩/待按 backend-test-system 重建的信号。 - docs：新增多智能体收敛与移除兼容层计划摘要；更新 story-first 设计、backend-test-system、 multi-agent-refactor-plan、实施总结等。 BREAKING CHANGE: 后端对外契约、回忆录章节字段与若干路由/任务行为已变更；大量 API 测试被移除， CI 若依赖这些用例需按新策略补测或调整流水线。
This commit is contained in:
Kevin
@@ -6,21 +6,31 @@ MemoirOrchestrator：按 segment 编排流水线，调用各 Specialist Agent。
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, Callable, Dict, List, Set, Tuple
|
||||
|
||||
from app.core.logging import get_logger
|
||||
from app.features.conversation.models import Segment
|
||||
from app.agents.state_schema import MemoirStateSchema
|
||||
|
||||
from app.agents.memoir.extraction_agent import ExtractionAgent, ExtractionResult
|
||||
from app.agents.memoir.classification_agent import (
|
||||
ClassificationAgent,
|
||||
)
|
||||
from app.agents.memoir.classification_agent import (
|
||||
_detect_stage as detect_stage_from_keywords,
|
||||
)
|
||||
from app.agents.memoir.extraction_agent import ExtractionAgent, ExtractionResult
|
||||
from app.agents.state_schema import MemoirStateSchema
|
||||
from app.core.logging import get_logger
|
||||
from app.features.conversation.models import Segment
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
@dataclass
|
||||
class PreparedMemoirBatches:
|
||||
"""Explicit batching result: updated state + segments grouped by chapter category."""
|
||||
|
||||
state: MemoirStateSchema
|
||||
category_to_segments: Dict[str, List[Segment]]
|
||||
|
||||
|
||||
class MemoirOrchestrator:
|
||||
"""
|
||||
回忆录生成编排器。
|
||||
@@ -32,6 +42,57 @@ class MemoirOrchestrator:
|
||||
self.extraction_agent = ExtractionAgent()
|
||||
self.classification_agent = ClassificationAgent()
|
||||
|
||||
def prepare_batches(
|
||||
self,
|
||||
*,
|
||||
segments: List[Segment],
|
||||
llm: Any,
|
||||
get_or_create_state: Callable[[], MemoirStateSchema],
|
||||
update_slot: Callable[[str, str, str, List[str]], MemoirStateSchema],
|
||||
) -> PreparedMemoirBatches:
|
||||
"""
|
||||
遍历 segments：Extraction → slot 更新 → Classification → 按 category 分桶。
|
||||
不含锁与写章节/故事（由调用方显式执行）。
|
||||
"""
|
||||
state = get_or_create_state()
|
||||
category_to_segments: Dict[str, List[Segment]] = {}
|
||||
|
||||
for segment in segments:
|
||||
text = segment.transcript_text or ""
|
||||
initial_stage = detect_stage_from_keywords(
|
||||
text, state.current_stage or "childhood"
|
||||
)
|
||||
stage_slots_raw = state.slots.get(initial_stage, {}) or {}
|
||||
|
||||
result: ExtractionResult = self.extraction_agent.extract(
|
||||
user_message=text,
|
||||
current_stage=state.current_stage or "childhood",
|
||||
stage_slots=stage_slots_raw,
|
||||
llm=llm,
|
||||
)
|
||||
detected_stage = result.detected_stage
|
||||
for slot_name, snippet in result.slots.items():
|
||||
state = update_slot(detected_stage, slot_name, snippet, [segment.id])
|
||||
|
||||
chapter_category = self.classification_agent.classify(
|
||||
text=text,
|
||||
fallback_stage=detected_stage,
|
||||
llm=llm,
|
||||
)
|
||||
if chapter_category is None:
|
||||
logger.debug(
|
||||
"段落无回忆录价值，跳过: segment_id=%s transcript=%s",
|
||||
segment.id,
|
||||
getattr(segment, "transcript_text", None) or "",
|
||||
)
|
||||
continue
|
||||
category_to_segments.setdefault(chapter_category, []).append(segment)
|
||||
|
||||
return PreparedMemoirBatches(
|
||||
state=state,
|
||||
category_to_segments=category_to_segments,
|
||||
)
|
||||
|
||||
def run(
|
||||
self,
|
||||
*,
|
||||
@@ -63,41 +124,17 @@ class MemoirOrchestrator:
|
||||
返回 (chapters_to_enqueue, processed_count)。
|
||||
raise_retry 用于锁竞争时抛出 Celery retry。
|
||||
"""
|
||||
state = get_or_create_state()
|
||||
prepared = self.prepare_batches(
|
||||
segments=segments,
|
||||
llm=llm,
|
||||
get_or_create_state=get_or_create_state,
|
||||
update_slot=update_slot,
|
||||
)
|
||||
state = prepared.state
|
||||
chapters_to_enqueue: Set[str] = set()
|
||||
category_to_segments: Dict[str, List[Segment]] = {}
|
||||
category_to_segments = prepared.category_to_segments
|
||||
|
||||
# 1) 遍历 segments：ExtractionAgent → 更新 slots；ClassificationAgent → 聚合
|
||||
for segment in segments:
|
||||
text = segment.transcript_text or ""
|
||||
# 关键词预检测阶段，用于 slot 查找（与原有逻辑一致）
|
||||
initial_stage = detect_stage_from_keywords(
|
||||
text, state.current_stage or "childhood"
|
||||
)
|
||||
stage_slots_raw = state.slots.get(initial_stage, {}) or {}
|
||||
|
||||
result: ExtractionResult = self.extraction_agent.extract(
|
||||
user_message=text,
|
||||
current_stage=state.current_stage or "childhood",
|
||||
stage_slots=stage_slots_raw,
|
||||
llm=llm,
|
||||
)
|
||||
detected_stage = result.detected_stage
|
||||
for slot_name, snippet in result.slots.items():
|
||||
state = update_slot(detected_stage, slot_name, snippet, [segment.id])
|
||||
|
||||
# ClassificationAgent
|
||||
chapter_category = self.classification_agent.classify(
|
||||
text=text,
|
||||
fallback_stage=detected_stage,
|
||||
llm=llm,
|
||||
)
|
||||
if chapter_category is None:
|
||||
logger.info("段落无回忆录价值，跳过: segment_id=%s", segment.id)
|
||||
continue
|
||||
category_to_segments.setdefault(chapter_category, []).append(segment)
|
||||
|
||||
# 2) 按 category 调用 process_category：叙事生成、持久化、封面入队标记
|
||||
# 按 category 调用 process_category：叙事生成、持久化、封面入队标记
|
||||
for chapter_category, category_segments in category_to_segments.items():
|
||||
if not acquire_lock(chapter_category):
|
||||
logger.warning(
|
||||
|
||||
Reference in New Issue
Block a user