feat(api): Memory compaction 管线与调度修复,同步环境变量示例
Memory compaction(近重复 chunk 软排除) - 新增 compaction 调度:Redis debounce、scheduler gate、增量游标;任务结束时 finalize,避免 gate 长期占用并处理运行期新 trigger。 - Celery memory_compaction_run:debounce 未到点则 retry;用户级 Redis 锁;成功路径更新游标并 finalize;异常时释放 scheduler gate 并 self.retry,避免静默卡死调度与瞬时失败不重试。 - compaction_service:多层判定 + canonical 打分;无 embedding 时停止前移游标(awaiting_embeddings);curation details 补全 trigger 等上下文。 - ingest_transcript_sync:同步路径尽力写入 embedding,与异步 ingest 行为对齐,避免 compaction 永远扫不到无向量 chunk。 - repo:新增 update_chunk_embedding_sync。 测试 - 扩展 test_memory_compaction:调度合并、finalize、ingest embedding、无向量游标、异常路径 gate+retry 等回归用
This commit is contained in:
@@ -18,6 +18,7 @@ from app.agents.state_schema import MemoirStateSchema, SlotData, default_state
|
||||
from app.core.db import get_sync_db
|
||||
from app.core.dependencies import get_llm_provider
|
||||
from app.core.logging import get_logger
|
||||
from app.core.memory_compaction_schedule import schedule_memory_compaction_run
|
||||
from app.features.conversation.models import Segment
|
||||
from app.features.memoir.cover_eligibility import (
|
||||
chapter_needs_cover_enqueue,
|
||||
@@ -418,6 +419,17 @@ def process_memoir_segments(self, user_id: str, segment_ids: List[str]):
|
||||
if try_enqueue_generate_chapter_cover(chapter_id, source="pipeline"):
|
||||
logger.info(f"派发章节封面任务: chapter={chapter_id}")
|
||||
|
||||
schedule_memory_compaction_run(
|
||||
user_id,
|
||||
{
|
||||
"trigger_source": "memoir_segments",
|
||||
"trigger_time": datetime.now(timezone.utc).isoformat(),
|
||||
"pipeline_run_id": str(task_id),
|
||||
"story_dispatch_ids": sorted(story_dispatch_ids),
|
||||
"chapters_to_enqueue": sorted(chapters_to_enqueue),
|
||||
},
|
||||
)
|
||||
|
||||
categories_processed = sorted(prepared.category_to_segments.keys())
|
||||
logger.info(
|
||||
"回忆录处理完成: user_id={} task_id={} segment_count={} "
|
||||
|
||||
Reference in New Issue
Block a user