聊天和回忆录证据检索都走 pgvector,去掉 Postgres FTS/content_tsv,新迁移删掉 content_tsv 列(部署要先 alembic upgrade)。
Embedding 端口增加 is_available(),聊天和回忆录日志用统一方式表示向量是否真能调用。 记忆整理(compaction)支持 Beat 定期扫用户; 事实抽取提示与 subject 归一化,减少同一人多种称呼;
This commit is contained in:
@@ -15,7 +15,8 @@ This document summarizes production-oriented behavior for the memoir narrative p
|
||||
| `memoir_fidelity_fail_open_on_parse_error` | `False` | When `True`, fidelity JSON/LLM failures pass the gate even for new stories (rollback only via ops need). |
|
||||
| `memoir_narrative_evidence_overlap_min_chars` | `14` | Deterministic overlap check between body and evidence plain text. |
|
||||
| `memoir_title_slots_require_body_or_oral_match` | `True` | Narrows title-generation slot inputs to body/oral overlap. |
|
||||
| `memory_fact_search_use_recent_fallback` | `False` | When `False`, fact FTS misses do **not** fall back to “recent confirmed facts” (reduces contradictory/unrelated facts in prompts). |
|
||||
| `memory_fact_search_use_recent_fallback` | `False` | When `False`, fact ILIKE misses do **not** fall back to “recent confirmed facts” (reduces contradictory/unrelated facts in prompts). |
|
||||
| `memory_compaction_enabled` | `True` | Near-duplicate chunk soft-exclude; requires Celery worker + **Beat** for periodic `memory_compaction_sweep`. |
|
||||
| `memoir_recompose_retry_on_lock_contention` | `True` | Chapter recompose retries with backoff when the chapter pipeline lock is held. |
|
||||
| `memoir_phase2_singleflight_immediate` | `True` | Immediate Phase 2 `send_task` uses a stable `task_id` per user/category to reduce duplicate queue entries. |
|
||||
| `chapter_pipeline_lock_ttl_seconds` | `360` | Shared lock TTL for Phase 2 and `recompose_chapter`; tune with longest expected runtimes. |
|
||||
|
||||
@@ -4,32 +4,31 @@
|
||||
|
||||
| 路径 | 入口 | 检索能力 |
|
||||
|------|------|----------|
|
||||
| **异步(HTTP / MemoirService) | `MemoryService.retrieve` → `HybridRetriever` → `evidence.retrieve_evidence_bundle_async` | **FTS + 向量(pgvector)**,RRF 融合 chunks;facts / timeline 按 **query ILIKE**,无命中则 **fallback** 最近条;rolling + ILIKE **摘要**;**stories**(标题/摘要匹配) |
|
||||
| **同步(Celery) | `retrieve_evidence_sync`(`repo` 薄封装 → `evidence.retrieve_evidence_bundle_sync`) | **仅 FTS** chunks;其余与上类似(无向量) |
|
||||
| **异步(HTTP / MemoirService)** | `MemoryService.retrieve` → `HybridRetriever` → `evidence.retrieve_evidence_bundle_async` | **向量(pgvector)** chunks;facts / timeline 按 **query ILIKE**,无命中则 **fallback** 最近条;rolling + ILIKE **摘要**;**stories**(标题/摘要匹配) |
|
||||
| **同步(Celery)** | `retrieve_evidence_sync`(注入 `get_embedding_provider()` → `evidence.retrieve_evidence_bundle_sync`) | **向量** chunks + 同上元数据;与异步路径对齐 |
|
||||
|
||||
证据组装在 `app/features/memory/evidence.py`;`memory/repo` 仅提供原子查询(chunk FTS、facts/timeline 搜索、摘要列表等),story 合并在 evidence 层完成。
|
||||
证据组装在 `app/features/memory/evidence.py`;`memory/repo` 提供原子查询(chunk 向量、facts/timeline 搜索、摘要列表等),story 合并在 evidence 层完成。
|
||||
|
||||
## 为何 Celery 与 Hybrid 不完全一致
|
||||
## 依赖 embedding
|
||||
|
||||
- `ingest_transcript_sync` 仅写入 chunk + **FTS**,**跳过 embedding**(见 `MemoryService.ingest_transcript_sync` 注释),与异步 ingest 行为对齐策略不同。
|
||||
- 在 worker 中补齐同步向量检索需注入 `EmbeddingProvider` 与同会话查询,成本与可用性需单独评估。
|
||||
|
||||
业务上应假设:**线上章节生成任务**以 FTS 证据为主;**异步 API** 若配置了 embedding,检索语义更富。
|
||||
- 未配置 `ZHIPU_API_KEY`(或 provider `_client` 为空)时,chunk 检索为空列表,仍会返回 facts/timeline/summaries/stories(按 query ILIKE)。
|
||||
- 日志:`HybridRetriever` / `retrieve_evidence_bundle_sync` 在无 provider 或空向量时会打 warning。
|
||||
|
||||
## 空 query
|
||||
|
||||
- 默认:`relevant_*` 均为空(与历史行为一致)。
|
||||
- 若设置 `memory_evidence_empty_query_include_rolling=true`:返回**无 chunk FTS**,但含 **rolling 摘要**、最近 facts / timeline(用于「浏览」模式)。
|
||||
- 若设置 `memory_evidence_empty_query_include_rolling=true`:返回**无 chunk**,但含 **rolling 摘要**、最近 facts / timeline(用于「浏览」模式)。
|
||||
|
||||
## 富化(ingest 后 LLM)
|
||||
|
||||
- `memory_enrichment_enabled`(默认 `true`):`ingest_transcript` / `ingest_transcript_sync` 后执行摘要、事实、时间线;`false` 时跳过。
|
||||
- `memory_enrichment_max_chars`:截断送入 LLM 的文本长度。
|
||||
- 同一 `memory_source_id` 的时间线在重跑富化前会先删后插入,避免重复事件。
|
||||
- Ingest 写入 **embedding**(best-effort);历史 FTS 列 `content_tsv` 已由迁移 `0007_drop_chunk_content_tsv` 删除。
|
||||
|
||||
## Celery 任务中的顺序
|
||||
|
||||
`process_memoir_segments`(`app/tasks/memoir_tasks.py`)在**同一任务**内先执行 `ingest_transcript_sync`(并 `commit`),再执行 `MemoirOrchestrator` 与 `run_story_pipeline_for_category_batch`。因此 `retrieve_evidence_sync` 能看到**本批刚写入**的 memory chunks(无竞态)。
|
||||
`process_memoir_segments`(`app/tasks/memoir_tasks.py`)在**同一任务**内先执行 `ingest_transcript_sync`(并 `commit`),再执行 `MemoirOrchestrator` 与 `run_story_pipeline_for_category_batch`。因此 `retrieve_evidence_sync` 能看到**本批刚写入**的 memory chunks(无竞态),前提是 embedding API 已成功写入向量。
|
||||
|
||||
章节分类上,若模型返回 **none** 或命中零散档案启发式,Story 侧会统一落入 **`summary` 章节**并继续叙事落库,与「本批 transcript 已进 memory」一致,避免误以为内容被丢弃。
|
||||
|
||||
|
||||
Reference in New Issue
Block a user