Commit Graph

26 Commits

Author SHA1 Message Date
Sully
f09ae248f9 feat: OpenTelemetry LGTM observability, dev tooling, and memoir UX fixes (#31) (#32)
* add staging ios app build script

* feat(api): add OpenTelemetry LGTM stack for local observability

Wire OTel traces, metrics, and logs through a collector to Tempo,
Prometheus, and Loki, with custom LLM instrumentation, dev compose overlay,
Grafana provisioning, env templates, and development.sh auto-start.



* feat: expand observability, harden dev tooling, and fix expo staging UX

Add business and LLM Prometheus metrics with Grafana dashboards, alerting,
and a metrics verification script. Wire telemetry through adapters and core
LLM paths, and document the local LGTM workflow.

Fix development.sh for macOS bash 3.2, open Grafana and eval-web in Chrome,
and repair eval-web auto-open (unbound EVAL_WEB_BROWSER_SCHEDULED). Merge
internal-eval into the main dev script with improved compose handling.

Require EXPO_PUBLIC_* at build time, improve iOS HTTP ATS for staging IPs,
show memoir empty state instead of load errors when no chapters exist, and
add jest env setup plus chapter list response normalization.



* chore: enable Grafana Assistant Cursor plugin



* fix: memoir empty state and repair withdrawn 0020_chapters_book_id stamp

Show empty memoir UI when the chapter list succeeds with no items; treat auth/404 as non-fatal. Extend alembic revision repair so local dev DBs stamped with the removed 0020_chapters_book_id migration can roll back and upgrade to 0019.



---------

Co-authored-by: Kevin <kevin@brighteng.org>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 15:14:13 +08:00
Kevin
7e64fc3faf Merge commit e95582a: PR #20 proactive chat, topic chips, low-info turn plan
- Merge staging workflow parent and resolve conflicts with English/i18n and WS pool
- Re-greeting: language-aware fallbacks and prompts; router passes user_language
- RealtimeSession: topic suggestion callbacks + TTS sync path preserved

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-12 11:03:35 +08:00
Kevin
ccdc4e4277 feat(i18n): persist language preference and thread through chat, memoir, TTS
- Add users.language_preference (Alembic 0018, default zh); capture at signup/SMS
  only; expose on auth and profile APIs
- Lite English prompts for chat and memoir; localized stage labels and agent
  names (Life Echo / 岁月知己)
- Tencent TTS: language-aware synthesis, ModelType=1 for 501004, English chunking
- WebSocket pipeline: emit all AGENT_RESPONSE segments when TTS cancels; INFO logs
  for tts_this_turn and TTS decisions; on-demand TTS logging
- Expo: device language on auth, i18n tiers/agent name, [SPLIT] streaming UX fixes
- Tests for migration, prompts, pipeline, router tts_this_turn, reply segments

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-11 16:16:49 +08:00
Claude
55cfbc7f80 feat: agent proactively re-engages users on returning sessions
Two complementary changes to reduce conversation cold-start friction:

A. Returning-user re-greeting (backend)
- When WS reconnects to a non-empty conversation and last_message_at is older
  than chat_re_greeting_idle_hours (default 6h), the agent emits a warm
  continuation message that references prior history instead of staying silent.
- Self-debouncing: the AI message updates last_message_at, so reconnects
  within the window will not re-trigger.
- Skipped while profile collection is still pending.

D. Topic suggestion chips (backend + Expo)
- New WS message type topic_suggestions carries 3-4 quick-start chips derived
  from the current memoir stage's empty slots (deterministic, no extra LLM
  cost). Sent alongside opening / re-greeting / resume.
- Expo chat screen renders a horizontally-scrollable chip row above the input
  bar; tapping a chip sends the chip's text as a user message and clears the
  row. Sending any text/voice also clears the chips.
2026-05-07 15:39:33 +00:00
Kevin
71fbd39e32 feat(api)!: memory single chain — async MemoryService, strict eval closure
Route all memory ingest/retrieve/enrichment/compaction through async MemoryService.
Remove legacy sync memory implementations (ingest/retrieve/compaction); Celery and
memoir Phase2 call asyncio.run into MemoryService-backed helpers.

Memoir Phase1 batch ingest uses MemoryService.ingest_transcripts_batch; drop chapters.
evidence_bundle_json mirror (Alembic 0015). Evaluation uses snapshot/link-only bundles;
raise EvidenceClosureMissing instead of partial/fallback lineage tiers.

Split memoir state into NarrativeCoverageState and InterviewControlState; delete the
_interview_meta_store adapter layer. Remove rolling-query and recent-fact fallback
settings from config and evidence assembly.

Update judges, docs, tests, and PlaygroundPage alignment.

Made-with: Cursor
2026-04-30 14:11:50 +08:00
Kevin
ac436b87a2 feat(api): 收敛对话与记忆流程边界,引入 LLM 网关与专用服务
- MemoryService 异步路径委托 MemoryIngestService / MemoryRetrievalService;富化派发经 MemoryEnrichmentScheduler
- WebSocket pipeline 经 ChatTurnService 与显式 DTO 编排单轮对话;回忆录片段入队由 MemoirIngestScheduler 封装
- 新增 LlmGateway(LlmUseCase),各 agent、任务与适配器对齐 ports
- 补充 memory 提示适配、runtime 类型、memory-retrieval 文档、ai-touchpoints 说明与扫描脚本及配套测试

Made-with: Cursor
2026-04-30 09:17:01 +08:00
Kevin
3121d1384d WIP: memory system improvements (in progress)
Interview/chat prompt layers, reply planner, style profiles, memory
injection, interview meta store, and related tests. Work not finished.

Made-with: Cursor
2026-04-22 16:56:28 +08:00
Kevin
ccf7125473 fix(chat): 重复追问被拦截时再多问一次模型
防重复问句会把整段回复削成「这一段我记住了。」只剩一句套话时,用带纠偏说明的 system 再调一次 LLM,尽量避免用户只看到干巴巴_ack。仍只重试一次,并打日志与 meta 标记 duplicate_question_guard_llm_retry。
2026-04-10 15:35:34 +08:00
Kevin
5ff495729e feat(chat): server-side interview turn plan (mode, anchor slot, snippet)
- Add plan_interview_turn: emotion_first / memoir_push / follow_user_only
- Inject hard directive block at top of guided system prompt
- Pass stage_switched_this_turn from ChatOrchestrator after stage detection
- Log interview_turn_plan for observability; add unit tests
2026-04-10 14:34:38 +08:00
Kevin
df6eafeae2 feat(chat): host-style memoir prompts and strip parenthetical stage directions
- Add strip_parenthetical_asides_for_chat in reply pipeline before [SPLIT]
- Expand output_rules bans (performance parens) and voice as warm host
- Refocus opening/guided prompts on pulling conversation toward memoir oral history
- Align interview opening fallbacks with memoir-first tone
- Add unit tests for parenthetical stripping
2026-04-10 13:55:56 +08:00
yangshilin
e1341c6d18 feat:
1. 建立问题库大纲,对应每个人生阶段槽位
2. 鼓励使用更生活化的交流语言共情与总结
3. 降低评审模型可能发生截断的概率
4. 成稿质量维度强化情感表达和上下文连贯性
2026-04-09 15:32:35 +08:00
Kevin
064ad2161d refactor(eval+memoir):精简内部评测路由与服务,composite/对话摘要与 judge 能力补强
- 访谈:新增 interview_state_hints,联动 orchestrator 与提示词
- 回忆录:story_pipeline_sync/state/memory/post_commit 与 Celery 任务调整
- 基建:开发用 celery broker、compose/development 脚本、依赖注入
- eval-web:移除数据集/实验/版本等页面与流式轮询,突出 Playground
- 文档与单测同步
2026-04-08 21:36:12 +08:00
Kevin
2a0c80987d test/ 调整prompt,提高共情能力 2026-04-08 17:10:09 +08:00
Kevin
309a051038 feat: 回忆录证据血缘与内部评测可追溯,顺带对齐本地评测台与 CI
数据库与模型:新增多版迁移(章节证据快照、对话血缘、记忆事实/时间线 lineage 等),把「成稿 ↔ 对话/记忆」的溯源信息落到表结构里。
业务链路:会话与 WS、回忆录/故事流水线、记忆写入与 enrichment 等跟着接上线索与快照;新增章节证据快照与评测侧 EvalTraceService 等模块,方便组评审用的证据包。
内部评测:自动化 run 与手工 memoir 评审共用可追溯证据;rubric/ judge 相关脚本与文档有配套调整。
app-eval-web:Memoir/实验详情里能展开看证据摘要与 evidence_trace(含对话轮次 id);Vite 代理与 development.sh 注入的 API 端口与当前默认内部评测端口一致,避免改端口后页面连错服务。
工程杂项:GitHub Actions / 仓库说明有更新;各适配器与支付/配额/plan 等多处为小改动或跟随主改动的收尾;新增/扩充了?
2026-04-08 15:37:09 +08:00
Kevin
2fded6fbd9 refactor(chat): AI-native prompts, remove interview heuristics
- Drop interview_reply_length and utterance_substance; always run stage LLM
  and memory retrieval when enabled; trim Settings fields and .env.example.
- Replace guided/opening prompts with compact fact blocks plus unified
  behavior guidance; slim background_voice and persona to tone hints.
- InterviewAgent uses fixed chat_interview max_tokens/chars/segments.

Also includes stacked work: profile followup/extract path, evaluation rubric
and judge schema updates, transcript SPLIT handling in execution service,
user export markdown split tests, and golden case fixture.
2026-04-06 22:23:46 +08:00
Kevin
4cfa3843a7 chore/ 精简展示AI活动的日志 2026-04-03 13:49:24 +08:00
Kevin
bb16d3a5c9 refactor(agents): 抽取阶段常量与对话上下文;快档 LLM;图片 prompt 可禁止回退
访谈与阶段
- 新增 app/agents/stage_constants.py:集中 CHAT_STAGES、章节分类/顺序、阶段到默认 memoir 类别等,与 MemoirState 默认槽位顺序对齐;减少散落在 prompts 内的重复常量。
- 新增 app/agents/chat/prompt_context.py:以 ChatPromptContext 汇总 guided 系统提示所需字段(阶段、槽位、轮次、人设、记忆证据、回复长度模式、背景声线、职业等),统一走 get_guided_conversation_prompt。
- 大幅收敛 app/agents/chat/prompts_conversation.py;调整 prompts.py、stage_prompts.py、stage_detection.py;同步 interview_agent、profile_agent、helpers 与 state_schema,使对话侧构造提示的方式一致、可测。

回忆录流水线
- memoir/prompts.py 删除已迁至 stage_constants / 独立模板的大段常量与图片占位相关逻辑;classification / extraction / fidelity / narrative agents 与 orchest(全量历史仍可用于计数,注入模型时按轮次与字符上限截断)、image_prompt_fallback_disabled。
- dependencies 增加 get_llm_provider_fast(LRU 缓存,可与默认共用密钥与 base_url)。

任务与编排
- memoir_tasks:prepare_batches 注入 llm_fast;开启独立快档模型时打结构化日志。
- chapter_cover_tasks、story_image_tasks:与图片 prompt / JSON 工具路径或策略变更对齐(import 与行为一致)。
- story_pipeline_sync 等小处同步。

其它核心
- langchain_llm、text_normalize 随上述调用链微调。

开发者体验
- .cursor/settings.json:启用 redis-development、postman 插件。

测试
- 新增 test_image_prompt_policy:覆盖「禁止回退」等图片 prompt 策略。
- 更新 test_interview_prompts、test_interview_reply_length、test_experience_regressions、test_json_and_memory_utils,匹配新常量位置、json_utils 与对话/长度行为。
2026-04-02 12:00:00 +08:00
Kevin
53d9e003af feat(api): 叙事 prompt、职业上下文、读路径章节、WS 解耦与错误脱敏
- 回忆录:事实边界补充允许清单;传记文体示例与 JSON 叙事要求对齐
- default 职业提示 occupation_context;cadre/military 退休语境
- GET 章节读路径零写入,prepare_chapter_read_view + markdown_for_response
- 文本归一抽到 core/text_normalize;移除弃用 reply 策略与 recompose_chapters_for_story
- ConversationService:WS 连接/用户段落/结束对话;对外错误固定文案
- 测试:HTTP 脱敏契约、章节读视图、occupation 与 background_voice
2026-04-01 11:55:52 +08:00
Kevin
69a673e6c6 feat(api): 访谈人格/回复长度策略、口述归一、背景语气与输入净稿全链路
Chat 访谈
- 新增 persona 系统(default / warm_listener / curious_guide)与 background_voice 语气层
- 回复长度由 compute_reply_plan 统一决策(brief / standard / expanded),融合信息密度启发式
- 输入净稿(input_normalize):编排层可选 rules/llm 归一用户口语后再喂模型与记忆检索
- 记忆证据注入:按用户话检索 memory evidence 并注入 prompt

Memoir 回忆录
- 口述归一(oral_normalize):segment 原文保留,story 管线取派生净稿作叙事输入
- segment 入队批次门闸:累计字数 + 最长等待秒数,减少零碎提交
- fidelity_check / prompts / narrative_agent 微调
- Alembic 0005:清理跨章节 story 外键

Infra
- Dockerfile 加入 ffmpeg
- pyproject.toml 新增依赖并同步 uv.lock
- .env.example / .env.production 补全新配置项

Tests
- 新增 test_background_voice、test_chat_input_normalize、test_experience_regressions
- 扩展 test_interview_prompts、test_interview_reply_length、test_story_route_oral_invariant

Made-with: Cursor
2026-03-31 23:55:26 +08:00
Kevin
e4bf0710c7 feat(memory,conversation): 记忆富化/证据包、时间线幂等字段与对话分段全链路
数据库
- 新增迁移 0003:timeline_events.memory_source_id 外键 → memory_sources,便于按 ingest 源做时间线幂等

后端 - 记忆
- 新增 ingest 后 LLM 富化(摘要/事实/时间线),可配置开关与最大字符数
- 新增证据包组装:合并 chunk、摘要、事实、时间线、故事等检索结果;支持空 query 时是否仍带 rolling 等开关
- repo/retriever/service/router/schemas/summarizer/timeline/extractor 等扩展;文档 memory-retrieval.md 更新

后端 - 对话 WS
- 增加 PING/PONG;分段 ASR 日志与空音频处理;转写失败与「无助手回复」错误提示更明确
- 助手多段回复持久化使用统一分隔符,与分段逻辑一致

后端 - Agent
- reply_limits:按 [SPLIT] 与段落拆段,并保证非空 fallback,供 WS 与 TTS 多段下发

后端 - 回忆录任务
- transcript ingest 记录 source_id;任务成功结?
2026-03-27 16:24:43 +08:00
Kevin
a3f61fcc0f feat(api+app): 对话阶段化、回忆录流水线与客户端会话体验
- DB: segments 用户输入文本(Alembic 0002)
- Chat: 阶段检测/阶段提示/回复限制,编排与访谈/画像 prompts 调整
- Memoir: 忠实度检查 agent,叙事与分类等链路更新
- Core: agent 日志、Alembic 启动、LangChain/日志/配置等
- Story: time_hints;Memory 检索与相关测试
- Expo: 助手头像、会话页与消息拆分、实时会话与文案/i18n
- Docs/scripts/tests: 迁移脚本、LLM JSON/记忆检索文档、新增单测
2026-03-26 12:13:36 +08:00
Kevin
70070216c4 修复一些已知问题 2026-03-20 17:25:42 +08:00
Kevin
7317bf10cd fix/various fixes 2026-03-20 15:15:35 +08:00
Kevin
c6e07ce5ca chore/ 删除无用文件 2026-03-19 14:36:40 +08:00
yangshilin
67fb5d2cb6 refactor: 进一步重构agents目录结构 符合多agent架构 2026-03-19 10:54:48 +08:00
yangshilin
b16bb2b96c feat & refactor: 重构agents目录结构;AI回复模块agent结构封装 2026-03-19 10:36:55 +08:00