docs: 添加重构相关文档。添加cursor rules
This commit is contained in:
Kevin
24
.cursor/rules/Backend-Develop-Guideline.mdc
Normal file
24
.cursor/rules/Backend-Develop-Guideline.mdc
Normal file
@@ -0,0 +1,24 @@
|
||||
---
|
||||
alwaysApply: true
|
||||
---
|
||||
|
||||
# Backend Fastapi
|
||||
|
||||
## 架构规则
|
||||
1. **`common/` / `utils/` / `shared/` / `helpers/` 禁止创建**：只有 truly cross-cutting 能力进 `core/`，业务 DTO/校验/错误码归 feature
|
||||
2. **router 禁止操作 DB**：所有数据访问通过 repo，所有业务逻辑通过 service
|
||||
3. **service 禁止 import adapter**：只依赖 port protocol，通过 DI 注入。代码中出现 `from adapters.xxx import ...` 即为违规
|
||||
4. **feature 间禁止 import router**：跨 feature 调用通过 service 注入
|
||||
5. **所有 schema 变更走 Alembic**：禁止直接 DDL
|
||||
6. **新增 provider 必须实现 port protocol**：不得在 feature 中直接调用 SDK
|
||||
7. **事务边界：repo 不提交，service 管事务**：`repo.py` 只做 `add/delete/query`，`commit/rollback` 由 service 或 UoW 统一执行。`get_async_db()` 不自动 commit
|
||||
8. **port 边界不可打穿**：service 需要厂商增强能力时，必须扩充 port 或定义第二个窄 port，禁止直接引用 adapter 扩展方法
|
||||
9. **quota 是独立 feature**：conversation、memoir、payment 如需配额检查，通过注入 `QuotaService`，不得直接 import quota 内部函数
|
||||
10. **成功响应不强制包装**：`core/errors.py` 只统一错误响应格式 `{error_code, message, request_id}`，成功响应直接返回 Pydantic model / FileResponse / 原始结构
|
||||
|
||||
## 依赖管理（uv）
|
||||
11. **依赖统一用 uv 管理**：禁止直接 `pip install`、禁止手动编辑 `pyproject.toml` 的 `[project.dependencies]` 或 `[dependency-groups]`
|
||||
12. **新增依赖用 `uv add <pkg>`**，dev 依赖用 `uv add --dev <pkg>`，移除用 `uv remove <pkg>`
|
||||
13. **`uv.lock` 必须纳入版本控制**：保证 CI 和本地环境精确一致
|
||||
14. **安装环境统一用 `uv sync`**：开发环境 `uv sync --dev`，生产环境 `uv sync --no-dev`
|
||||
15. **运行命令统一用 `uv run`**：如 `uv run pytest`、`uv run alembic upgrade head`、`uv run uvicorn ...`
|
||||
5
.cursor/rules/Use-context7-to-lookup-documentation.mdc
Normal file
5
.cursor/rules/Use-context7-to-lookup-documentation.mdc
Normal file
@@ -0,0 +1,5 @@
|
||||
---
|
||||
alwaysApply: true
|
||||
---
|
||||
|
||||
Always use Context7 when I need library/API documentation, code generation, setup or configuration steps without me having to explicitly ask.
|
||||
575
docs/plans/backend-architectural-refactor-plan.md
Normal file
575
docs/plans/backend-architectural-refactor-plan.md
Normal file
@@ -0,0 +1,575 @@
|
||||
---
|
||||
name: 架构重构计划
|
||||
overview: 将现有 Life Echo API 从混合分层结构重构为 feature-first flat 架构，引入 ports/adapters 实现 provider-agnostic 基础设施抽象，集成 Alembic 做 schema 变更治理，统一日志到 loguru。
|
||||
todos:
|
||||
- id: phase0-core
|
||||
content: "Phase 0: 搭建 app/core/ 骨架（config, db, logging, errors, security, dependencies, middleware）"
|
||||
status: pending
|
||||
- id: phase0-txn
|
||||
content: "Phase 0: 重写 get_async_db() 去掉自动 commit，定义事务边界规则"
|
||||
status: pending
|
||||
- id: phase0-alembic
|
||||
content: "Phase 0: 初始化 Alembic，手写空 baseline，stamp head（不依赖 autogenerate）"
|
||||
status: pending
|
||||
- id: phase0-loguru
|
||||
content: "Phase 0: 集成 loguru + InterceptHandler + request_id middleware"
|
||||
status: pending
|
||||
- id: phase0-errors
|
||||
content: "Phase 0: 实现全局异常体系 + exception handlers（仅错误响应统一，成功响应不包装）"
|
||||
status: pending
|
||||
- id: phase1-ports
|
||||
content: "Phase 1: 定义 6 个 port protocol（sms, llm, image_gen, storage, tts, asr）"
|
||||
status: pending
|
||||
- id: phase1-adapters
|
||||
content: "Phase 1: 封装现有 SDK 代码为 adapter 实现"
|
||||
status: pending
|
||||
- id: phase1-di
|
||||
content: "Phase 1: 在 core/dependencies.py 注册 DI factory"
|
||||
status: pending
|
||||
- id: phase2-content
|
||||
content: "Phase 2: 迁移 content feature（faq, legal, home）"
|
||||
status: pending
|
||||
- id: phase2-plan
|
||||
content: "Phase 2: 迁移 plan feature"
|
||||
status: pending
|
||||
- id: phase2-quota
|
||||
content: "Phase 2: 提取 quota 为独立 feature（QuotaService + router）"
|
||||
status: pending
|
||||
- id: phase2-user
|
||||
content: "Phase 2: 迁移 user feature（User model）"
|
||||
status: pending
|
||||
- id: phase2-auth
|
||||
content: "Phase 2: 迁移 auth feature（RefreshToken + SmsVerificationCode models，依赖 user + sms port）"
|
||||
status: pending
|
||||
- id: phase2-payment
|
||||
content: "Phase 2: 迁移 payment feature"
|
||||
status: pending
|
||||
- id: phase2-memoir
|
||||
content: "Phase 2: 迁移 memoir feature（books + chapters + memoir_state 合并）"
|
||||
status: pending
|
||||
- id: phase2-conversation
|
||||
content: "Phase 2: 迁移 conversation feature + WebSocket 拆为 ws/ 子包（6 个文件）"
|
||||
status: pending
|
||||
- id: phase3-cleanup
|
||||
content: "Phase 3: 清理旧目录，更新 Docker/CI/测试路径"
|
||||
status: pending
|
||||
isProject: false
|
||||
---
|
||||
|
||||
# Life Echo API 架构重构计划
|
||||
|
||||
## 一、现状诊断
|
||||
|
||||
当前项目位于 `api/` 下，采用混合分层结构：
|
||||
|
||||
```
|
||||
api/
|
||||
main.py
|
||||
routers/ # 16 个路由文件，部分含大量业务逻辑
|
||||
services/ # 混合了业务服务和外部 SDK 调用
|
||||
database/ # 单文件 models.py（9 个模型）+ database.py
|
||||
agents/ # AI agent 层（conversation, memory, memoir）
|
||||
middleware/ # 仅 auth.py
|
||||
payment/ # 唯一较完整的 feature 模块
|
||||
tasks/ # Celery 后台任务
|
||||
migrations/ # 手写 SQL，无版本管理
|
||||
```
|
||||
|
||||
**核心问题**：
|
||||
|
||||
- 路由层过胖，直接操作 DB 和调用外部服务
|
||||
- 外部服务耦合：SMS（腾讯云）、存储（腾讯 COS）、TTS（OpenAI）直接写死 SDK
|
||||
- 无统一配置类，`os.getenv()` 散落各处
|
||||
- 无 Alembic，手写 SQL 迁移无版本追踪
|
||||
- 标准库 logging，无结构化日志
|
||||
- 无全局异常处理
|
||||
- WebSocket 路由 ~1000 行，职责过重
|
||||
|
||||
**已有较好抽象的部分**（可复用/迁移）：
|
||||
|
||||
- `payment/`：已有 service facade + 双 provider
|
||||
- `services/llm_service.py`：通过 LangChain 已做 provider 切换
|
||||
- ASR：已有 provider 切换机制
|
||||
|
||||
---
|
||||
|
||||
## 二、目标目录结构
|
||||
|
||||
```
|
||||
api/
|
||||
app/
|
||||
main.py # 仅做组装：include_router, middleware, lifespan
|
||||
core/
|
||||
config.py # 统一 BaseSettings（pydantic-settings）
|
||||
db.py # Base, engine, session factory, get_async_db
|
||||
logging.py # loguru 配置
|
||||
errors.py # 全局异常体系 + exception handlers
|
||||
security.py # JWT 签发/校验、密码哈希
|
||||
middleware.py # CORS, request_id, logging middleware
|
||||
dependencies.py # 全局共享依赖（get_current_user, port DI factory）
|
||||
pagination.py # 通用分页
|
||||
|
||||
features/
|
||||
auth/
|
||||
router.py # 注册、登录、SMS 登录、刷新、登出
|
||||
schemas.py
|
||||
service.py # 认证业务逻辑
|
||||
models.py # RefreshToken, SmsVerificationCode
|
||||
repo.py
|
||||
deps.py # feature 特有依赖（如 sms_port 注入）
|
||||
|
||||
conversation/
|
||||
router.py # 对话 CRUD
|
||||
schemas.py
|
||||
service.py # 对话编排（调用 agent、quota 检查等）
|
||||
models.py # Conversation, Segment
|
||||
repo.py
|
||||
ws/ # WebSocket 子包（从 1067 行 websocket.py 拆分）
|
||||
router.py # WebSocket endpoint，仅做连接生命周期
|
||||
connection_manager.py # 连接注册/注销、active_connections 字典
|
||||
message_types.py # MessageType 枚举 + 消息序列化
|
||||
pipeline.py # 消息分发管道：audio_segment -> ASR -> agent -> TTS -> 响应
|
||||
profile_collector.py # 资料收集模式：缺失字段检测、提取、回填
|
||||
quota_guard.py # 配额校验 hook（从 routers/quota 独立）
|
||||
|
||||
memoir/
|
||||
router.py # books, chapters, memoir_state 合并
|
||||
schemas.py
|
||||
service.py # 回忆录编排（章节生成、状态流转）
|
||||
models.py # Book, Chapter, MemoirState
|
||||
repo.py
|
||||
processor.py # 从 agents/memoir_processor.py 迁入
|
||||
|
||||
payment/
|
||||
router.py
|
||||
schemas.py
|
||||
service.py # 统一支付门面（已有良好抽象，迁移即可）
|
||||
models.py # Order
|
||||
repo.py
|
||||
deps.py
|
||||
|
||||
user/
|
||||
router.py # profile, subscription, feedback
|
||||
schemas.py
|
||||
service.py
|
||||
models.py # User
|
||||
repo.py
|
||||
|
||||
quota/
|
||||
service.py # PLAN_QUOTAS 配置 + check_can_send/check_can_organize
|
||||
schemas.py # QuotaCheckResponse
|
||||
router.py # GET /api/quota/check
|
||||
deps.py # get_quota_service 注入
|
||||
|
||||
plan/
|
||||
router.py
|
||||
schemas.py
|
||||
service.py
|
||||
|
||||
content/
|
||||
router.py # faq, legal, home（轻量静态内容）
|
||||
schemas.py
|
||||
|
||||
agents/
|
||||
conversation_agent.py
|
||||
memory_agent.py
|
||||
prompts/
|
||||
conversation_prompts.py
|
||||
memory_prompts.py
|
||||
profile_prompts.py
|
||||
|
||||
ports/ # 能力接口定义（Protocol / ABC）
|
||||
sms.py # SmsSender protocol
|
||||
llm.py # LLMProvider protocol
|
||||
image_gen.py # ImageGenerator protocol
|
||||
storage.py # ObjectStorage protocol
|
||||
tts.py # TTSProvider protocol
|
||||
asr.py # ASRProvider protocol
|
||||
|
||||
adapters/ # 具体 provider 实现
|
||||
sms/
|
||||
tencent.py
|
||||
llm/
|
||||
deepseek.py # 基于现有 llm_service.py
|
||||
image_gen/
|
||||
liblib.py
|
||||
storage/
|
||||
tencent_cos.py
|
||||
tts/
|
||||
openai_tts.py
|
||||
asr/
|
||||
whisper_local.py
|
||||
tencent_asr.py
|
||||
payment/
|
||||
wechat.py
|
||||
alipay.py
|
||||
|
||||
tasks/
|
||||
celery_app.py
|
||||
memoir_tasks.py
|
||||
|
||||
alembic/
|
||||
env.py
|
||||
versions/
|
||||
alembic.ini
|
||||
|
||||
tests/
|
||||
...
|
||||
```
|
||||
|
||||
### 目录结构修订说明
|
||||
|
||||
**修订 1：`core/db.py` 显式列出。** 存放 `Base = declarative_base()`、engine/session factory、`get_async_db` 生成器。所有 feature models 共享此 Base。
|
||||
|
||||
**修订 2：`core/response.py` 移除。** 统一响应包装只用于错误响应（在 `core/errors.py` 的 exception handler 中处理）。成功响应直接返回 Pydantic model / `FileResponse` / 原始 dict，不强制 `{code, data, message}` 包装，避免对 FastAPI schema 自动生成、文件下载、支付回调、WebSocket 造成额外负担。
|
||||
|
||||
**修订 3：auth models 与 user models 分离。** `RefreshToken` 和 `SmsVerificationCode` 归入 `features/auth/models.py`，因为它们是 identity/session 资产，认证流程 (`auth/service.py`) 是唯一强依赖方。`User` 模型留在 `features/user/models.py`，auth 通过 import user models 引用 User（允许的依赖方向：auth -> user）。
|
||||
|
||||
**修订 4：WebSocket 从单个 `ws_handler.py` 拆为 `ws/` 子包。** 当前 `routers/websocket.py` 有 1067 行，同时承担连接管理、消息类型定义、音频分段处理、ASR 转写、配额校验、用户资料收集、Agent 编排、DB 落库、Celery 任务提交。仅改名为 `ws_handler.py` 会原样搬运复杂度。拆分为 6 个文件，每个职责单一。
|
||||
|
||||
**修订 5：`quota` 提升为独立 feature。** 当前 `routers/quota.py` 包含 `PLAN_QUOTAS` 配置、`get_segment_count`、`get_chapter_count`、`check_can_send_message`、`check_can_submit_organize`，被 conversation（WebSocket line 628）、memoir（任务提交前 line 1050）、plan 共同依赖。放在任何单个 feature 内都会造成反向依赖。独立为 `features/quota/`，其他 feature 的 service 通过注入 `QuotaService` 调用。
|
||||
|
||||
---
|
||||
|
||||
## 三、事务边界与 Session 管理
|
||||
|
||||
### 3.1 现状问题
|
||||
|
||||
当前存在两种矛盾的事务策略：
|
||||
|
||||
- `get_async_db()` 在请求结束时自动 commit（[database.py line 76](api/database/database.py)），但 router 内部又有显式 `await db.commit()`（如 [auth.py line 194](api/routers/auth.py)），导致双重 commit 或语义不清。
|
||||
- Celery 任务直接用同步 `SessionLocal()`（[memoir_tasks.py line 18](api/tasks/memoir_tasks.py)），手动管理 session 生命周期，无统一规则。
|
||||
- WebSocket handler 用 `async for db in get_async_db()` 获取独立 session（[websocket.py line 276](api/routers/websocket.py)），绕过 FastAPI 依赖注入。
|
||||
|
||||
### 3.2 新规则
|
||||
|
||||
**核心原则：repo 不提交，service 统一管理事务边界。**
|
||||
|
||||
- `core/db.py` 中的 `get_async_db()` 改为"干净 session"：只负责创建和关闭，不自动 commit/rollback。
|
||||
- 事务控制权交给 service 层：service 方法内显式 `await db.commit()` 或使用 `async with db.begin():` 块。
|
||||
- `repo.py` 只做查询和 `db.add()` / `db.delete()`，绝不调用 `commit()` 或 `flush()`。
|
||||
- router 层不接触 session，通过 `Depends` 注入 service（service 内部持有 session）。
|
||||
|
||||
**Celery 任务**：在 `tasks/` 中定义 `get_sync_db()` context manager，同样遵循"service commit"规则。任务函数调用 service 的同步版本或直接在任务函数中管理事务。
|
||||
|
||||
**WebSocket**：`ws/pipeline.py` 中每个消息处理周期获取独立 session（通过 `async with AsyncSessionLocal() as session:`），处理完成后由 pipeline 显式 commit。
|
||||
|
||||
```python
|
||||
# core/db.py - 新的 session 生成器
|
||||
async def get_async_db() -> AsyncGenerator[AsyncSession, None]:
|
||||
async with AsyncSessionLocal() as session:
|
||||
try:
|
||||
yield session
|
||||
finally:
|
||||
await session.close()
|
||||
|
||||
# features/auth/service.py - service 管理事务
|
||||
class AuthService:
|
||||
def __init__(self, db: AsyncSession, sms: SmsSender):
|
||||
self._db = db
|
||||
self._sms = sms
|
||||
|
||||
async def register(self, request: RegisterRequest) -> TokenResponse:
|
||||
user = User(...)
|
||||
self._db.add(user)
|
||||
refresh_token = RefreshToken(...)
|
||||
self._db.add(refresh_token)
|
||||
await self._db.commit()
|
||||
await self._db.refresh(user)
|
||||
return TokenResponse(...)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 四、依赖方向规则
|
||||
|
||||
用以下有向图描述允许的依赖方向（箭头 = "依赖于"）：
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
Router["features/*/router.py"] --> Service["features/*/service.py"]
|
||||
Service --> Repo["features/*/repo.py"]
|
||||
Service --> Ports["ports/*"]
|
||||
Service --> QuotaSvc["features/quota/service.py"]
|
||||
Repo --> Models["features/*/models.py"]
|
||||
Adapters["adapters/*"] -.->|"实现"| Ports
|
||||
Core["core/*"] --> Nothing["（无外部依赖）"]
|
||||
Router --> Core
|
||||
Service --> Core
|
||||
Tasks["tasks/*"] --> Service
|
||||
Agents["agents/*"] --> Ports
|
||||
WsPipeline["conversation/ws/pipeline.py"] --> Service
|
||||
WsPipeline --> Ports
|
||||
```
|
||||
|
||||
|
||||
|
||||
**硬性禁止**：
|
||||
|
||||
- `router.py` 不得直接 import adapter 或操作 DB session
|
||||
- `service.py` 不得 import 具体 adapter，只依赖 port protocol
|
||||
- `ports/` 不得 import `adapters/`（方向反转）
|
||||
- feature 之间不得互相 import router；跨 feature 调用通过 service 注入
|
||||
- `repo.py` 不得调用 `commit()` / `rollback()` / `flush()`
|
||||
|
||||
---
|
||||
|
||||
## 四、Provider-Agnostic Ports 设计
|
||||
|
||||
### 4.1 核心可移植契约（Core Portable Contract）
|
||||
|
||||
每个 port 定义为 `typing.Protocol`，仅包含业务真正依赖的最小稳定能力：
|
||||
|
||||
**SMS** (`ports/sms.py`)：
|
||||
|
||||
- `send_verification_code(phone: str, code: str) -> bool`
|
||||
|
||||
**LLM** (`ports/llm.py`)：
|
||||
|
||||
- `complete(messages: list[Message], temperature: float, ...) -> str`
|
||||
- `stream(messages: list[Message], ...) -> AsyncIterator[str]`
|
||||
|
||||
**ImageGen** (`ports/image_gen.py`)：
|
||||
|
||||
- `generate(prompt: str, size: ImageSize, ...) -> ImageResult`
|
||||
- `check_status(task_id: str) -> TaskStatus`
|
||||
|
||||
**Storage** (`ports/storage.py`)：
|
||||
|
||||
- `upload(key: str, data: bytes, content_type: str) -> str`
|
||||
- `get_url(key: str, expires: int) -> str`
|
||||
- `delete(key: str) -> None`
|
||||
|
||||
**TTS** (`ports/tts.py`)：
|
||||
|
||||
- `synthesize(text: str, voice: str) -> bytes`
|
||||
|
||||
**ASR** (`ports/asr.py`)：
|
||||
|
||||
- `transcribe(audio: bytes, format: str) -> str`
|
||||
|
||||
### 5.2 Provider Extensions 规则
|
||||
|
||||
**绝对禁止 service 直接拿 adapter 扩展方法。** 这会打穿 port 边界，让 service 再次耦合到具体 provider。
|
||||
|
||||
处理厂商特有能力的正确方式：
|
||||
|
||||
- **方案 A（首选）：扩充现有 port。** 如果能力具备跨 provider 通用性（如 image_gen 的 inpaint），在 port protocol 上新增方法，各 adapter 按能力实现或抛 `NotImplementedError`。
|
||||
- **方案 B：定义第二个 port。** 如果能力确实只属于某个 provider 的特殊功能（如 Liblib 的模板管理），定义一个独立的窄 port（如 `ImageTemplateManager`），只在需要的 feature deps 中注入。service 依赖的仍然是 port protocol，而非 adapter 类。
|
||||
- **方案 C：adapter 内部自行消化。** 厂商特有的配置项（如 Liblib template UUID、COS 生命周期策略）作为 adapter 构造参数传入，不暴露给 service。adapter 在实现 port 方法时自行使用这些配置。
|
||||
|
||||
**判断标准**：如果 service 代码中出现了 `from adapters.xxx import ...`，说明边界被打穿了。
|
||||
|
||||
### 4.3 DI 装配
|
||||
|
||||
在 `core/dependencies.py` 中根据 `config.py` 的配置选择 adapter：
|
||||
|
||||
```python
|
||||
def get_sms_sender() -> SmsSender:
|
||||
if settings.sms_provider == "tencent":
|
||||
return TencentSmsSender(settings.tencent_sms)
|
||||
raise ValueError(f"Unknown SMS provider: {settings.sms_provider}")
|
||||
```
|
||||
|
||||
feature 的 `deps.py` 或 router 通过 `Depends(get_sms_sender)` 注入。
|
||||
|
||||
---
|
||||
|
||||
## 五、Alembic 集成与 Schema 变更治理
|
||||
|
||||
### 5.1 初始化
|
||||
|
||||
- 在 `api/` 根目录执行 `alembic init alembic`
|
||||
- 修改 `alembic/env.py`，导入所有 feature 的 `models.py` 到统一 `Base.metadata`
|
||||
- `alembic.ini` 中 `sqlalchemy.url` 读取 `DATABASE_URL`
|
||||
|
||||
### 6.2 初始基线（不依赖 autogenerate）
|
||||
|
||||
当前 schema 来源是 `Base.metadata.create_all()` + 9 个手写 SQL 迁移文件，两者可能已经出现漂移。直接 autogenerate 大概率产出不准确的 diff。
|
||||
|
||||
**正确步骤**：
|
||||
|
||||
1. 在 staging/dev 环境连接真实数据库，用 `pg_dump --schema-only` 导出当前实际 schema 作为基准参照。
|
||||
2. 手写一个空的 Alembic revision 作为 baseline：`alembic revision -m "baseline_empty"`，upgrade/downgrade 均为 pass。
|
||||
3. 对现有数据库执行 `alembic stamp head`，将当前库标记为已在 head。
|
||||
4. **此后**才开始用 autogenerate 生成增量迁移，每次 review 时对照 `pg_dump` 确认无遗漏。
|
||||
5. 将 `migrations/` 目录下的 9 个手写 SQL 归档到 `migrations_legacy/`，不再使用。
|
||||
|
||||
### 5.3 变更治理规则
|
||||
|
||||
- 所有 schema 变更必须通过 Alembic migration 合入，禁止直接改 DB
|
||||
- `autogenerate` 只做草稿，PR 中必须人工 review
|
||||
- 破坏性变更（删列、改类型）采用 expand/contract：先加新列 -> 双写 -> 迁移数据 -> 切读路径 -> 删旧列
|
||||
- data migration 和 schema migration 分开写
|
||||
- 每次部署前校验 `alembic current` == `alembic heads`
|
||||
- `main.py` 移除 `Base.metadata.create_all()`，改为部署流程执行 `alembic upgrade head`
|
||||
|
||||
### 6.4 Model 拆分
|
||||
|
||||
将 `database/models.py` 的 9 个模型拆入对应 feature：
|
||||
|
||||
- `User` -> `features/user/models.py`
|
||||
- `RefreshToken`, `SmsVerificationCode` -> `features/auth/models.py`（identity/session 资产，auth 是唯一强依赖方）
|
||||
- `Conversation`, `Segment` -> `features/conversation/models.py`
|
||||
- `Book`, `Chapter`, `MemoirState` -> `features/memoir/models.py`
|
||||
- `Order` -> `features/payment/models.py`
|
||||
|
||||
所有模型共享同一个 `Base = declarative_base()`（放 `core/db.py`），Alembic `env.py` 统一 import 所有 feature models。
|
||||
|
||||
**跨 feature model 引用规则**：auth/models.py 中的 `RefreshToken.user_id` 外键指向 `users.id`（表名级引用），不需要 import User 类。如需 ORM relationship，通过字符串引用 `relationship("User", ...)`。
|
||||
|
||||
---
|
||||
|
||||
## 六、日志统一化（loguru）
|
||||
|
||||
### 6.1 配置
|
||||
|
||||
在 `core/logging.py` 中：
|
||||
|
||||
- 移除标准库 `logging.basicConfig`
|
||||
- 配置 loguru：JSON 格式输出、按级别分文件、rotation/retention
|
||||
- 拦截标准库 logging（`InterceptHandler`），使 uvicorn / sqlalchemy / celery 日志统一走 loguru
|
||||
|
||||
### 6.2 Request Logging Middleware
|
||||
|
||||
新增 middleware，每个请求注入 `request_id`（UUID），通过 loguru 的 `contextualize` 绑定到所有日志：
|
||||
|
||||
```python
|
||||
with logger.contextualize(request_id=request_id, user_id=user_id):
|
||||
response = await call_next(request)
|
||||
```
|
||||
|
||||
### 6.3 Provider 调用日志
|
||||
|
||||
在每个 adapter 中统一记录：provider 名称、操作、耗时、成功/失败、错误分类。
|
||||
|
||||
---
|
||||
|
||||
## 七、全局异常处理
|
||||
|
||||
在 `core/errors.py` 中定义异常体系：
|
||||
|
||||
```python
|
||||
class AppError(Exception):
|
||||
status_code: int
|
||||
error_code: str
|
||||
message: str
|
||||
|
||||
class NotFoundError(AppError): ...
|
||||
class AuthenticationError(AppError): ...
|
||||
class ProviderError(AppError): ...
|
||||
```
|
||||
|
||||
在 `main.py` 注册全局 exception handler，统一返回格式：
|
||||
|
||||
```json
|
||||
{"error_code": "PROVIDER_ERROR", "message": "...", "request_id": "..."}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 八、统一配置
|
||||
|
||||
将散落的 `os.getenv()` 收归 `core/config.py`，使用 pydantic-settings `BaseSettings`：
|
||||
|
||||
```python
|
||||
class Settings(BaseSettings):
|
||||
database_url: str
|
||||
redis_url: str
|
||||
secret_key: str
|
||||
sms_provider: str = "tencent"
|
||||
storage_provider: str = "tencent_cos"
|
||||
image_gen_provider: str = "liblib"
|
||||
# 嵌套配置
|
||||
tencent_sms: TencentSmsConfig
|
||||
tencent_cos: TencentCosConfig
|
||||
...
|
||||
|
||||
model_config = SettingsConfigDict(env_file=".env", env_nested_delimiter="__")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 九、迁移执行策略
|
||||
|
||||
### Phase 0 - 基础设施搭建（无功能变更）
|
||||
|
||||
1. **迁移到 uv**：`uv init`，将 `requirements.txt` 的依赖迁入 `pyproject.toml` 的 `[project.dependencies]`，删除 `requirements.txt`，生成 `uv.lock` 并纳入版本控制
|
||||
2. 创建 `app/core/` 骨架：config, db, logging, errors, security, dependencies, middleware
|
||||
3. 重写 `get_async_db()` 去掉自动 commit，定义 `get_sync_db()` context manager 供 Celery 使用
|
||||
4. 初始化 Alembic：手写空 baseline revision + `stamp head`（不用 autogenerate）
|
||||
5. 集成 loguru，配置 InterceptHandler 拦截标准库 logging
|
||||
6. 添加 request_id middleware
|
||||
7. 定义全局异常体系（仅统一错误响应格式）
|
||||
|
||||
**验证点**：`uv sync --dev` 能安装全部依赖，应用能以新 core 启动，所有现有 API 不受影响。注意 `get_async_db()` 的 commit 语义变更需要同步修改所有现有 router 中缺少显式 commit 的写操作。
|
||||
|
||||
### Phase 1 - Ports 定义 + Adapters 迁移
|
||||
|
||||
1. 定义 6 个 port protocol（sms, llm, image_gen, storage, tts, asr）
|
||||
2. 将现有 SDK 代码封装为 adapter，实现对应 protocol
|
||||
3. 在 `core/dependencies.py` 中注册 DI factory
|
||||
4. payment adapter 从 `payment/` 直接迁移（已有好的抽象）
|
||||
|
||||
**验证点**：所有 adapter 通过 port protocol 类型检查，现有功能不变。
|
||||
|
||||
### Phase 2 - Feature 模块重构（逐个 feature）
|
||||
|
||||
按依赖关系从外到内逐个迁移，建议顺序：
|
||||
|
||||
1. `content`（faq, legal, home）— 最简单，验证 feature 骨架
|
||||
2. `plan` — 轻量，少依赖
|
||||
3. `quota` — 提升为独立 feature，定义 `QuotaService`
|
||||
4. `user` — 拆出 User model
|
||||
5. `auth` — 拆出 RefreshToken/SmsVerificationCode models，依赖 user models + sms port
|
||||
6. `payment` — 迁移现有 payment/ 模块
|
||||
7. `memoir` — 合并 books/chapters/memoir_state，迁移 processor
|
||||
8. `conversation` — 最复杂，包含 WebSocket 深度拆分（见下）
|
||||
|
||||
**每个 feature 迁移步骤**：
|
||||
|
||||
- 创建 `models.py`（从全局 models.py 拆出）
|
||||
- 创建 `repo.py`（从 router 中提取 DB 操作，禁止 commit）
|
||||
- 创建 `service.py`（从 router 中提取业务逻辑，负责事务提交）
|
||||
- 创建 `schemas.py`（收集 Pydantic schema）
|
||||
- 精简 `router.py`（只保留 HTTP 边界：参数解析、响应状态码、调用 service）
|
||||
- 更新 `main.py` 的 `include_router`
|
||||
|
||||
**conversation WebSocket 拆分详细步骤**：
|
||||
|
||||
当前 `routers/websocket.py`（1067 行）需拆为 `features/conversation/ws/` 子包：
|
||||
|
||||
- `message_types.py`：提取 `MessageType` 枚举（当前 line 32-45）+ 消息构造工具函数
|
||||
- `connection_manager.py`：提取 `ConnectionManager` 类（当前 line 49-120），只保留连接注册/注销/发送，移除 agent/runner 实例持有
|
||||
- `profile_collector.py`：提取 `_get_missing_profile_fields`、`_get_filled_profile_fields`、`_apply_extracted_profile`（当前 line 854-898）和资料收集模式分支（line 917-952）
|
||||
- `quota_guard.py`：封装配额校验逻辑（当前 line 627-637），调用 `quota/service.py`
|
||||
- `pipeline.py`：核心消息处理管道，编排 ASR -> 落库 -> Agent -> TTS -> 响应，依赖上述模块 + ports（asr, tts, llm）
|
||||
- `router.py`：WebSocket endpoint，仅做连接生命周期管理（accept, 主循环, disconnect），将每条消息委托给 pipeline
|
||||
|
||||
**验证点**：每迁移一个 feature，运行测试确保该 feature API 行为不变。
|
||||
|
||||
### Phase 3 - 清理与加固
|
||||
|
||||
1. 删除旧 `routers/`, `services/`, `database/models.py`, `middleware/`, `migrations/`
|
||||
2. 删除 `main.py` 中的 `init_db()` / `create_all()`
|
||||
3. 更新 Dockerfile：改用 uv 安装依赖（`COPY pyproject.toml uv.lock ./` + `uv sync --frozen --no-dev`），或用 `uv export --no-dev > requirements.txt` 导出后继续用 pip
|
||||
4. 更新 docker-compose、deploy.sh 中的路径和启动命令
|
||||
5. 更新 Celery worker 启动命令（`uv run celery ...`）
|
||||
6. 更新所有测试 import 路径
|
||||
7. 补充缺失的测试
|
||||
|
||||
---
|
||||
|
||||
## 十一、硬性规则（写入项目规范）
|
||||
|
||||
1. `**common/` / `utils/` / `shared/` / `helpers/` 禁止创建**：只有 truly cross-cutting 能力进 `core/`，业务 DTO/校验/错误码归 feature
|
||||
2. **router 禁止操作 DB**：所有数据访问通过 repo，所有业务逻辑通过 service
|
||||
3. **service 禁止 import adapter**：只依赖 port protocol，通过 DI 注入。代码中出现 `from adapters.xxx import ...` 即为违规
|
||||
4. **feature 间禁止 import router**：跨 feature 调用通过 service 注入
|
||||
5. **所有 schema 变更走 Alembic**：禁止直接 DDL
|
||||
6. **新增 provider 必须实现 port protocol**：不得在 feature 中直接调用 SDK
|
||||
7. **事务边界：repo 不提交，service 管事务**：`repo.py` 只做 `add/delete/query`，`commit/rollback` 由 service 或 UoW 统一执行。`get_async_db()` 不自动 commit
|
||||
8. **port 边界不可打穿**：service 需要厂商增强能力时，必须扩充 port 或定义第二个窄 port，禁止直接引用 adapter 扩展方法
|
||||
9. **quota 是独立 feature**：conversation、memoir、payment 如需配额检查，通过注入 `QuotaService`，不得直接 import quota 内部函数
|
||||
10. **成功响应不强制包装**：`core/errors.py` 只统一错误响应格式 `{error_code, message, request_id}`，成功响应直接返回 Pydantic model / FileResponse / 原始结构
|
||||
|
||||
559
docs/plans/backend-test-system.md
Normal file
559
docs/plans/backend-test-system.md
Normal file
@@ -0,0 +1,559 @@
|
||||
# Life Echo API Pytest 测试覆盖计划
|
||||
|
||||
本计划假设架构重构计划已完成，项目已迁移至 `app/` 下的 feature-first 结构。
|
||||
|
||||
策略：**删除全部旧 unittest，用 pytest 从零重写。**
|
||||
|
||||
---
|
||||
|
||||
## 一、旧测试处置
|
||||
|
||||
### 删除（13 个 unittest 文件）
|
||||
|
||||
全部删除。重写时参考其覆盖场景：
|
||||
|
||||
| 旧文件 | 覆盖场景 | 重写到 |
|
||||
|--------|----------|--------|
|
||||
| `test_memoir_image_settings.py` | from_env 默认值、无效整数回退 | `unit/adapters/test_image_gen_liblib.py` |
|
||||
| `test_memoir_image_schema.py` | normalize 无效 status、缺字段 | `unit/features/test_memoir_service.py` |
|
||||
| `test_memoir_image_parser.py` | 占位符解析、双大括号 | `unit/features/test_memoir_service.py` |
|
||||
| `test_memoir_image_provider.py` | 签名、提交、轮询、下载 | `unit/adapters/test_image_gen_liblib.py` |
|
||||
| `test_memoir_image_storage.py` | 单例、上传、下载 URL、错误 | `unit/adapters/test_storage_tencent_cos.py` |
|
||||
| `test_memoir_image_prompting.py` | 无 LLM 回退、JSON 解析 | `unit/features/test_memoir_service.py` |
|
||||
| `test_memoir_image_bootstrap.py` | 启用/禁用、动态上限 | `unit/features/test_memoir_service.py` |
|
||||
| `test_generate_chapter_images_task.py` | 锁、重试、幂等、JPEG->PNG | `unit/tasks/test_memoir_tasks.py` |
|
||||
| `test_process_memoir_segments_image_enqueue.py` | markdown JSON、入队、禁用 | `unit/tasks/test_memoir_tasks.py` |
|
||||
| `test_memoir_tasks_redis.py` | Redis 锁复用 | `unit/tasks/test_memoir_tasks.py` |
|
||||
| `test_pdf_service_images.py` | 图片宽高比、签名失败跳过 | `unit/features/test_memoir_service.py` |
|
||||
| `test_chapters_router_images.py` | 签名 URL、畸形资产 | `unit/features/test_memoir_router.py` |
|
||||
| `test_websocket_baseline.py` | 无效 token、文本/音频/转写、结束、乱序、去重、重连 | `unit/features/test_ws_pipeline.py` |
|
||||
|
||||
### 移出（2 个手工脚本）
|
||||
|
||||
- `test_sms_verification.py` -> `scripts/manual/sms_verification.py`
|
||||
- `test_conversation.py` -> `scripts/manual/conversation_e2e.py`
|
||||
|
||||
---
|
||||
|
||||
## 二、测试目录结构
|
||||
|
||||
```
|
||||
api/
|
||||
pyproject.toml # 生产依赖 + dev 依赖 + pytest/coverage 配置
|
||||
uv.lock # lockfile，纳入版本控制
|
||||
Makefile
|
||||
|
||||
tests/
|
||||
conftest.py # unit 层 fixtures
|
||||
factories.py # make_user(), make_conversation() 等
|
||||
fakes/ # port fake 实现（无 __init__.py）
|
||||
sms.py
|
||||
llm.py
|
||||
storage.py
|
||||
image_gen.py
|
||||
tts.py
|
||||
asr.py
|
||||
helpers.py
|
||||
|
||||
unit/
|
||||
core/
|
||||
test_config.py
|
||||
test_security.py
|
||||
test_errors.py
|
||||
test_middleware.py
|
||||
adapters/
|
||||
test_sms_tencent.py
|
||||
test_llm_deepseek.py
|
||||
test_image_gen_liblib.py
|
||||
test_storage_tencent_cos.py
|
||||
test_tts_openai.py
|
||||
test_asr_whisper.py
|
||||
test_asr_tencent.py
|
||||
test_payment_wechat.py
|
||||
test_payment_alipay.py
|
||||
features/
|
||||
test_auth_service.py
|
||||
test_auth_router.py
|
||||
test_conversation_service.py
|
||||
test_conversation_router.py
|
||||
test_ws_connection_manager.py
|
||||
test_ws_message_types.py
|
||||
test_ws_pipeline.py
|
||||
test_ws_profile_collector.py
|
||||
test_ws_quota_guard.py
|
||||
test_memoir_service.py
|
||||
test_memoir_router.py
|
||||
test_payment_service.py
|
||||
test_payment_router.py
|
||||
test_user_service.py
|
||||
test_user_router.py
|
||||
test_quota_service.py
|
||||
test_quota_router.py
|
||||
test_plan_service.py
|
||||
test_plan_router.py
|
||||
test_content_router.py
|
||||
agents/
|
||||
test_conversation_agent.py
|
||||
test_memory_agent.py
|
||||
tasks/
|
||||
test_memoir_tasks.py
|
||||
|
||||
integration/
|
||||
conftest.py
|
||||
test_auth_repo.py
|
||||
test_conversation_repo.py
|
||||
test_memoir_repo.py
|
||||
test_payment_repo.py
|
||||
test_user_repo.py
|
||||
test_quota_repo.py
|
||||
test_auth_service_write.py
|
||||
test_payment_service_write.py
|
||||
test_conversation_service_write.py
|
||||
test_memoir_tasks_db.py
|
||||
test_auth_flow.py
|
||||
test_conversation_flow.py
|
||||
test_payment_flow.py
|
||||
test_alembic.py
|
||||
```
|
||||
|
||||
### 设计原则
|
||||
|
||||
- **无 `__init__.py`**：测试目录不放 `__init__.py`，避免包导入副作用。
|
||||
- **repo 测试全部放 integration/**：不用 SQLite 模拟 Postgres。SQLite 掩盖 JSON/JSONB、约束、排序、方言差异。
|
||||
- **router 测试在 unit/**：override service stub，不跑真实 service/repo/DB。
|
||||
- **service 测试分两层**：unit mock repo 测编排；integration 真实 Postgres 测关键写路径的事务边界。
|
||||
- features 下文件名 flat，减少嵌套。
|
||||
|
||||
---
|
||||
|
||||
## 三、基建代码
|
||||
|
||||
### 3.1 pyproject.toml
|
||||
|
||||
生产依赖通过 `uv add <pkg>` 添加到 `[project.dependencies]`，dev 依赖通过 `uv add --dev <pkg>` 添加到 `[dependency-groups]`。禁止手动编辑依赖列表。
|
||||
|
||||
用 `uv sync --dev` 安装全部依赖（含 dev），`uv.lock` 纳入版本控制。不在 `addopts` 中全局过滤 marker。
|
||||
|
||||
### 3.2 Makefile
|
||||
|
||||
```makefile
|
||||
.PHONY: test test-unit test-integration test-all test-cov
|
||||
|
||||
test: test-unit ## 默认只跑 unit
|
||||
|
||||
test-unit: ## 零外部依赖
|
||||
uv run pytest tests/unit/ -x -q
|
||||
|
||||
test-integration: ## 需要 Postgres + Redis
|
||||
uv run pytest tests/integration/ -x -q
|
||||
|
||||
test-all: ## 全部
|
||||
uv run pytest tests/ -x -q
|
||||
|
||||
test-cov: ## 带覆盖率
|
||||
uv run pytest tests/ --cov=app --cov-report=term-missing
|
||||
```
|
||||
|
||||
### 3.3 tests/conftest.py（unit 测试用）
|
||||
|
||||
关键设计：router 测试 override **service**（不是 port），service 测试直接构造实例注入 fake ports。两者不重叠。
|
||||
|
||||
```python
|
||||
import pytest
|
||||
from unittest.mock import AsyncMock
|
||||
from httpx import ASGITransport, AsyncClient
|
||||
|
||||
|
||||
# ---- Service stubs ----
|
||||
# 每个 feature service 一个 AsyncMock，预设 return_value 匹配方法签名。
|
||||
# 以 auth 为例，其余 feature 同理。
|
||||
@pytest.fixture
|
||||
def auth_service_stub():
|
||||
stub = AsyncMock()
|
||||
stub.register.return_value = {"access_token": "t", "refresh_token": "r"}
|
||||
stub.login.return_value = {"access_token": "t", "refresh_token": "r"}
|
||||
return stub
|
||||
|
||||
@pytest.fixture
|
||||
def conversation_service_stub():
|
||||
return AsyncMock()
|
||||
|
||||
# memoir_service_stub, payment_service_stub, user_service_stub,
|
||||
# quota_service_stub, plan_service_stub 同理，均为 AsyncMock()
|
||||
|
||||
|
||||
# ---- Fake ports（service 层单测用）----
|
||||
# 每个 port 一个 fixture，从 tests/fakes/ 导入。
|
||||
@pytest.fixture
|
||||
def fake_sms():
|
||||
from tests.fakes.sms import InMemorySmsSender
|
||||
return InMemorySmsSender()
|
||||
|
||||
@pytest.fixture
|
||||
def fake_llm():
|
||||
from tests.fakes.llm import StubLLMProvider
|
||||
return StubLLMProvider(default_response="测试回复")
|
||||
|
||||
# fake_storage, fake_image_gen, fake_tts, fake_asr 同理
|
||||
|
||||
|
||||
# ---- Router 测试用 app（override service 层）----
|
||||
@pytest.fixture
|
||||
def app(auth_service_stub, conversation_service_stub, ...):
|
||||
from app.main import create_app
|
||||
from app.features.auth.deps import get_auth_service
|
||||
from app.features.conversation.deps import get_conversation_service
|
||||
# ... 其余 feature deps
|
||||
|
||||
application = create_app()
|
||||
application.dependency_overrides[get_auth_service] = lambda: auth_service_stub
|
||||
application.dependency_overrides[get_conversation_service] = lambda: conversation_service_stub
|
||||
# ... 其余 feature overrides
|
||||
return application
|
||||
|
||||
@pytest.fixture
|
||||
async def client(app):
|
||||
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as c:
|
||||
yield c
|
||||
|
||||
|
||||
# ---- 认证 fixture ----
|
||||
# 必须 override get_current_user 本身，不能只提供 JWT header。
|
||||
# get_current_user 通常会查 DB 加载用户，只发 header 会泄漏真实依赖。
|
||||
@pytest.fixture
|
||||
def stub_current_user():
|
||||
from tests.factories import make_user
|
||||
return make_user(id="test-user-id", subscription_type="free")
|
||||
|
||||
@pytest.fixture
|
||||
def authenticated_app(app, stub_current_user):
|
||||
from app.core.dependencies import get_current_user
|
||||
app.dependency_overrides[get_current_user] = lambda: stub_current_user
|
||||
return app
|
||||
|
||||
@pytest.fixture
|
||||
async def auth_client(authenticated_app):
|
||||
async with AsyncClient(
|
||||
transport=ASGITransport(app=authenticated_app),
|
||||
base_url="http://test",
|
||||
) as c:
|
||||
yield c
|
||||
```
|
||||
|
||||
### 3.4 integration/conftest.py（savepoint 隔离）
|
||||
|
||||
```python
|
||||
import pytest
|
||||
from sqlalchemy.ext.asyncio import create_async_engine, AsyncSession
|
||||
from sqlalchemy import event, create_engine
|
||||
from app.core.db import Base
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
async def pg_engine():
|
||||
import os
|
||||
url = os.environ["TEST_DATABASE_URL"]
|
||||
engine = create_async_engine(url)
|
||||
|
||||
from alembic.config import Config
|
||||
from alembic.command import upgrade
|
||||
cfg = Config("alembic.ini")
|
||||
cfg.set_main_option("sqlalchemy.url", url.replace("+asyncpg", "+psycopg"))
|
||||
upgrade(cfg, "head")
|
||||
|
||||
yield engine
|
||||
await engine.dispose()
|
||||
|
||||
@pytest.fixture
|
||||
async def pg_session(pg_engine):
|
||||
"""
|
||||
连接级事务 + savepoint。
|
||||
service 内部 commit() 变为 savepoint release，
|
||||
测试结束时外层事务 rollback 撤销一切。
|
||||
|
||||
硬性前提：被测代码必须复用此 session，不得自行创建 session。
|
||||
自行创建 session 的代码（如 Celery 任务）不走此 fixture。
|
||||
"""
|
||||
conn = await pg_engine.connect()
|
||||
txn = await conn.begin()
|
||||
session = AsyncSession(bind=conn)
|
||||
|
||||
nested = await conn.begin_nested()
|
||||
|
||||
@event.listens_for(session.sync_session, "after_transaction_end")
|
||||
def restart_savepoint(s, transaction):
|
||||
nonlocal nested
|
||||
if transaction.nested and not transaction._parent.nested:
|
||||
nested = conn.sync_connection.begin_nested()
|
||||
|
||||
yield session
|
||||
|
||||
await session.close()
|
||||
await txn.rollback()
|
||||
await conn.close()
|
||||
|
||||
# ---- Celery 任务用同步 session（不走 savepoint，用 truncate 清理）----
|
||||
@pytest.fixture
|
||||
def sync_session(pg_engine):
|
||||
import os
|
||||
sync_url = os.environ["TEST_DATABASE_URL"].replace("+asyncpg", "+psycopg")
|
||||
sync_engine = create_engine(sync_url)
|
||||
from sqlalchemy.orm import Session
|
||||
with Session(sync_engine) as session:
|
||||
yield session
|
||||
with sync_engine.begin() as conn:
|
||||
for table in reversed(Base.metadata.sorted_tables):
|
||||
conn.execute(table.delete())
|
||||
|
||||
# ---- Integration app（注入 pg_session + fake ports）----
|
||||
@pytest.fixture
|
||||
async def integration_app(pg_session, fake_sms, fake_llm, fake_storage,
|
||||
fake_image_gen, fake_tts, fake_asr):
|
||||
from app.main import create_app
|
||||
from app.core.db import get_async_db
|
||||
from app.core.dependencies import (
|
||||
get_sms_sender, get_llm_provider, get_storage,
|
||||
get_image_generator, get_tts_provider, get_asr_provider,
|
||||
)
|
||||
application = create_app()
|
||||
application.dependency_overrides[get_async_db] = lambda: pg_session
|
||||
application.dependency_overrides[get_sms_sender] = lambda: fake_sms
|
||||
application.dependency_overrides[get_llm_provider] = lambda: fake_llm
|
||||
application.dependency_overrides[get_storage] = lambda: fake_storage
|
||||
application.dependency_overrides[get_image_generator] = lambda: fake_image_gen
|
||||
application.dependency_overrides[get_tts_provider] = lambda: fake_tts
|
||||
application.dependency_overrides[get_asr_provider] = lambda: fake_asr
|
||||
yield application
|
||||
application.dependency_overrides.clear()
|
||||
|
||||
@pytest.fixture
|
||||
async def integration_client(integration_app):
|
||||
async with AsyncClient(
|
||||
transport=ASGITransport(app=integration_app),
|
||||
base_url="http://test",
|
||||
) as c:
|
||||
yield c
|
||||
```
|
||||
|
||||
### 3.5 factories.py
|
||||
|
||||
不使用 factory-boy（`SQLAlchemyModelFactory` 默认为同步 Session，和 AsyncSession 配合需大量封装）。用简单同步 helper 函数：
|
||||
|
||||
```python
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
from app.features.user.models import User
|
||||
from app.features.conversation.models import Conversation
|
||||
|
||||
_seq = 0
|
||||
|
||||
def make_user(**overrides) -> User:
|
||||
global _seq; _seq += 1
|
||||
defaults = dict(
|
||||
id=str(uuid.uuid4()),
|
||||
phone=f"1380000{_seq:04d}",
|
||||
nickname=f"用户{_seq}",
|
||||
subscription_type="free",
|
||||
created_at=datetime.now(timezone.utc),
|
||||
)
|
||||
defaults.update(overrides)
|
||||
return User(**defaults)
|
||||
|
||||
# make_conversation, make_segment, make_book, make_chapter,
|
||||
# make_memoir_state, make_order, make_refresh_token, make_sms_code 同理。
|
||||
# 同步构造 model 实例，integration 测试中 session.add() + await session.flush() 入库。
|
||||
```
|
||||
|
||||
### 3.6 Fake Port 实现
|
||||
|
||||
`tests/fakes/` 下每个 port 一个文件，无 `__init__.py`，实现完整 Protocol 签名：
|
||||
|
||||
- **sms.py** -- `InMemorySmsSender`：`sent: list` 记录历史，`should_fail` 可切换失败模式
|
||||
- **llm.py** -- `StubLLMProvider`：`default_response`，`responses: list`（按序消耗），`call_count`
|
||||
- **storage.py** -- `InMemoryStorage`：`dict[str, bytes]` 后端
|
||||
- **image_gen.py** -- `StubImageGenerator`：`generate()` 返回预设结果，`check_status()` 按调用次数切换
|
||||
- **tts.py** -- `StubTTSProvider`：返回固定 bytes
|
||||
- **asr.py** -- `StubASRProvider`：`default_text`，空串模拟转写失败
|
||||
|
||||
---
|
||||
|
||||
## 四、分层测试策略
|
||||
|
||||
### 4.1 Service unit 测试（unit/features/test_*_service.py）
|
||||
|
||||
mock repo + fake ports，测编排逻辑和错误分支。每个 feature 预计 10-25 case。
|
||||
|
||||
```python
|
||||
# 示例：展示 service unit test 的边界——mock repo，不碰 DB
|
||||
async def test_auth_register_duplicate_phone(fake_sms):
|
||||
repo = AsyncMock()
|
||||
repo.find_by_phone.return_value = make_user(phone="13800000001")
|
||||
|
||||
service = AuthService(repo=repo, sms=fake_sms)
|
||||
with pytest.raises(ConflictError):
|
||||
await service.register(RegisterRequest(phone="13800000001", ...))
|
||||
```
|
||||
|
||||
### 4.2 Service integration 测试（integration/test_*_service_write.py）
|
||||
|
||||
真实 Postgres（savepoint 隔离）+ 真实 repo + fake ports。**只覆盖关键写路径**的事务边界和 ORM 状态持久化：
|
||||
|
||||
- **auth register**：User + RefreshToken 在同一事务创建，失败时两者都不残留
|
||||
- **payment callback**：订单状态变更 + 用户订阅升级在同一事务
|
||||
- **conversation end**：段落状态标记 + Celery 任务提交边界
|
||||
|
||||
```python
|
||||
# 示例：展示与 unit test 的关键区别——验证 DB 中真实存在数据
|
||||
@pytest.mark.integration
|
||||
async def test_register_creates_user_and_token_atomically(pg_session, fake_sms):
|
||||
repo = AuthRepo(pg_session)
|
||||
service = AuthService(repo=repo, sms=fake_sms)
|
||||
|
||||
result = await service.register(RegisterRequest(phone="13800000001", ...))
|
||||
|
||||
user = await pg_session.get(User, result.user_id)
|
||||
assert user is not None
|
||||
assert user.phone == "13800000001"
|
||||
|
||||
tokens = await repo.find_refresh_tokens_by_user(user.id)
|
||||
assert len(tokens) == 1
|
||||
```
|
||||
|
||||
每个 feature 写路径预计 3-8 case，总计约 15-25 case。
|
||||
|
||||
### 4.3 Router unit 测试（unit/features/test_*_router.py）
|
||||
|
||||
override service stub + get_current_user，只验证 HTTP 契约。与 service 测试不重叠：router 不验证"重复手机号返回什么"，只验证"service 抛 ConflictError 时 router 返回 409"。
|
||||
|
||||
```python
|
||||
# 示例：展示 router test 的边界——验证 HTTP 契约，不碰 service 内部
|
||||
async def test_register_calls_service(client, auth_service_stub):
|
||||
resp = await client.post("/api/auth/register", json={"phone": "13800000001", ...})
|
||||
assert resp.status_code == 201
|
||||
auth_service_stub.register.assert_called_once()
|
||||
|
||||
async def test_profile_requires_auth(client):
|
||||
resp = await client.get("/api/user/profile")
|
||||
assert resp.status_code == 401
|
||||
```
|
||||
|
||||
每个 feature 预计 8-20 case。
|
||||
|
||||
### 4.4 Repo integration 测试（integration/test_*_repo.py）
|
||||
|
||||
真实 Postgres（savepoint 隔离），factory helper 构造数据。不用 SQLite。
|
||||
|
||||
```python
|
||||
# 示例：展示 repo test 的边界——真实 Postgres，验证 SQL 正确性
|
||||
@pytest.mark.integration
|
||||
async def test_find_refresh_token(pg_session):
|
||||
user = make_user()
|
||||
pg_session.add(user)
|
||||
token = make_refresh_token(user_id=user.id, token="abc123")
|
||||
pg_session.add(token)
|
||||
await pg_session.flush()
|
||||
|
||||
repo = AuthRepo(pg_session)
|
||||
found = await repo.find_refresh_token("abc123")
|
||||
assert found is not None
|
||||
assert found.user_id == user.id
|
||||
```
|
||||
|
||||
每个 feature 预计 5-15 case。
|
||||
|
||||
### 4.5 Adapter unit 测试（unit/adapters/）
|
||||
|
||||
`respx` mock httpx / `unittest.mock.patch` mock SDK client。每个 adapter 至少：成功路径 + 网络超时 + rate limit + 响应格式异常。每个 adapter 预计 8-15 case。
|
||||
|
||||
### 4.6 WebSocket unit 测试（unit/features/test_ws_*.py）
|
||||
|
||||
5 个文件对应 ws/ 子包 5 个模块：
|
||||
|
||||
- **test_ws_connection_manager.py**：注册/注销、并发同 ID、断开容错（6-8 case）
|
||||
- **test_ws_message_types.py**：枚举完整性、序列化（5-6 case）
|
||||
- **test_ws_pipeline.py**：文本->Agent->落库、音频->ASR->Agent->TTS、无效 token、ASR 失败降级、乱序聚合、幂等去重、结束->整理、重连（15-20 case）
|
||||
- **test_ws_profile_collector.py**：缺失检测、提取、已满跳过（5-8 case）
|
||||
- **test_ws_quota_guard.py**：配额内通过、耗尽拒绝、plan 差异（5-8 case）
|
||||
|
||||
### 4.7 Agents unit 测试
|
||||
|
||||
注入 StubLLMProvider，验证 prompt 构建、响应解析、异常降级。每个 agent 8-12 case。
|
||||
|
||||
### 4.8 Celery Tasks 测试——两层
|
||||
|
||||
**Unit**（unit/tasks/test_memoir_tasks.py）：mock repo/service 测编排逻辑。20-25 case。
|
||||
|
||||
**Integration**（integration/test_memoir_tasks_db.py）：真实 Postgres + fake ports，验证 DB 状态机持久化。纯 mock 测不到 Chapter 状态转换、segment is_processed 标记、image_assets JSON 持久化。
|
||||
|
||||
由于 Celery 任务使用同步 Session（不经过 DI），savepoint 隔离不住，改用 truncate 策略（见 `integration/conftest.py` 中的 `sync_session` fixture）。5-10 case，只覆盖关键状态转换。
|
||||
|
||||
---
|
||||
|
||||
## 五、端到端流程 + Alembic
|
||||
|
||||
### 端到端流程（integration/）
|
||||
|
||||
- **test_auth_flow.py**：注册 -> 登录 -> 获取 profile -> 刷新 -> 登出 -> 旧 token 失效
|
||||
- **test_conversation_flow.py**：创建对话 -> 发消息 -> Agent 回复 -> 消息列表 -> 结束 -> 触发整理
|
||||
- **test_payment_flow.py**：查看计划 -> 创建订单 -> 模拟回调 -> 验证订阅类型变更
|
||||
|
||||
### Alembic 测试
|
||||
|
||||
Forward-only smoke，不做 downgrade（业务 migration 不一定可逆）：
|
||||
- `test_alembic_upgrade_clean_db`：空库 upgrade head 无报错
|
||||
- `test_alembic_current_at_head`：验证测试库 current == head
|
||||
|
||||
---
|
||||
|
||||
## 六、CI 集成
|
||||
|
||||
```yaml
|
||||
jobs:
|
||||
test:
|
||||
name: Run Tests
|
||||
runs-on: ubuntu-latest
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16
|
||||
env: { POSTGRES_DB: life_echo_test, POSTGRES_PASSWORD: test }
|
||||
ports: ["5432:5432"]
|
||||
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
|
||||
redis:
|
||||
image: redis:7
|
||||
ports: ["6379:6379"]
|
||||
defaults:
|
||||
run:
|
||||
working-directory: api
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: astral-sh/setup-uv@v4
|
||||
- run: uv sync --dev
|
||||
- name: Unit tests
|
||||
run: make test-unit
|
||||
- name: Integration tests
|
||||
run: make test-integration
|
||||
env:
|
||||
TEST_DATABASE_URL: postgresql+asyncpg://postgres:test@localhost:5432/life_echo_test
|
||||
REDIS_URL: redis://localhost:6379/0
|
||||
- uses: codecov/codecov-action@v4
|
||||
|
||||
build-and-push:
|
||||
needs: test
|
||||
# ... 现有 build 步骤 ...
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 七、测试规则
|
||||
|
||||
1. **新代码必须附带测试**，PR 不通过无测试的业务变更
|
||||
2. **全部使用 pytest 函数 + fixture 风格**
|
||||
3. **repo 测试只在 integration/ 中跑 Postgres**，不用 SQLite
|
||||
4. **service unit 测试 mock repo + fake ports**；关键写路径补 service integration 测试（真实 Postgres + fake ports）
|
||||
5. **router 测试 override service stub + get_current_user**，只验证 HTTP 契约
|
||||
6. **adapter 测试 mock 外部调用**，至少成功 + 2 种错误场景
|
||||
7. **每个 test function 状态独立**（unit 无 DB；integration 用 savepoint 隔离）
|
||||
8. **savepoint 硬性前提：被测代码必须复用注入的 session，不得自行创建**。Celery 任务用独立测试库 + truncate
|
||||
9. **integration 标记 `@pytest.mark.integration`**
|
||||
10. **默认命令 `make test`**（unit），`make test-all` 全部
|
||||
11. **测试目录无 `__init__.py`**
|
||||
12. **依赖通过 uv 管理**：`uv sync --dev` 安装，`uv add --dev <pkg>` 新增 dev 依赖，禁止 `pip install`
|
||||
Reference in New Issue
Block a user