Files
life-echo/docs/plans/backend-test-system.md

578 lines
21 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Life Echo API Pytest 测试覆盖计划
本计划假设架构重构计划已完成,项目已迁移至 `app/` 下的 feature-first 结构。
策略:**删除全部旧 unittest用 pytest 从零重写。**
---
## 一、旧测试处置
### 删除13 个 unittest 文件)
全部删除。重写时参考其覆盖场景:
| 旧文件 | 覆盖场景 | 重写到 |
|--------|----------|--------|
| `test_memoir_image_settings.py` | from_env 默认值、无效整数回退 | `unit/adapters/test_image_gen_liblib.py` |
| `test_memoir_image_schema.py` | normalize 无效 status、缺字段 | `unit/features/test_memoir_service.py` |
| `test_memoir_image_parser.py` | 占位符解析、双大括号 | `unit/features/test_memoir_service.py` |
| `test_memoir_image_provider.py` | 签名、提交、轮询、下载 | `unit/adapters/test_image_gen_liblib.py` |
| `test_memoir_image_storage.py` | 单例、上传、下载 URL、错误 | `unit/adapters/test_storage_tencent_cos.py` |
| `test_memoir_image_prompting.py` | 无 LLM 回退、JSON 解析 | `unit/features/test_memoir_service.py` |
| `test_memoir_image_bootstrap.py` | 启用/禁用、动态上限 | `unit/features/test_memoir_service.py` |
| `test_generate_chapter_images_task.py` | 锁、重试、幂等、JPEG->PNG | `unit/tasks/test_memoir_tasks.py` |
| `test_process_memoir_segments_image_enqueue.py` | markdown JSON、入队、禁用 | `unit/tasks/test_memoir_tasks.py` |
| `test_memoir_tasks_redis.py` | Redis 锁复用 | `unit/tasks/test_memoir_tasks.py` |
| `test_pdf_service_images.py` | 图片宽高比、签名失败跳过 | `unit/features/test_memoir_service.py` |
| `test_chapters_router_images.py` | 签名 URL、畸形资产 | `unit/features/test_memoir_router.py` |
| `test_websocket_baseline.py` | 无效 token、文本/音频/转写、结束、乱序、去重、重连 | `unit/features/test_ws_pipeline.py` |
### 移出2 个手工脚本)
- `test_sms_verification.py` -> `scripts/manual/sms_verification.py`
- `test_conversation.py` -> `scripts/manual/conversation_e2e.py`
---
## 二、测试目录结构
```
api/
pyproject.toml # 生产依赖 + dev 依赖 + pytest/coverage 配置
uv.lock # lockfile纳入版本控制
Makefile
tests/
conftest.py # unit 层 fixtures
factories.py # make_user(), make_conversation() 等
fakes/ # port fake 实现(无 __init__.py
sms.py
llm.py
storage.py
image_gen.py
tts.py
asr.py
helpers.py
unit/
core/
test_config.py
test_security.py
test_errors.py
test_middleware.py
adapters/
test_sms_tencent.py
test_llm_deepseek.py
test_image_gen_liblib.py
test_storage_tencent_cos.py
test_tts_openai.py
test_asr_whisper.py
test_asr_tencent.py
test_payment_wechat.py
test_payment_alipay.py
features/
test_auth_service.py
test_auth_router.py
test_conversation_service.py
test_conversation_router.py
test_ws_connection_manager.py
test_ws_message_types.py
test_ws_pipeline.py
test_ws_profile_collector.py
test_ws_quota_guard.py
test_memoir_service.py
test_memoir_router.py
test_payment_service.py
test_payment_router.py
test_user_service.py
test_user_router.py
test_quota_service.py
test_quota_router.py
test_plan_service.py
test_plan_router.py
test_content_router.py
agents/
test_conversation_agent.py
test_memory_agent.py
tasks/
test_memoir_tasks.py
integration/
conftest.py
test_auth_repo.py
test_conversation_repo.py
test_memoir_repo.py
test_payment_repo.py
test_user_repo.py
test_quota_repo.py
test_auth_service_write.py
test_payment_service_write.py
test_conversation_service_write.py
test_memoir_tasks_db.py
test_auth_flow.py
test_conversation_flow.py
test_payment_flow.py
test_alembic.py
```
### 设计原则
- **无 `__init__.py`**:测试目录不放 `__init__.py`,避免包导入副作用。
- **repo 测试全部放 integration/**:不用 SQLite 模拟 Postgres。SQLite 掩盖 JSON/JSONB、约束、排序、方言差异。
- **router 测试在 unit/**override service stub不跑真实 service/repo/DB。
- **service 测试分两层**unit mock repo 测编排integration 真实 Postgres 测关键写路径的事务边界。
- features 下文件名 flat减少嵌套。
---
## 三、基建代码
### 3.1 pyproject.toml
生产依赖通过 `uv add <pkg>` 添加到 `[project.dependencies]`dev 依赖通过 `uv add --dev <pkg>` 添加到 `[dependency-groups]`。禁止手动编辑依赖列表。
`uv sync --dev` 安装全部依赖(含 dev`uv.lock` 纳入版本控制。不在 `addopts` 中全局过滤 marker。
当前开发工具链职责约定:
- `ruff`:负责 `lint` + `format`
- `pytest`:统一测试入口
- `pytest-asyncio`:负责 `async def` 测试和异步 fixture
- `pytest-cov`:负责覆盖率统计
- `pyright`:负责类型检查,但不替代测试和 `lint`
### 3.2 Makefile
```makefile
.PHONY: lint format typecheck test test-unit test-integration test-all test-cov
lint: ## 代码检查
uv run ruff check .
format: ## 代码格式化
uv run ruff format .
typecheck: ## 类型检查
uv run pyright
test: test-unit ## 默认只跑 unit
test-unit: ## 零外部依赖
uv run pytest tests/unit/ -x -q
test-integration: ## 需要 Postgres + Redis
uv run pytest tests/integration/ -x -q
test-all: ## 全部
uv run pytest tests/ -x -q
test-cov: ## 带覆盖率
uv run pytest tests/ --cov=app --cov-report=term-missing
```
### 3.3 tests/conftest.pyunit 测试用)
关键设计router 测试 override **service**(不是 portservice 测试直接构造实例注入 fake ports。两者不重叠。
```python
import pytest
from unittest.mock import AsyncMock
from httpx import ASGITransport, AsyncClient
# ---- Service stubs ----
# 每个 feature service 一个 AsyncMock预设 return_value 匹配方法签名。
# 以 auth 为例,其余 feature 同理。
@pytest.fixture
def auth_service_stub():
stub = AsyncMock()
stub.register.return_value = {"access_token": "t", "refresh_token": "r"}
stub.login.return_value = {"access_token": "t", "refresh_token": "r"}
return stub
@pytest.fixture
def conversation_service_stub():
return AsyncMock()
# memoir_service_stub, payment_service_stub, user_service_stub,
# quota_service_stub, plan_service_stub 同理,均为 AsyncMock()
# ---- Fake portsservice 层单测用)----
# 每个 port 一个 fixture从 tests/fakes/ 导入。
@pytest.fixture
def fake_sms():
from tests.fakes.sms import InMemorySmsSender
return InMemorySmsSender()
@pytest.fixture
def fake_llm():
from tests.fakes.llm import StubLLMProvider
return StubLLMProvider(default_response="测试回复")
# fake_storage, fake_image_gen, fake_tts, fake_asr 同理
# ---- Router 测试用 appoverride service 层)----
@pytest.fixture
def app(auth_service_stub, conversation_service_stub, ...):
from app.main import create_app
from app.features.auth.deps import get_auth_service
from app.features.conversation.deps import get_conversation_service
# ... 其余 feature deps
application = create_app()
application.dependency_overrides[get_auth_service] = lambda: auth_service_stub
application.dependency_overrides[get_conversation_service] = lambda: conversation_service_stub
# ... 其余 feature overrides
return application
@pytest.fixture
async def client(app):
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as c:
yield c
# ---- 认证 fixture ----
# 必须 override get_current_user 本身,不能只提供 JWT header。
# get_current_user 通常会查 DB 加载用户,只发 header 会泄漏真实依赖。
@pytest.fixture
def stub_current_user():
from tests.factories import make_user
return make_user(id="test-user-id", subscription_type="free")
@pytest.fixture
def authenticated_app(app, stub_current_user):
from app.core.dependencies import get_current_user
app.dependency_overrides[get_current_user] = lambda: stub_current_user
return app
@pytest.fixture
async def auth_client(authenticated_app):
async with AsyncClient(
transport=ASGITransport(app=authenticated_app),
base_url="http://test",
) as c:
yield c
```
### 3.4 integration/conftest.pysavepoint 隔离)
```python
import pytest
from sqlalchemy.ext.asyncio import create_async_engine, AsyncSession
from sqlalchemy import event, create_engine
from app.core.db import Base
@pytest.fixture(scope="session")
async def pg_engine():
import os
url = os.environ["TEST_DATABASE_URL"]
engine = create_async_engine(url)
from alembic.config import Config
from alembic.command import upgrade
cfg = Config("alembic.ini")
cfg.set_main_option("sqlalchemy.url", url.replace("+asyncpg", "+psycopg"))
upgrade(cfg, "head")
yield engine
await engine.dispose()
@pytest.fixture
async def pg_session(pg_engine):
"""
连接级事务 + savepoint。
service 内部 commit() 变为 savepoint release
测试结束时外层事务 rollback 撤销一切。
硬性前提:被测代码必须复用此 session不得自行创建 session。
自行创建 session 的代码(如 Celery 任务)不走此 fixture。
"""
conn = await pg_engine.connect()
txn = await conn.begin()
session = AsyncSession(bind=conn)
nested = await conn.begin_nested()
@event.listens_for(session.sync_session, "after_transaction_end")
def restart_savepoint(s, transaction):
nonlocal nested
if transaction.nested and not transaction._parent.nested:
nested = conn.sync_connection.begin_nested()
yield session
await session.close()
await txn.rollback()
await conn.close()
# ---- Celery 任务用同步 session不走 savepoint用 truncate 清理)----
@pytest.fixture
def sync_session(pg_engine):
import os
sync_url = os.environ["TEST_DATABASE_URL"].replace("+asyncpg", "+psycopg")
sync_engine = create_engine(sync_url)
from sqlalchemy.orm import Session
with Session(sync_engine) as session:
yield session
with sync_engine.begin() as conn:
for table in reversed(Base.metadata.sorted_tables):
conn.execute(table.delete())
# ---- Integration app注入 pg_session + fake ports----
@pytest.fixture
async def integration_app(pg_session, fake_sms, fake_llm, fake_storage,
fake_image_gen, fake_tts, fake_asr):
from app.main import create_app
from app.core.db import get_async_db
from app.core.dependencies import (
get_sms_sender, get_llm_provider, get_storage,
get_image_generator, get_tts_provider, get_asr_provider,
)
application = create_app()
application.dependency_overrides[get_async_db] = lambda: pg_session
application.dependency_overrides[get_sms_sender] = lambda: fake_sms
application.dependency_overrides[get_llm_provider] = lambda: fake_llm
application.dependency_overrides[get_storage] = lambda: fake_storage
application.dependency_overrides[get_image_generator] = lambda: fake_image_gen
application.dependency_overrides[get_tts_provider] = lambda: fake_tts
application.dependency_overrides[get_asr_provider] = lambda: fake_asr
yield application
application.dependency_overrides.clear()
@pytest.fixture
async def integration_client(integration_app):
async with AsyncClient(
transport=ASGITransport(app=integration_app),
base_url="http://test",
) as c:
yield c
```
### 3.5 factories.py
不使用 factory-boy`SQLAlchemyModelFactory` 默认为同步 Session和 AsyncSession 配合需大量封装)。用简单同步 helper 函数:
```python
import uuid
from datetime import datetime, timezone
from app.features.user.models import User
from app.features.conversation.models import Conversation
_seq = 0
def make_user(**overrides) -> User:
global _seq; _seq += 1
defaults = dict(
id=str(uuid.uuid4()),
phone=f"1380000{_seq:04d}",
nickname=f"用户{_seq}",
subscription_type="free",
created_at=datetime.now(timezone.utc),
)
defaults.update(overrides)
return User(**defaults)
# make_conversation, make_segment, make_book, make_chapter,
# make_memoir_state, make_order, make_refresh_token, make_sms_code 同理。
# 同步构造 model 实例integration 测试中 session.add() + await session.flush() 入库。
```
### 3.6 Fake Port 实现
`tests/fakes/` 下每个 port 一个文件,无 `__init__.py`,实现完整 Protocol 签名:
- **sms.py** -- `InMemorySmsSender``sent: list` 记录历史,`should_fail` 可切换失败模式
- **llm.py** -- `StubLLMProvider``default_response``responses: list`(按序消耗),`call_count`
- **storage.py** -- `InMemoryStorage``dict[str, bytes]` 后端
- **image_gen.py** -- `StubImageGenerator``generate()` 返回预设结果,`check_status()` 按调用次数切换
- **tts.py** -- `StubTTSProvider`:返回固定 bytes
- **asr.py** -- `StubASRProvider``default_text`,空串模拟转写失败
---
## 四、分层测试策略
### 4.1 Service unit 测试unit/features/test_*_service.py
mock repo + fake ports测编排逻辑和错误分支。每个 feature 预计 10-25 case。
```python
# 示例:展示 service unit test 的边界——mock repo不碰 DB
async def test_auth_register_duplicate_phone(fake_sms):
repo = AsyncMock()
repo.find_by_phone.return_value = make_user(phone="13800000001")
service = AuthService(repo=repo, sms=fake_sms)
with pytest.raises(ConflictError):
await service.register(RegisterRequest(phone="13800000001", ...))
```
### 4.2 Service integration 测试integration/test_*_service_write.py
真实 Postgressavepoint 隔离)+ 真实 repo + fake ports。**只覆盖关键写路径**的事务边界和 ORM 状态持久化:
- **auth register**User + RefreshToken 在同一事务创建,失败时两者都不残留
- **payment callback**:订单状态变更 + 用户订阅升级在同一事务
- **conversation end**:段落状态标记 + Celery 任务提交边界
```python
# 示例:展示与 unit test 的关键区别——验证 DB 中真实存在数据
@pytest.mark.integration
async def test_register_creates_user_and_token_atomically(pg_session, fake_sms):
repo = AuthRepo(pg_session)
service = AuthService(repo=repo, sms=fake_sms)
result = await service.register(RegisterRequest(phone="13800000001", ...))
user = await pg_session.get(User, result.user_id)
assert user is not None
assert user.phone == "13800000001"
tokens = await repo.find_refresh_tokens_by_user(user.id)
assert len(tokens) == 1
```
每个 feature 写路径预计 3-8 case总计约 15-25 case。
### 4.3 Router unit 测试unit/features/test_*_router.py
override service stub + get_current_user只验证 HTTP 契约。与 service 测试不重叠router 不验证"重复手机号返回什么",只验证"service 抛 ConflictError 时 router 返回 409"。
```python
# 示例:展示 router test 的边界——验证 HTTP 契约,不碰 service 内部
async def test_register_calls_service(client, auth_service_stub):
resp = await client.post("/api/auth/register", json={"phone": "13800000001", ...})
assert resp.status_code == 201
auth_service_stub.register.assert_called_once()
async def test_profile_requires_auth(client):
resp = await client.get("/api/user/profile")
assert resp.status_code == 401
```
每个 feature 预计 8-20 case。
### 4.4 Repo integration 测试integration/test_*_repo.py
真实 Postgressavepoint 隔离factory helper 构造数据。不用 SQLite。
```python
# 示例:展示 repo test 的边界——真实 Postgres验证 SQL 正确性
@pytest.mark.integration
async def test_find_refresh_token(pg_session):
user = make_user()
pg_session.add(user)
token = make_refresh_token(user_id=user.id, token="abc123")
pg_session.add(token)
await pg_session.flush()
repo = AuthRepo(pg_session)
found = await repo.find_refresh_token("abc123")
assert found is not None
assert found.user_id == user.id
```
每个 feature 预计 5-15 case。
### 4.5 Adapter unit 测试unit/adapters/
`respx` mock httpx / `unittest.mock.patch` mock SDK client。每个 adapter 至少:成功路径 + 网络超时 + rate limit + 响应格式异常。每个 adapter 预计 8-15 case。
### 4.6 WebSocket unit 测试unit/features/test_ws_*.py
5 个文件对应 ws/ 子包 5 个模块:
- **test_ws_connection_manager.py**:注册/注销、并发同 ID、断开容错6-8 case
- **test_ws_message_types.py**枚举完整性、序列化5-6 case
- **test_ws_pipeline.py**:文本->Agent->落库、音频->ASR->Agent->TTS、无效 token、ASR 失败降级、乱序聚合、幂等去重、结束->整理、重连15-20 case
- **test_ws_profile_collector.py**缺失检测、提取、已满跳过5-8 case
- **test_ws_quota_guard.py**配额内通过、耗尽拒绝、plan 差异5-8 case
### 4.7 Agents unit 测试
注入 StubLLMProvider验证 prompt 构建、响应解析、异常降级。每个 agent 8-12 case。
### 4.8 Celery Tasks 测试——两层
**Unit**unit/tasks/test_memoir_tasks.pymock repo/service 测编排逻辑。20-25 case。
**Integration**integration/test_memoir_tasks_db.py真实 Postgres + fake ports验证 DB 状态机持久化。纯 mock 测不到 Chapter 状态转换、segment is_processed 标记、image_assets JSON 持久化。
由于 Celery 任务使用同步 Session不经过 DIsavepoint 隔离不住,改用 truncate 策略(见 `integration/conftest.py` 中的 `sync_session` fixture。5-10 case只覆盖关键状态转换。
---
## 五、端到端流程 + Alembic
### 端到端流程integration/
- **test_auth_flow.py**:注册 -> 登录 -> 获取 profile -> 刷新 -> 登出 -> 旧 token 失效
- **test_conversation_flow.py**:创建对话 -> 发消息 -> Agent 回复 -> 消息列表 -> 结束 -> 触发整理
- **test_payment_flow.py**:查看计划 -> 创建订单 -> 模拟回调 -> 验证订阅类型变更
### Alembic 测试
Forward-only smoke不做 downgrade业务 migration 不一定可逆):
- `test_alembic_upgrade_clean_db`:空库 upgrade head 无报错
- `test_alembic_current_at_head`:验证测试库 current == head
---
## 六、CI 集成
```yaml
jobs:
test:
name: Run Tests
runs-on: ubuntu-latest
services:
postgres:
image: postgres:16
env: { POSTGRES_DB: life_echo_test, POSTGRES_PASSWORD: test }
ports: ["5432:5432"]
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
redis:
image: redis:7
ports: ["6379:6379"]
defaults:
run:
working-directory: api
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v4
- run: uv sync --dev
- name: Unit tests
run: make test-unit
- name: Integration tests
run: make test-integration
env:
TEST_DATABASE_URL: postgresql+asyncpg://postgres:test@localhost:5432/life_echo_test
REDIS_URL: redis://localhost:6379/0
- uses: codecov/codecov-action@v4
build-and-push:
needs: test
# ... 现有 build 步骤 ...
```
---
## 七、测试规则
1. **新代码必须附带测试**PR 不通过无测试的业务变更
2. **全部使用 pytest 函数 + fixture 风格**
3. **repo 测试只在 integration/ 中跑 Postgres**,不用 SQLite
4. **service unit 测试 mock repo + fake ports**;关键写路径补 service integration 测试(真实 Postgres + fake ports
5. **router 测试 override service stub + get_current_user**,只验证 HTTP 契约
6. **adapter 测试 mock 外部调用**,至少成功 + 2 种错误场景
7. **每个 test function 状态独立**unit 无 DBintegration 用 savepoint 隔离)
8. **savepoint 硬性前提:被测代码必须复用注入的 session不得自行创建**。Celery 任务用独立测试库 + truncate
9. **integration 标记 `@pytest.mark.integration`**
10. **默认命令 `make test`**unit`make test-all` 全部
11. **测试目录无 `__init__.py`**
12. **依赖通过 uv 管理**`uv sync --dev` 安装,`uv add --dev <pkg>` 新增 dev 依赖,禁止 `pip install`
13. **类型检查由 `pyright` 负责**:它用于发现静态类型问题,但不能替代 `ruff``pytest`