feat(eval): memoir A/B chapter judging and eval-web parity with dialogue

- Judge baseline excerpt and library chapter separately; build_memoir_compare_summary for gate, nine-dim and leaf deltas.

- Memoir SSE chapter payload: baseline_judge, compare_summary, baseline_judge_error.

- MemoirJudgeOutput: loose score coercion and post-validate clamp; memoir judge prompt caps from settings.

- app-eval-web: two-column MemoirScoreCard layout, MemoirCompareSummary, chapter blocks and CSS.

- Add memoir_compare_summary, log_events, celery_log_context, memoir_pipeline_progress; tests and migration 0014.

- Misc: memory/evidence and enrichment paths, task/orchestrator updates, internal-eval docs, env examples.
This commit is contained in:
Kevin
2026-04-10 10:23:43 +08:00
parent b0251e5b26
commit ac49bc7f23
59 changed files with 4773 additions and 696 deletions

View File

@@ -6,16 +6,26 @@ loguru 统一日志配置 + InterceptHandler 拦截第三方库的标准库 logg
直接走 loguru sink占位符用 **``{}``**(勿用 ``%s``,否则不会插值)。
**禁止**用 ``import logging`` 取业务 logger适配器层与第三方 SDK 除外)。
- **第三方**uvicorn、celery、httpx、langchain 等):仍用标准库 ``logging``,经 ``InterceptHandler`` 汇入 loguru。
默认将 ``celery*``、``httpx``/``httpcore`` 调到 WARNING避免刷屏任务边界见 ``app.tasks.celery_app`` 中 ``event=celery_task_*``。
级别:
- INFO面向运维的稳定摘要。
- DEBUG可含完整上下文、用户内容;仅受控环境长期开启
- INFO面向运维的稳定摘要(生产/预发推荐长期保持)
- DEBUG可含 prompt/响应预览或哈希;会显著增噪与体积,仅短时排障;可与 ``AGENT_LOG_MAX_CHARS`` / ``AGENT_LOG_PROMPT_MODE`` 配合
由 ``Settings.log_level`` 控制 sink``LOG_LEVEL````LOG_LEVEL=DEBUG`` 时业务 ``logger.debug`` 可见。
不打开全局 DEBUG 也可设 ``LOG_AGENT_VERBOSE=1`` 查看 Agent 单行耗时与规模(见 ``app.core.agent_logging``)。
**实践说明**:开发/终端用「人类可读」单行格式若上生产聚合ELK、Loki、CloudWatch建议**另加** JSON sink``serialize=True`` 或自定义 ``format``)与现有 stderr 并存,便于检索与关联,而不是在控制台格式里硬塞结构化字段。
Agent / LLM 诊断见 ``app.core.agent_logging````LOG_AGENT_VERBOSE``、``AGENT_LOG_MAX_CHARS`` 见配置说明。
**字段约定(可读性)**
- 机读键用英文 ``snake_case``:优先 ``event=...``,其余 ``key=value`` 空格分隔;与人相关的说明用 ``msg=中文短句``(可含空格),放在行尾或紧邻 ``event`` 后。
- **HTTP**``request_id`` 由中间件 ``contextualize``;业务处可 ``logger.bind(**correlation_bind_kwargs(user_id=..., memoir_correlation_id=...))``(见 ``app.core.log_events``)。
- **Celery**``task_prerun`` 会通过 ``app.core.celery_log_context`` 注入 ``user_id`` / ``correlation_id`` / ``task_id`` 等到 loguru ``extra``(不覆盖已有 ``bind````task_postrun`` 清除,避免串任务。
- **耗时**:业务里程碑的结束行带 ``duration_ms````perf_counter`` × 1000LLM 细粒度见 ``app.core.agent_logging`` 的 ``agent_span`` / ``LOG_AGENT_VERBOSE``。
- **级别**INFO=里程碑与任务起止DEBUG=体积与路径WARNING=可恢复失败与降级。
Agent / LLM 诊断见 ``app.core.agent_logging````LOG_AGENT_VERBOSE``、``AGENT_LOG_MAX_CHARS``、``AGENT_LOG_PROMPT_MODE``、``AGENT_LOG_PROMPT_DEDUP`` 见 ``api/.env.example`` 与 ``Settings``。
"""
from __future__ import annotations
@@ -28,6 +38,11 @@ from typing import TYPE_CHECKING, Any
from loguru import logger
from app.core.config import settings
from app.core.log_events import (
celery_prerun_extras,
correlation_bind_kwargs,
format_log_event,
)
if TYPE_CHECKING:
from loguru import Logger
@@ -94,33 +109,66 @@ def _stdlib_emit_display(log_record: logging.LogRecord) -> tuple[str, int]:
def _stderr_format(record: Any) -> str:
"""控制台 sink 格式:无有效 request_id 时不占一列 ``-``,减少 Celery/Worker 噪声"""
"""控制台 sinkrequest_id / correlation_id / user_id 有值时才显示对应列"""
rid = str(record["extra"].get("request_id") or "").strip()
rid_part = "<dim>{extra[request_id]}</dim> | " if rid and rid != "-" else ""
rid_part = "<dim>rid={extra[request_id]}</dim> | " if rid and rid != "-" else ""
cid = str(record["extra"].get("correlation_id") or "").strip()
cid_part = "<dim>corr={extra[correlation_id]}</dim> | " if cid else ""
uid = str(record["extra"].get("user_id") or "").strip()
uid_part = "<dim>uid={extra[user_id]}</dim> | " if uid else ""
return (
"<green>{time:YYYY-MM-DD HH:mm:ss.SSS}</green> | "
"<level>{level.name: <8}</level> | "
"<cyan>{extra[module]}</cyan>:<cyan>{function}</cyan>:<cyan>{line}</cyan> | "
f"{rid_part}"
f"{rid_part}{cid_part}{uid_part}"
"<level>{message}</level>\n{exception}"
)
def _merge_celery_worker_extra(record: Any) -> None:
"""把 ContextVar 中的 Celery 上下文字段并入本条 loguru 记录(不覆盖已有非空 extra"""
try:
from app.core.celery_log_context import get_celery_log_extras
ctx = get_celery_log_extras()
if not ctx:
return
except Exception:
return
ex = record["extra"]
for k, v in ctx.items():
if not v:
continue
cur = ex.get(k)
if cur is None or str(cur).strip() in ("", "-"):
ex[k] = v
def _apply_third_party_log_levels() -> None:
"""在全局 sink 为 DEBUG/TRACE 时压低 Celery/httpx 噪声;可通过 CELERY_LOG_LEVEL / HTTPX_LOG_LEVEL 覆盖。"""
"""压低 Celery/httpx 框架日志噪声。
根 logger 为 NOTSET 时,子 logger 若也为 NOTSET有效级别会变成 0NOTSETINFO 会全部通过,
因此这里**必须**写死默认级别,不能依赖 NOTSET「继承」。
默认(未设 CELERY_LOG_LEVEL / HTTPX_LOG_LEVEL
- ``LOG_LEVEL`` 为 TRACE/DEBUGCelery→INFOhttpx/httpcore→WARNING
- 否则Celery 与 httpx/httpcore→WARNING保留业务 loguru 与 ``event=celery_task_*`` 摘要)
需要框架原始行时,设置 ``CELERY_LOG_LEVEL=INFO``、``HTTPX_LOG_LEVEL=INFO`` 等。
"""
sink = _sink_min_level()
verbose = sink in ("TRACE", "DEBUG")
# 无效环境变量时的回退:与「未设置变量」分支一致,禁止 NOTSET
default_celery = logging.INFO if verbose else logging.WARNING
default_httpx = logging.WARNING
raw_c = (settings.celery_log_level or "").strip()
if raw_c:
parsed = _parse_stdlib_level(raw_c)
cel_level = (
parsed
if parsed is not None
else (logging.INFO if verbose else logging.NOTSET)
)
cel_level = parsed if parsed is not None else default_celery
else:
cel_level = logging.INFO if verbose else logging.NOTSET
cel_level = default_celery
for name in ("celery", "celery.worker"):
logging.getLogger(name).setLevel(cel_level)
@@ -128,13 +176,9 @@ def _apply_third_party_log_levels() -> None:
raw_h = (settings.httpx_log_level or "").strip()
if raw_h:
parsed = _parse_stdlib_level(raw_h)
httpx_level = (
parsed
if parsed is not None
else (logging.WARNING if verbose else logging.NOTSET)
)
httpx_level = parsed if parsed is not None else default_httpx
else:
httpx_level = logging.WARNING if verbose else logging.NOTSET
httpx_level = default_httpx
for name in ("httpx", "httpcore"):
logging.getLogger(name).setLevel(httpx_level)
@@ -175,6 +219,7 @@ def setup_logging() -> None:
Celery 需 ``worker_hijack_root_logger=False``,否则会覆盖根 logger。
"""
global logger
logger.remove()
logger.add(
@@ -185,7 +230,20 @@ def setup_logging() -> None:
diagnose=False,
)
json_path = (settings.log_json_file or "").strip()
if json_path:
logger.add(
json_path,
level=_sink_min_level(),
serialize=True,
rotation="20 MB",
retention="7 days",
encoding="utf-8",
enqueue=True,
)
logger.configure(extra={"request_id": "-", "module": "-"})
logger = logger.patch(_merge_celery_worker_extra)
# 仅 root 挂 InterceptHandler避免子 logger 与 root 各处理一次导致重复行
root = logging.getLogger()
@@ -201,4 +259,12 @@ def get_logger(name: str) -> Logger:
# 供 middleware 等使用 ``contextualize`` 的同一 loguru 实例(与 get_logger 同源)
__all__ = ["logger", "setup_logging", "get_logger", "InterceptHandler"]
__all__ = [
"logger",
"setup_logging",
"get_logger",
"InterceptHandler",
"format_log_event",
"correlation_bind_kwargs",
"celery_prerun_extras",
]