feat(eval): memoir A/B chapter judging and eval-web parity with dialogue
- Judge baseline excerpt and library chapter separately; build_memoir_compare_summary for gate, nine-dim and leaf deltas. - Memoir SSE chapter payload: baseline_judge, compare_summary, baseline_judge_error. - MemoirJudgeOutput: loose score coercion and post-validate clamp; memoir judge prompt caps from settings. - app-eval-web: two-column MemoirScoreCard layout, MemoirCompareSummary, chapter blocks and CSS. - Add memoir_compare_summary, log_events, celery_log_context, memoir_pipeline_progress; tests and migration 0014. - Misc: memory/evidence and enrichment paths, task/orchestrator updates, internal-eval docs, env examples.
This commit is contained in:
@@ -6,16 +6,26 @@ loguru 统一日志配置 + InterceptHandler 拦截第三方库的标准库 logg
|
||||
直接走 loguru sink;占位符用 **``{}``**(勿用 ``%s``,否则不会插值)。
|
||||
**禁止**用 ``import logging`` 取业务 logger(适配器层与第三方 SDK 除外)。
|
||||
- **第三方**(uvicorn、celery、httpx、langchain 等):仍用标准库 ``logging``,经 ``InterceptHandler`` 汇入 loguru。
|
||||
默认将 ``celery*``、``httpx``/``httpcore`` 调到 WARNING,避免刷屏;任务边界见 ``app.tasks.celery_app`` 中 ``event=celery_task_*``。
|
||||
|
||||
级别:
|
||||
- INFO:面向运维的稳定摘要。
|
||||
- DEBUG:可含完整上下文、用户内容;仅受控环境长期开启。
|
||||
- INFO:面向运维的稳定摘要(生产/预发推荐长期保持)。
|
||||
- DEBUG:可含 prompt/响应预览或哈希;会显著增噪与体积,仅短时排障;可与 ``AGENT_LOG_MAX_CHARS`` / ``AGENT_LOG_PROMPT_MODE`` 配合。
|
||||
|
||||
由 ``Settings.log_level`` 控制 sink(``LOG_LEVEL``);``LOG_LEVEL=DEBUG`` 时业务 ``logger.debug`` 可见。
|
||||
不打开全局 DEBUG 也可设 ``LOG_AGENT_VERBOSE=1`` 查看 Agent 单行耗时与规模(见 ``app.core.agent_logging``)。
|
||||
|
||||
**实践说明**:开发/终端用「人类可读」单行格式;若上生产聚合(ELK、Loki、CloudWatch),建议**另加** JSON sink(``serialize=True`` 或自定义 ``format``)与现有 stderr 并存,便于检索与关联,而不是在控制台格式里硬塞结构化字段。
|
||||
|
||||
Agent / LLM 诊断见 ``app.core.agent_logging``;``LOG_AGENT_VERBOSE``、``AGENT_LOG_MAX_CHARS`` 见配置说明。
|
||||
**字段约定(可读性)**:
|
||||
|
||||
- 机读键用英文 ``snake_case``:优先 ``event=...``,其余 ``key=value`` 空格分隔;与人相关的说明用 ``msg=中文短句``(可含空格),放在行尾或紧邻 ``event`` 后。
|
||||
- **HTTP**:``request_id`` 由中间件 ``contextualize``;业务处可 ``logger.bind(**correlation_bind_kwargs(user_id=..., memoir_correlation_id=...))``(见 ``app.core.log_events``)。
|
||||
- **Celery**:``task_prerun`` 会通过 ``app.core.celery_log_context`` 注入 ``user_id`` / ``correlation_id`` / ``task_id`` 等到 loguru ``extra``(不覆盖已有 ``bind``);``task_postrun`` 清除,避免串任务。
|
||||
- **耗时**:业务里程碑的结束行带 ``duration_ms``(``perf_counter`` × 1000);LLM 细粒度见 ``app.core.agent_logging`` 的 ``agent_span`` / ``LOG_AGENT_VERBOSE``。
|
||||
- **级别**:INFO=里程碑与任务起止;DEBUG=体积与路径;WARNING=可恢复失败与降级。
|
||||
|
||||
Agent / LLM 诊断见 ``app.core.agent_logging``;``LOG_AGENT_VERBOSE``、``AGENT_LOG_MAX_CHARS``、``AGENT_LOG_PROMPT_MODE``、``AGENT_LOG_PROMPT_DEDUP`` 见 ``api/.env.example`` 与 ``Settings``。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -28,6 +38,11 @@ from typing import TYPE_CHECKING, Any
|
||||
from loguru import logger
|
||||
|
||||
from app.core.config import settings
|
||||
from app.core.log_events import (
|
||||
celery_prerun_extras,
|
||||
correlation_bind_kwargs,
|
||||
format_log_event,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from loguru import Logger
|
||||
@@ -94,33 +109,66 @@ def _stdlib_emit_display(log_record: logging.LogRecord) -> tuple[str, int]:
|
||||
|
||||
|
||||
def _stderr_format(record: Any) -> str:
|
||||
"""控制台 sink 格式:无有效 request_id 时不占一列 ``-``,减少 Celery/Worker 噪声。"""
|
||||
"""控制台 sink:request_id / correlation_id / user_id 有值时才显示对应列。"""
|
||||
rid = str(record["extra"].get("request_id") or "").strip()
|
||||
rid_part = "<dim>{extra[request_id]}</dim> | " if rid and rid != "-" else ""
|
||||
rid_part = "<dim>rid={extra[request_id]}</dim> | " if rid and rid != "-" else ""
|
||||
cid = str(record["extra"].get("correlation_id") or "").strip()
|
||||
cid_part = "<dim>corr={extra[correlation_id]}</dim> | " if cid else ""
|
||||
uid = str(record["extra"].get("user_id") or "").strip()
|
||||
uid_part = "<dim>uid={extra[user_id]}</dim> | " if uid else ""
|
||||
return (
|
||||
"<green>{time:YYYY-MM-DD HH:mm:ss.SSS}</green> | "
|
||||
"<level>{level.name: <8}</level> | "
|
||||
"<cyan>{extra[module]}</cyan>:<cyan>{function}</cyan>:<cyan>{line}</cyan> | "
|
||||
f"{rid_part}"
|
||||
f"{rid_part}{cid_part}{uid_part}"
|
||||
"<level>{message}</level>\n{exception}"
|
||||
)
|
||||
|
||||
|
||||
def _merge_celery_worker_extra(record: Any) -> None:
|
||||
"""把 ContextVar 中的 Celery 上下文字段并入本条 loguru 记录(不覆盖已有非空 extra)。"""
|
||||
try:
|
||||
from app.core.celery_log_context import get_celery_log_extras
|
||||
|
||||
ctx = get_celery_log_extras()
|
||||
if not ctx:
|
||||
return
|
||||
except Exception:
|
||||
return
|
||||
ex = record["extra"]
|
||||
for k, v in ctx.items():
|
||||
if not v:
|
||||
continue
|
||||
cur = ex.get(k)
|
||||
if cur is None or str(cur).strip() in ("", "-"):
|
||||
ex[k] = v
|
||||
|
||||
|
||||
def _apply_third_party_log_levels() -> None:
|
||||
"""在全局 sink 为 DEBUG/TRACE 时压低 Celery/httpx 噪声;可通过 CELERY_LOG_LEVEL / HTTPX_LOG_LEVEL 覆盖。"""
|
||||
"""压低 Celery/httpx 框架日志噪声。
|
||||
|
||||
根 logger 为 NOTSET 时,子 logger 若也为 NOTSET,有效级别会变成 0(NOTSET),INFO 会全部通过,
|
||||
因此这里**必须**写死默认级别,不能依赖 NOTSET「继承」。
|
||||
|
||||
默认(未设 CELERY_LOG_LEVEL / HTTPX_LOG_LEVEL):
|
||||
- ``LOG_LEVEL`` 为 TRACE/DEBUG:Celery→INFO,httpx/httpcore→WARNING
|
||||
- 否则:Celery 与 httpx/httpcore→WARNING(保留业务 loguru 与 ``event=celery_task_*`` 摘要)
|
||||
|
||||
需要框架原始行时,设置 ``CELERY_LOG_LEVEL=INFO``、``HTTPX_LOG_LEVEL=INFO`` 等。
|
||||
"""
|
||||
sink = _sink_min_level()
|
||||
verbose = sink in ("TRACE", "DEBUG")
|
||||
|
||||
# 无效环境变量时的回退:与「未设置变量」分支一致,禁止 NOTSET
|
||||
default_celery = logging.INFO if verbose else logging.WARNING
|
||||
default_httpx = logging.WARNING
|
||||
|
||||
raw_c = (settings.celery_log_level or "").strip()
|
||||
if raw_c:
|
||||
parsed = _parse_stdlib_level(raw_c)
|
||||
cel_level = (
|
||||
parsed
|
||||
if parsed is not None
|
||||
else (logging.INFO if verbose else logging.NOTSET)
|
||||
)
|
||||
cel_level = parsed if parsed is not None else default_celery
|
||||
else:
|
||||
cel_level = logging.INFO if verbose else logging.NOTSET
|
||||
cel_level = default_celery
|
||||
|
||||
for name in ("celery", "celery.worker"):
|
||||
logging.getLogger(name).setLevel(cel_level)
|
||||
@@ -128,13 +176,9 @@ def _apply_third_party_log_levels() -> None:
|
||||
raw_h = (settings.httpx_log_level or "").strip()
|
||||
if raw_h:
|
||||
parsed = _parse_stdlib_level(raw_h)
|
||||
httpx_level = (
|
||||
parsed
|
||||
if parsed is not None
|
||||
else (logging.WARNING if verbose else logging.NOTSET)
|
||||
)
|
||||
httpx_level = parsed if parsed is not None else default_httpx
|
||||
else:
|
||||
httpx_level = logging.WARNING if verbose else logging.NOTSET
|
||||
httpx_level = default_httpx
|
||||
|
||||
for name in ("httpx", "httpcore"):
|
||||
logging.getLogger(name).setLevel(httpx_level)
|
||||
@@ -175,6 +219,7 @@ def setup_logging() -> None:
|
||||
|
||||
Celery 需 ``worker_hijack_root_logger=False``,否则会覆盖根 logger。
|
||||
"""
|
||||
global logger
|
||||
logger.remove()
|
||||
|
||||
logger.add(
|
||||
@@ -185,7 +230,20 @@ def setup_logging() -> None:
|
||||
diagnose=False,
|
||||
)
|
||||
|
||||
json_path = (settings.log_json_file or "").strip()
|
||||
if json_path:
|
||||
logger.add(
|
||||
json_path,
|
||||
level=_sink_min_level(),
|
||||
serialize=True,
|
||||
rotation="20 MB",
|
||||
retention="7 days",
|
||||
encoding="utf-8",
|
||||
enqueue=True,
|
||||
)
|
||||
|
||||
logger.configure(extra={"request_id": "-", "module": "-"})
|
||||
logger = logger.patch(_merge_celery_worker_extra)
|
||||
|
||||
# 仅 root 挂 InterceptHandler,避免子 logger 与 root 各处理一次导致重复行
|
||||
root = logging.getLogger()
|
||||
@@ -201,4 +259,12 @@ def get_logger(name: str) -> Logger:
|
||||
|
||||
|
||||
# 供 middleware 等使用 ``contextualize`` 的同一 loguru 实例(与 get_logger 同源)
|
||||
__all__ = ["logger", "setup_logging", "get_logger", "InterceptHandler"]
|
||||
__all__ = [
|
||||
"logger",
|
||||
"setup_logging",
|
||||
"get_logger",
|
||||
"InterceptHandler",
|
||||
"format_log_event",
|
||||
"correlation_bind_kwargs",
|
||||
"celery_prerun_extras",
|
||||
]
|
||||
|
||||
Reference in New Issue
Block a user