更新日志维护策略
This commit is contained in:
parent
039a3e1bdc
commit
33065f6c09
11
README.md
11
README.md
@ -89,9 +89,9 @@ packaging/
|
||||
- chat 支持执行中按 `Ctrl+C` 中断,保存 checkpoint 后再 `resume`。
|
||||
- chat 支持 `set KEY=VALUE` 和 `load params <路径>` 热更新当前运行参数,并同步回写运行中的 `config.txt` 与 checkpoint。
|
||||
- 支持通过 `--llm-action-analysis-prompt-file`、`PAM_LLM_ACTION_ANALYSIS_PROMPT_FILE` 或 chat 内 `llm config action_analysis_prompt_file=...` 自定义 action 审核提示词。
|
||||
- 增加统一运行日志,默认写入 `logs/pam_deploy_agent.log`,覆盖 CLI/chat、LLM 调用、action 路由、脚本/MCP 调用、LangGraph、checkpoint 等关键流程。
|
||||
- 增加统一运行日志,默认写入 `logs/pam_deploy_agent.log`,覆盖 CLI/chat、LLM 调用、action 路由、脚本/MCP 调用、LangGraph、checkpoint 等关键流程,并按天切分、默认保留 14 个历史日切文件。
|
||||
- chat 支持 `llm test [文本]`,可用当前 LLM client 做一次轻量调用,确认真实 LLM 或规则 fallback 是否正常加载。
|
||||
- 添加基础测试,当前本地结果为 `72 passed, 3 skipped`。
|
||||
- 添加基础测试,当前本地结果为 `73 passed, 3 skipped`。
|
||||
|
||||
未完成:
|
||||
|
||||
@ -316,16 +316,17 @@ PAM> exit
|
||||
|
||||
## 日志
|
||||
|
||||
Agent 默认写入运行日志到 `logs/pam_deploy_agent.log`。日志覆盖 CLI/chat 输入、LLM 请求和响应摘要、action 路由、脚本/MCP 调用、LangGraph 节点、checkpoint 保存、暂停/续跑等关键流程。日志会递归脱敏 `CLIENT_SECRET`、`MCP_CLIENT_SECRET`、token、Authorization、api_key、password 等字段,并截断长文本。
|
||||
Agent 默认写入运行日志到 `logs/pam_deploy_agent.log`。日志覆盖 CLI/chat 输入、LLM 请求和响应摘要、action 路由、脚本/MCP 调用、LangGraph 节点、checkpoint 保存、暂停/续跑等关键流程。日志会在本地时间每日 0 点后首次写入时自动切分,历史文件形如 `pam_deploy_agent.log.YYYY-MM-DD`,默认保留 14 个历史日切文件。日志会递归脱敏 `CLIENT_SECRET`、`MCP_CLIENT_SECRET`、token、Authorization、api_key、password 等字段,并截断长文本。
|
||||
|
||||
可通过环境变量调整日志位置和级别:
|
||||
可通过环境变量调整日志位置、级别和保留策略:
|
||||
|
||||
```bash
|
||||
export PAM_AGENT_LOG_FILE=logs/pam_deploy_agent.log
|
||||
export PAM_AGENT_LOG_LEVEL=INFO
|
||||
export PAM_AGENT_LOG_RETENTION_DAYS=14
|
||||
```
|
||||
|
||||
调试 LLM 或 MCP 调用时可临时设为 `DEBUG`,但仍建议把日志目录放在受控位置。
|
||||
调试 LLM 或 MCP 调用时可临时把 `PAM_AGENT_LOG_LEVEL` 设为 `DEBUG`。`PAM_AGENT_LOG_RETENTION_DAYS` 表示保留的历史日切文件数量,设为 `0` 时不自动清理历史切分文件;仍建议把日志目录放在受控位置。
|
||||
|
||||
预演:
|
||||
|
||||
|
||||
@ -83,7 +83,7 @@ cd pam-deploy-agent-linux-x86_64
|
||||
- 进度查询和健康检查重试参数可通过 `POLL_INTERVAL_SEC`、`DOWNLOAD_POLL_MAX_ATTEMPTS`、`UPGRADE_POLL_MAX_ATTEMPTS`、`VERIFY_INTERVAL_SEC`、`VERIFY_MAX_ATTEMPTS` 配置。
|
||||
- 支持通过 `--llm-action-analysis-prompt-file` 或 chat 内 `llm config action_analysis_prompt_file=...` 自定义 action 审核提示词。
|
||||
- chat 支持 `llm test [文本]` 测试当前 LLM client 是否正常加载。
|
||||
- 默认运行日志写入 `logs/pam_deploy_agent.log`,可通过 `PAM_AGENT_LOG_FILE` 和 `PAM_AGENT_LOG_LEVEL` 调整。
|
||||
- 默认运行日志写入 `logs/pam_deploy_agent.log`,按天切分并默认保留 14 个历史日切文件,可通过 `PAM_AGENT_LOG_FILE`、`PAM_AGENT_LOG_LEVEL` 和 `PAM_AGENT_LOG_RETENTION_DAYS` 调整。
|
||||
- 日志会脱敏 token、secret、api_key、Authorization 等字段;checkpoint 仍保存完整运行参数,请放在受控目录。
|
||||
|
||||
## 包大小评估
|
||||
|
||||
@ -192,16 +192,17 @@ PAM> llm fallback
|
||||
|
||||
## 日志
|
||||
|
||||
Agent 默认写入运行日志到 `logs/pam_deploy_agent.log`。日志覆盖 chat/CLI 输入、LLM 请求和响应摘要、action 路由、脚本/MCP 调用、LangGraph 节点、checkpoint 保存、暂停/续跑等关键流程。
|
||||
Agent 默认写入运行日志到 `logs/pam_deploy_agent.log`。日志覆盖 chat/CLI 输入、LLM 请求和响应摘要、action 路由、脚本/MCP 调用、LangGraph 节点、checkpoint 保存、暂停/续跑等关键流程。日志会在本地时间每日 0 点后首次写入时自动切分,历史文件形如 `pam_deploy_agent.log.YYYY-MM-DD`,默认保留 14 个历史日切文件。
|
||||
|
||||
可通过环境变量调整日志位置和级别:
|
||||
可通过环境变量调整日志位置、级别和保留策略:
|
||||
|
||||
```bash
|
||||
export PAM_AGENT_LOG_FILE=logs/pam_deploy_agent.log
|
||||
export PAM_AGENT_LOG_LEVEL=INFO
|
||||
export PAM_AGENT_LOG_RETENTION_DAYS=14
|
||||
```
|
||||
|
||||
日志会递归脱敏 `CLIENT_SECRET`、`MCP_CLIENT_SECRET`、token、Authorization、api_key、password 等字段,并截断长文本。checkpoint 仍会保存完整运行参数,请放在受控目录。
|
||||
日志会递归脱敏 `CLIENT_SECRET`、`MCP_CLIENT_SECRET`、token、Authorization、api_key、password 等字段,并截断长文本。`PAM_AGENT_LOG_RETENTION_DAYS` 表示保留的历史日切文件数量,设为 `0` 时不自动清理历史切分文件。checkpoint 仍会保存完整运行参数,请放在受控目录。
|
||||
|
||||
## 策略说明
|
||||
|
||||
|
||||
@ -168,6 +168,9 @@ LLM 环境变量:
|
||||
PAM_AGENT_LOG_LEVEL
|
||||
日志级别,默认 INFO。排查 LLM/MCP 时可临时设为 DEBUG。
|
||||
|
||||
PAM_AGENT_LOG_RETENTION_DAYS
|
||||
历史日切日志保留数量,默认 14。设为 0 时不自动清理历史切分文件。
|
||||
|
||||
示例:
|
||||
./run.sh chat --config doc_scripts/config.txt.example --strategy fake --checkpoint runtime/checkpoints/demo.json
|
||||
|
||||
@ -197,7 +200,7 @@ LLM 环境变量:
|
||||
7. PARENT_VERSION_NUMBER 是云下载可选参数;空值不发送,非空时传给 parentVersionNumber。
|
||||
8. chat 执行过程中会播报每个 action 的开始、完成或失败;普通问候不会触发 LLM/结构化分析。
|
||||
9. chat 内可使用 params、events、rollback、list checkpoints、load checkpoint、load params、llm config、llm test、mcp config 等命令。
|
||||
10. 日志默认写入 logs/pam_deploy_agent.log,并会脱敏 token、secret、api_key、Authorization 等字段。
|
||||
10. 日志默认写入 logs/pam_deploy_agent.log,按天切分并默认保留 14 个历史日切文件;日志会脱敏 token、secret、api_key、Authorization 等字段。
|
||||
11. checkpoint 会保存完整运行参数,请放在受控目录。
|
||||
HELP_TEXT
|
||||
}
|
||||
|
||||
@ -7,14 +7,17 @@ import logging
|
||||
import os
|
||||
import re
|
||||
from dataclasses import asdict, is_dataclass
|
||||
from logging.handlers import TimedRotatingFileHandler
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from .constants import SENSITIVE_KEYS
|
||||
|
||||
DEFAULT_LOG_FILE = Path("logs") / "pam_deploy_agent.log"
|
||||
DEFAULT_LOG_RETENTION_DAYS = 14
|
||||
LOG_FILE_ENV = "PAM_AGENT_LOG_FILE"
|
||||
LOG_LEVEL_ENV = "PAM_AGENT_LOG_LEVEL"
|
||||
LOG_RETENTION_DAYS_ENV = "PAM_AGENT_LOG_RETENTION_DAYS"
|
||||
_HANDLER_MARKER = "_pam_deploy_agent_handler"
|
||||
_SENSITIVE_NAME_PARTS = ("secret", "token", "authorization", "api_key", "apikey", "password")
|
||||
_ASSIGNMENT_PATTERN = re.compile(
|
||||
@ -28,23 +31,38 @@ _BEARER_PATTERN = re.compile(r"(?i)(bearer\s+)[A-Za-z0-9._~+\-/=]+")
|
||||
def configure_logging(
|
||||
log_file: str | Path | None = None,
|
||||
level: str | int | None = None,
|
||||
retention_days: int | str | None = None,
|
||||
) -> Path:
|
||||
"""配置 Agent 文件日志;重复调用不会重复添加 handler。"""
|
||||
"""配置 Agent 每日滚动文件日志;重复调用不会重复添加 handler。"""
|
||||
actual_path = Path(log_file or os.getenv(LOG_FILE_ENV) or DEFAULT_LOG_FILE)
|
||||
actual_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
actual_level = _resolve_level(level or os.getenv(LOG_LEVEL_ENV) or "INFO")
|
||||
actual_retention_days = _resolve_retention_days(
|
||||
retention_days if retention_days is not None else os.getenv(LOG_RETENTION_DAYS_ENV),
|
||||
)
|
||||
|
||||
package_logger = logging.getLogger("pam_deploy_graph")
|
||||
package_logger.setLevel(actual_level)
|
||||
package_logger.propagate = False
|
||||
|
||||
marker = str(actual_path.resolve())
|
||||
for handler in package_logger.handlers:
|
||||
for handler in list(package_logger.handlers):
|
||||
if getattr(handler, _HANDLER_MARKER, "") == marker:
|
||||
if isinstance(handler, TimedRotatingFileHandler):
|
||||
handler.setLevel(actual_level)
|
||||
handler.backupCount = actual_retention_days
|
||||
return actual_path
|
||||
package_logger.removeHandler(handler)
|
||||
handler.close()
|
||||
break
|
||||
|
||||
handler = logging.FileHandler(actual_path, encoding="utf-8")
|
||||
handler = TimedRotatingFileHandler(
|
||||
actual_path,
|
||||
when="midnight",
|
||||
interval=1,
|
||||
backupCount=actual_retention_days,
|
||||
encoding="utf-8",
|
||||
)
|
||||
setattr(handler, _HANDLER_MARKER, marker)
|
||||
handler.setLevel(actual_level)
|
||||
handler.setFormatter(
|
||||
@ -54,7 +72,12 @@ def configure_logging(
|
||||
)
|
||||
)
|
||||
package_logger.addHandler(handler)
|
||||
package_logger.info("日志已初始化 path=%s level=%s", actual_path, logging.getLevelName(actual_level))
|
||||
package_logger.info(
|
||||
"日志已初始化 path=%s level=%s rotation=daily retention_days=%s",
|
||||
actual_path,
|
||||
logging.getLevelName(actual_level),
|
||||
actual_retention_days,
|
||||
)
|
||||
return actual_path
|
||||
|
||||
|
||||
@ -94,6 +117,17 @@ def _resolve_level(value: str | int) -> int:
|
||||
return resolved if isinstance(resolved, int) else logging.INFO
|
||||
|
||||
|
||||
def _resolve_retention_days(value: int | str | None) -> int:
|
||||
"""解析日志保留天数,非法值使用默认值。"""
|
||||
if value in (None, ""):
|
||||
return DEFAULT_LOG_RETENTION_DAYS
|
||||
try:
|
||||
days = int(str(value).strip())
|
||||
except (TypeError, ValueError):
|
||||
return DEFAULT_LOG_RETENTION_DAYS
|
||||
return max(days, 0)
|
||||
|
||||
|
||||
def _is_sensitive_key(key: str) -> bool:
|
||||
"""判断字段名是否应脱敏。"""
|
||||
if key in SENSITIVE_KEYS:
|
||||
|
||||
@ -1,4 +1,8 @@
|
||||
from pam_deploy_graph.logging_utils import json_for_log, redact_for_log
|
||||
import logging
|
||||
from logging.handlers import TimedRotatingFileHandler
|
||||
from pathlib import Path
|
||||
|
||||
from pam_deploy_graph.logging_utils import configure_logging, json_for_log, redact_for_log
|
||||
|
||||
|
||||
def test_redact_for_log_masks_sensitive_keys_and_inline_assignments():
|
||||
@ -26,3 +30,37 @@ def test_redact_for_log_masks_sensitive_keys_and_inline_assignments():
|
||||
assert "Bearer ***" in serialized
|
||||
assert "raw-token" not in serialized
|
||||
assert "plain-token" not in serialized
|
||||
|
||||
|
||||
def test_configure_logging_uses_daily_rotation_and_retention(tmp_path: Path):
|
||||
log_path = tmp_path / "pam_deploy_agent.log"
|
||||
package_logger = logging.getLogger("pam_deploy_graph")
|
||||
previous_handlers = list(package_logger.handlers)
|
||||
for handler in previous_handlers:
|
||||
package_logger.removeHandler(handler)
|
||||
|
||||
try:
|
||||
result = configure_logging(log_file=log_path, level="DEBUG", retention_days=3)
|
||||
|
||||
assert result == log_path
|
||||
handlers = [handler for handler in package_logger.handlers if isinstance(handler, TimedRotatingFileHandler)]
|
||||
assert len(handlers) == 1
|
||||
handler = handlers[0]
|
||||
assert Path(handler.baseFilename) == log_path.resolve()
|
||||
assert handler.when == "MIDNIGHT"
|
||||
assert handler.backupCount == 3
|
||||
assert package_logger.level == logging.DEBUG
|
||||
|
||||
configure_logging(log_file=log_path, level="INFO", retention_days=5)
|
||||
|
||||
handlers = [handler for handler in package_logger.handlers if isinstance(handler, TimedRotatingFileHandler)]
|
||||
assert len(handlers) == 1
|
||||
assert handlers[0] is handler
|
||||
assert handler.backupCount == 5
|
||||
assert package_logger.level == logging.INFO
|
||||
finally:
|
||||
for handler in list(package_logger.handlers):
|
||||
package_logger.removeHandler(handler)
|
||||
handler.close()
|
||||
for handler in previous_handlers:
|
||||
package_logger.addHandler(handler)
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user