Skip to content

feat(memory): 时间记忆退化 + DMN 健壮性修复 - #3

Merged
JohnRichard4096 merged 6 commits into
mainfrom
feat/temporal-memory-decay
Oct 14, 2026
Merged

JohnRichard4096 merged 6 commits into
mainfrom
feat/temporal-memory-decay

Conversation

@AmriaLin

@AmriaLin AmriaLin commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

本 PR 两件事,都落在 memory 插件的 L2 记忆层与 L3 离线巩固循环上。

1. 时间记忆退化(软衰减 + 分层清理)

记忆不再永久等权,越旧权重越低,超龄逐级退化直至清理。

  • 软衰减(检索期):query_notes 先多取 overfetch 倍候选,按 1/(1+d) × max(0.5^(age/half_life), floor) 重排后截回 top_k。半衰期随 importance 放大:low 为基准,medium 为 1+boost 倍,high 为 1+2·boost 倍。只重排、不删数据,得分回填元数据 decay_score。
  • 硬分层(清理期):按 importance 定寿命阈值(low = short_term_days、high = long_term_days、medium 取两者几何中值)。apscheduler 周期扫描,超龄先 high → medium → low 逐级降级,降到 low 仍超龄才删除。降级只改 importance,不重新嵌入。
  • 原来两个预留字段 short_term_expiry_days / long_term_expiry_days 是死配置,迁入 [decay] 配置块后真正生效。

新增 decay.py(纯逻辑,不碰 I/O)与 tasks.py(周期调度)。

2. DMN 审计修复

  • 调度:add_job 加 replace_existing;钩子内兜住 reschedule 异常,调度失败不再打断用户消息处理。
  • 启动:先挂触发钩子再启 runner,钩子失败则报 error 并跳过启动,不再让 DMN 静默死掉。
  • 知识库:create/update/delete 加 asyncio.Lock 串行;索引与知识文件改为 temp + fsync + os.replace 原子写。
  • duplicate_helper:改分片返回(limit/offset/has_more),并用已存向量预筛候选重复簇,不再一次性把全量正文灌进上下文。
  • 状态:惩罚计数随元状态持久化,重启后不退避归零。
  • prompt 注入:告警数先走轻量 count_user_notes,超阈值才拉全量算分布。
  • 杂项:target_user_id 收成只读属性;get_preset 失败不再裸抛;修正与隔离设计矛盾的过时注释。

验证

  • ruff check / ruff format --check 干净,py_compile 通过。
  • decay.py 纯逻辑单独跑过:寿命阈值 7 / 25.1 / 90、keep-demote-purge 分支、半衰期 30 / 60 / 90、衰减因子 0.5 与下限 0.05、重排顺序、top_k 截断、空结果、坏时间戳。
  • 未做端到端:本机没有装了该插件的 venv,没跑真实 bot + ChromaDB。

待定 / 未包含

  • subconscious_knowledge_list/_read/_search 的双重 @on_tools:其中一个没带 bound_to,会落到全局 ToolsManager。需要先确认是「有意给对话侧开只读知识库」还是漏删。
  • 运行期被重新排期导致多跑一轮:属行为变更,需先定语义。
  • DMN 用量单列、启动期三方对账挪后台、SubconsciousBackend 单例吃旧配置、session 缓存键、剩余同步 I/O。

兼容性

ConfigFile 移除顶层 short_term_expiry_days / long_term_expiry_days,改由 [decay] 提供。旧 config.toml 里的这两个键会被 pydantic 忽略(不报错),新默认值生效。

Summary by Sourcery

Implement time-based memory decay and strengthen DMN, knowledge-base, and duplicate-memory workflows for safer and more scalable operation.

New Features:

  • Add configurable time-based memory decay with retrieval-time score adjustment and periodic importance demotion or deletion.
  • Add permission-controlled conversation-side knowledge-base read tools.
  • Provide paginated duplicate-memory review with vector-based candidate grouping.

Bug Fixes:

  • Harden DMN scheduling and startup failure handling so scheduling errors do not interrupt messages or leave the runner silently inactive.
  • Prevent concurrent knowledge-base mutations and partial file/index writes.
  • Persist DMN penalty state across restarts and avoid unnecessary full-memory loads during prompt construction.
  • Handle unavailable presets without crashing a DMN cycle and align context-compaction configuration with the current core API.

Enhancements:

  • Expose decay scores in memory query metadata and reorganize decay settings under a dedicated configuration section.
  • Improve knowledge-base consistency and duplicate-review scalability while preserving isolated subconscious tools.

Build:

  • Bump the plugin version to 0.6.0 and update the lockfile.

Documentation:

  • Document memory decay behavior, configuration, decay scores, and knowledge-base read permissions.

- 软衰减(检索期):query_notes 多取 overfetch 倍候选,按
  1/(1+d) × 指数衰减重排后截回 top_k;importance 放大半衰期
- 硬分层(清理期):按 importance 定寿命阈值,超龄 high→medium→low
  逐级降级,降到 low 仍超龄才删除;apscheduler 周期执行
- 检索结果回填 decay_score 便于排查
- 原预留字段 short/long_term_expiry_days 迁入 [decay] 配置块并真正生效
- 调度:add_job 加 replace_existing;钩子内兜住 reschedule 异常,
  避免调度失败打断用户消息处理
- 启动:先挂触发钩子再启 runner,钩子失败则报 error 并跳过启动,
  不再让 DMN 静默死掉
- 知识库:create/update/delete 加 asyncio.Lock 串行;索引与知识文件
  改为 temp + fsync + os.replace 原子写,写一半崩溃不再毁索引
- duplicate_helper:改分片返回(limit/offset/has_more),并用已存向量
  预筛候选重复簇,不再一次性把全量正文灌进上下文
- 状态:惩罚计数随元状态持久化,重启后不退避归零
- prompt 注入:告警数先走轻量 count,超阈值才拉全量算分布
- 杂项:target_user_id 收成只读属性;get_preset 失败不再裸抛;
  修正 tools.py 与隔离设计矛盾的过时注释
@sourcery-ai

sourcery-ai Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Reviewer's Guide

本 PR 为 L2 记忆增加基于年龄和重要性的检索软衰减、逐级降级/删除的周期清理及新配置体系,同时通过调度容错、状态持久化、知识库锁与原子写、重复记忆分片和向量预筛,提升 L3 DMN 与知识库流程的可靠性和大规模处理能力。

Sequence diagram for time-decayed memory retrieval

sequenceDiagram
    participant Caller
    participant AsyncUserMemory
    participant ChromaDB
    participant DecayLogic

    Caller->>AsyncUserMemory: query_notes(query_text, top_k)
    AsyncUserMemory->>ChromaDB: get(where, include=[])
    ChromaDB-->>AsyncUserMemory: matching_count
    AsyncUserMemory->>ChromaDB: query(n_results=top_k*overfetch)
    ChromaDB-->>AsyncUserMemory: candidates with distances and metadatas
    AsyncUserMemory->>DecayLogic: rerank_query_result(result, top_k, policy)
    DecayLogic-->>AsyncUserMemory: reordered candidates with decay_score
    AsyncUserMemory-->>Caller: top_k results
Loading

Sequence diagram for resilient DMN startup and scheduling

sequenceDiagram
    participant Driver
    participant Hooks
    participant Runner
    participant Scheduler
    participant UserMessage

    Driver->>Hooks: _setup_subconscious_hook()
    alt hook setup succeeds
        Hooks-->>Driver: registered
        Driver->>Runner: start()
        Runner->>Scheduler: add_job(replace_existing=True)
        UserMessage->>Hooks: _subconscious_pre_hook(event)
        Hooks->>Runner: cancel_and_reschedule()
        alt reschedule fails
            Runner-->>Hooks: exception
            Hooks-->>UserMessage: log error and continue
        else reschedule succeeds
            Runner-->>Hooks: scheduled
        end
    else hook setup fails
        Hooks-->>Driver: exception
        Driver-->>Runner: do not start
    end
Loading

Flow diagram for tiered memory decay cleanup

flowchart TD
    A["Scheduled run_decay_sweep"] --> B["AsyncUserMemory.sweep_decay(policy)"]
    B --> C["plan_action(metadata, now, thresholds)"]
    C -->|"keep"| D["Keep memory"]
    C -->|"demote"| E["Update importance to next lower tier"]
    C -->|"purge"| F["Delete low-importance memory"]
    E --> G["ChromaDB.update"]
    F --> H["ChromaDB.delete"]
Loading

File-Level Changes

Change Details Files
引入可配置的时间记忆生命周期管理,将检索软衰减与后台分层清理拆分为纯逻辑和调度实现。
  • 新增衰减配置并将旧过期字段迁移至 [decay] 配置块。
  • 检索时按距离、年龄和重要性半衰期过取候选、重排、截断,并回填 decay_score。
  • 清理时按重要性阈值执行 keep、逐级 demote 或 purge,新增周期调度入口。
README.md
amrita_plugin_memory/config.py
amrita_plugin_memory/decay.py
amrita_plugin_memory/tasks.py
amrita_plugin_memory/vector.py
amrita_plugin_memory/__init__.py
增强 DMN 调度、启动和状态恢复的故障处理,避免静默失效或用户消息链路被后台异常打断。
  • 调度任务支持替换已有任务,重排异常被隔离,启动时先注册钩子并在失败时跳过 runner 启动。
  • 安全暴露 runner 的目标用户配置,并处理 preset 获取失败。
  • 持久化和恢复 penalty 计数,减少记忆提示词生成中的不必要全量读取。
amrita_plugin_memory/rethinking/_state.py
amrita_plugin_memory/rethinking/hooks.py
amrita_plugin_memory/rethinking/runner.py
串行化知识库变更并采用原子文件写入,提升多步文件、索引和向量操作的一致性与恢复能力。
  • 为 create/update/delete 增加异步锁。
  • 知识文件和索引写入改为临时文件、fsync、os.replace。
amrita_plugin_memory/rethinking/knowledge.py
降低重复记忆整理的上下文负担,并提供向量预筛候选重复簇。
  • duplicate helper 改为支持 limit/offset 分片返回及 has_more。
  • 利用已存 embedding 按相似度聚类候选重复项,再交由 LLM 做最终重复判断。
amrita_plugin_memory/rethinking/schemas.py
amrita_plugin_memory/rethinking/tools.py
调整工具注册和隔离相关接口行为,明确潜意识工具与对话侧工具的管理器边界。
  • 更新工具模块说明,保持 subconscious 工具绑定隔离管理器。
  • 将 target user 配置改为只读属性,并同步调用方。
amrita_plugin_memory/rethinking/tools.py
amrita_plugin_memory/rethinking/runner.py
amrita_plugin_memory/rethinking/_state.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 9 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="amrita_plugin_memory/vector.py" line_range="494" />
<code_context>
+                    reranked.pop(key, None)
+            return reranked

     @require_init
     async def get_all_notes(
</code_context>
<issue_to_address>
**Concurrent edits are lost**

When a note is edited after the sweep reads its metadata but before the sweep applies its action, `sweep_decay` reads metadata and then updates or deletes records without the per-user lock used by `update_note` and `delete_note`. The sweep applies its stale snapshot, overwriting an edit or deleting a newly promoted note, so the caller’s change is lost.

Hold the per-user lock across the sweep’s metadata read and its updates or deletions.

Also at `amrita_plugin_memory/vector.py:495`, `amrita_plugin_memory/vector.py:498-529`.
</issue_to_address>

### Comment 2
<location path="amrita_plugin_memory/vector.py" line_range="501" />
<code_context>
+        result = await any_to_thread(
+            self._collection.get,
+            include=["metadatas"],
+            limit=policy.sweep_batch_limit,
+        )
+        ids = result.get("ids") or []
</code_context>
<issue_to_address>
**Later memories never get swept**

When the collection exceeds `sweep_batch_limit` and records in the first batch remain present, `sweep_decay` calls `get` with a limit but no offset or continuation, so each run inspects the same first batch. Records beyond it are never considered and overdue memories remain unchanged.

Add offset-based pagination or a continuation cursor so sweeps inspect later records.

Also at `amrita_plugin_memory/vector.py:495`.
</issue_to_address>

### Comment 3
<location path="amrita_plugin_memory/rethinking/tools.py" line_range="447" />
<code_context>
+        limit = int(data.get("limit") or _DUPLICATE_DEFAULT_LIMIT)
+        limit = max(1, min(limit, _DUPLICATE_MAX_LIMIT))
+        offset = max(0, int(data.get("offset") or 0))
+        shard = items[offset : offset + limit]
+
+        threshold = float(data.get("similarity_threshold") or 0.85)
</code_context>
<issue_to_address>
**Later pages skip memories**

When the caller deletes entries from a returned shard before requesting the next offset, `subconscious_duplicate_helper` recomputes the sorted list and slices it using the supplied offset. Deletions shift later entries toward the start, so the next offset skips memories that were not reviewed.

Use pagination that remains stable when earlier entries are deleted, such as a continuation cursor rather than an offset into the changing list.

Also at `amrita_plugin_memory/rethinking/tools.py:489-490`, `amrita_plugin_memory/rethinking/schemas.py:184`.
</issue_to_address>

### Comment 4
<location path="amrita_plugin_memory/rethinking/tools.py" line_range="393-395" />
<code_context>
     pid = _get_partition_id()
     try:
-        result = await ope.get_all_notes(pid, include=["metadatas", "documents"])
+        result = await ope.get_all_notes(
+            pid, include=["metadatas", "documents", "embeddings"]
+        )
         if not result or not result.get("ids"):
             return _tools_ok(
</code_context>
<issue_to_address>
**Pagination still loads everything**

When the memory collection is large enough that fetching all documents and embeddings is expensive, `subconscious_duplicate_helper` calls `get_all_notes` for every document and embedding before applying `offset` and `limit`. Pagination bounds only the returned prompt, so loading the full collection can exhaust memory or make the tool time out.

Fetch only the requested page of documents and embeddings instead of loading the entire collection before slicing.
</issue_to_address>

### Comment 5
<location path="amrita_plugin_memory/vector.py" line_range="426-425" />
<code_context>
+                        fetch_include.append(key)
+            fetch_k = max(top_k, 1)
+            if policy.enabled and policy.overfetch > 1:
+                total = len(
+                    (
+                        await any_to_thread(
</code_context>
<issue_to_address>
**Memory searches slow as collections grow**

When decay is enabled, `overfetch` exceeds 1, and the user has a large memory collection, `query_notes` calls `get` to materialize every matching ID just to calculate `total` and cap `fetch_k`, adding work proportional to the collection before each vector query. Memory searches therefore become increasingly slow as the collection grows.

Use the bounded overfetch count directly or obtain the count without materializing every matching ID.

Also at `amrita_plugin_memory/vector.py:426-434`.
</issue_to_address>

### Comment 6
<location path="amrita_plugin_memory/decay.py" line_range="85-87" />
<code_context>
+        return float(short_days)
+    if importance == "high":
+        return float(long_days)
+    if long_days > short_days:
+        return math.sqrt(float(short_days) * float(long_days))
+    return float(short_days)
+
+
</code_context>
<issue_to_address>
**Tier expiry thresholds are wrong**

When `long_term_days` is less than `short_term_days`, `tier_days` falls back to `short_days` instead of using the geometric mean, while high-importance memories use the shorter `long_days` threshold. Medium and high memories therefore get inconsistent expiry thresholds, and high-importance rows can be demoted or purged first.

Handle configurations where `long_days` is less than or equal to `short_days` so tier thresholds retain the intended geometric-mean behavior and progression.
</issue_to_address>

### Comment 7
<location path="amrita_plugin_memory/decay.py" line_range="115-118" />
<code_context>
+
+def _inner_list(value: Any, size: int) -> list:
+    """取 QueryResult 里 list-of-list 的内层;缺失时补 None。"""
+    if isinstance(value, list) and value and isinstance(value[0], list):
+        inner = list(value[0])
+    else:
+        inner = []
+    if len(inner) < size:
+        inner.extend([None] * (size - len(inner)))
</code_context>
<issue_to_address>
**Requested embeddings are corrupted**

When a caller requests `embeddings` in `query_notes` while decay is enabled, `rerank_query_result` passes Chroma’s NumPy query embeddings through `_inner_list`, which accepts only Python lists. The helper pads the rejected array with `None`, so callers receive `None` values instead of the requested vectors.

Handle NumPy arrays when reordering embeddings in `rerank_query_result`.

Also at `amrita_plugin_memory/decay.py:167`.
</issue_to_address>

### Comment 8
<location path="amrita_plugin_memory/rethinking/tools.py" line_range="451-453" />
<code_context>
+
+        threshold = float(data.get("similarity_threshold") or 0.85)
+        cluster_ids = _cluster_candidates(
+            [str(it["id"]) for it in shard],
+            [it["embedding"] for it in shard],
+            threshold,
+        )
+        for it in shard:
</code_context>
<issue_to_address>
**Cross-page duplicates go unflagged**

When highly similar memories land on different page boundaries, `_cluster_candidates` receives embeddings only from the current shard, so similar memories on different pages are never compared or returned together as candidate duplicates.

Compare candidates across page boundaries so similar memories can be returned in the same candidate group.
</issue_to_address>

### Comment 9
<location path="amrita_plugin_memory/rethinking/runner.py" line_range="375" />
<code_context>
             {
                 "total_runs": self._total_runs,
                 "last_run_time": datetime.now(timezone.utc).isoformat(),
+                "penalty": _state.get_penalty_count(),
                 "last_abstracts": self._last_abstracts,
                 "pending_messages": _state.get_pending(),
</code_context>
<issue_to_address>
**Accumulated backoff is lost**

When the process restarts after a user message increments the penalty but before a later state save, `cancel_and_reschedule` increments the in-memory penalty without saving it, so `_load_state` restores the older persisted value after restart. The next reschedule then uses a shorter delay than the user activity established.

Save state when `cancel_and_reschedule` increments the penalty.

Also at `amrita_plugin_memory/rethinking/runner.py:376`.
</issue_to_address>

Sourcery assessment

Needs a human reviewer. 9 findings to address first, and 默认开启的后台清理会把超过 low 层寿命的记忆永久删除,并会先持久化降级 importance;如果保留策略或时间解析判断有误,用户数据会在正常运行中被不可逆地清除。回滚代码只能停止后续清理,无法恢复已经删除或降级的记录。.

Blocking findings: amrita_plugin_memory/vector.py:494, amrita_plugin_memory/vector.py:501, amrita_plugin_memory/rethinking/tools.py:447, amrita_plugin_memory/rethinking/tools.py:395, amrita_plugin_memory/vector.py:425, and 4 more


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

Comment thread amrita_plugin_memory/vector.py
Comment thread amrita_plugin_memory/vector.py Outdated
Comment thread amrita_plugin_memory/rethinking/tools.py Outdated
Comment thread amrita_plugin_memory/rethinking/tools.py Outdated
Comment thread amrita_plugin_memory/vector.py
Comment thread amrita_plugin_memory/decay.py Outdated
Comment thread amrita_plugin_memory/decay.py
Comment thread amrita_plugin_memory/rethinking/tools.py Outdated
Comment thread amrita_plugin_memory/rethinking/runner.py

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 9 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="amrita_plugin_memory/rethinking/tools.py" line_range="3-5" />
<code_context>
-"""潜意识工具处理器 — 模块导入时自动注册到全局 ToolsManager。"""
+"""潜意识工具处理器。
+
+所有 subconscious_* 通过 bound_to 注册到隔离的 Subconscious 工具管理器,
+不污染全局 ToolsManager;knowledge_suggest 面向对话 LLM,注册到全局管理器。
+"""

 import asyncio
</code_context>
<issue_to_address>
**DMN knowledge is globally readable**

When a conversation's LLM receives tools registered with the global `ToolsManager`, the unbound `@on_tools` decorators on `subconscious_knowledge_list`, `subconscious_knowledge_read`, and `subconscious_knowledge_search` register them with the global tool manager, so callers can enumerate and retrieve the shared knowledge base without an identity check; its contents are returned in tool responses.

Remove the unbound registrations or explicitly authorize conversation callers before returning knowledge-base contents.
</issue_to_address>

### Comment 2
<location path="amrita_plugin_memory/vector.py" line_range="495-498" />
<code_context>
             return len(result["ids"])
+
+    @require_init
+    async def sweep_decay(self, policy: DecayConfig) -> dict[str, int]:
+        """扫描记忆库:超龄先降级,降到 low 仍超龄则清理。"""
+        now = datetime.now(utc)
+        result = await any_to_thread(
+            self._collection.get,
+            include=["metadatas"],
</code_context>
<issue_to_address>
**Concurrent edits are lost**

When a memory is edited while a sweep is processing it, `sweep_decay` reads metadata and then updates or deletes records without the per-user lock used by `update_note`. A stale demotion overwrites newer metadata, or a stale purge deletes a memory that was promoted after the scan, so the caller’s edit disappears.

Hold the same per-user lock used by `update_note` across the sweep’s read and its updates or deletions, or recheck current metadata under that lock before applying each action.

Also at `amrita_plugin_memory/vector.py:499-529`.
</issue_to_address>

### Comment 3
<location path="amrita_plugin_memory/vector.py" line_range="501" />
<code_context>
+        result = await any_to_thread(
+            self._collection.get,
+            include=["metadatas"],
+            limit=policy.sweep_batch_limit,
+        )
+        ids = result.get("ids") or []
</code_context>
<issue_to_address>
**Later memories never decay**

When the collection contains more entries than `sweep_batch_limit`, `sweep_decay` requests a fixed-size page without an offset or continuation state, so scheduled runs keep examining the same first batch and leave later records untouched. Those records are never demoted or purged.

Advance through the collection with pagination or another continuation mechanism so scheduled sweeps eventually examine every record.

Also at `amrita_plugin_memory/vector.py:495-498`, `amrita_plugin_memory/vector.py:502`.
</issue_to_address>

### Comment 4
<location path="amrita_plugin_memory/rethinking/tools.py" line_range="446-447" />
<code_context>
+        total = len(items)
+        limit = int(data.get("limit") or _DUPLICATE_DEFAULT_LIMIT)
+        limit = max(1, min(limit, _DUPLICATE_MAX_LIMIT))
+        offset = max(0, int(data.get("offset") or 0))
+        shard = items[offset : offset + limit]
+
+        threshold = float(data.get("similarity_threshold") or 0.85)
</code_context>
<issue_to_address>
**Later shards skip memories**

When the caller updates or deletes entries from a shard before requesting the next shard, `subconscious_duplicate_helper` rebuilds the sorted list and slices it using the caller’s offset. Mutations shift later entries to lower offsets, so the next request skips unreviewed memories and duplicate cleanup can finish without checking them.

Use pagination that remains stable across caller mutations, or give callers a continuation mechanism that does not rely on offsets into the changing list.

Also at `amrita_plugin_memory/rethinking/tools.py:448`, `amrita_plugin_memory/rethinking/tools.py:489`.
</issue_to_address>

### Comment 5
<location path="amrita_plugin_memory/vector.py" line_range="426-429" />
<code_context>
+                        fetch_include.append(key)
+            fetch_k = max(top_k, 1)
+            if policy.enabled and policy.overfetch > 1:
+                total = len(
+                    (
+                        await any_to_thread(
+                            self._collection.get, where=where, include=[]
+                        )
+                    )["ids"]
</code_context>
<issue_to_address>
**Searches read every matching ID**

When decay is enabled and a user has a large memory collection, `query_notes` calls `Collection.get` to materialize every matching ID just to calculate `fetch_k`, before issuing the bounded vector query. This adds an O(N) read and allocation to each search while holding the per-user lock, increasing latency and delaying concurrent searches.

Choose `fetch_k` without fetching every matching ID, or use a bounded count operation that does not materialize the full ID list.

Also at `amrita_plugin_memory/vector.py:430-431`.
</issue_to_address>

### Comment 6
<location path="amrita_plugin_memory/rethinking/tools.py" line_range="394" />
<code_context>
     try:
-        result = await ope.get_all_notes(pid, include=["metadatas", "documents"])
+        result = await ope.get_all_notes(
+            pid, include=["metadatas", "documents", "embeddings"]
+        )
         if not result or not result.get("ids"):
</code_context>
<issue_to_address>
**Paged requests load the full collection**

When the duplicate helper is used on a large memory collection, `subconscious_duplicate_helper` retrieves every document and embedding before slicing the requested shard, so each page request still transfers and retains the full collection in memory. Large collections can stall the tool or exhaust available resources.

Fetch only the requested shard’s documents and embeddings instead of loading the full collection before slicing.

Also at `amrita_plugin_memory/vector.py:426-434`.
</issue_to_address>

### Comment 7
<location path="amrita_plugin_memory/rethinking/runner.py" line_range="375" />
<code_context>
             {
                 "total_runs": self._total_runs,
                 "last_run_time": datetime.now(timezone.utc).isoformat(),
+                "penalty": _state.get_penalty_count(),
                 "last_abstracts": self._last_abstracts,
                 "pending_messages": _state.get_pending(),
</code_context>
<issue_to_address>
**Reschedule backoff is lost**

When the process restarts after a reschedule but before the delayed cycle saves state, `cancel_and_reschedule` increments the penalty without persisting it, so `_load_state` restores the older count after a restart and the next schedule uses stale backoff.

Persist the updated penalty when `cancel_and_reschedule` increments it.

Also at `amrita_plugin_memory/rethinking/runner.py:242-243`, `amrita_plugin_memory/rethinking/runner.py:374`.
</issue_to_address>

### Comment 8
<location path="amrita_plugin_memory/rethinking/runner.py" line_range="242-243" />
<code_context>
         if save_ok:
             _state.reset_penalty()
+            # 惩罚清零也落盘,否则重启后残留旧退避
+            with contextlib.suppress(Exception):
+                await self._save_state()
             logger.info(
                 f"[Subconscious] Cycle done (total_runs={self._total_runs}), "
</code_context>
<issue_to_address>
**Penalty reset is not persisted**

When the follow-up state save fails after the reset, `_post_process` suppresses the error from `_save_state` after resetting the penalty, while the earlier save still contains the old value. A restart therefore reloads the stale backoff without any indication that the reset was not persisted.

Handle and log the failed save, and ensure the reset is persisted before treating it as complete.

Also at `amrita_plugin_memory/rethinking/runner.py:376`.
</issue_to_address>

### Comment 9
<location path="amrita_plugin_memory/decay.py" line_range="162-167" />
<code_context>
+    order = sorted(range(size), key=lambda i: scores[i], reverse=True)[:top_k]
+
+    out = dict(result)
+    for key in ("ids", "documents", "metadatas", "distances", "embeddings"):
+        value = result.get(key)
+        if value is None:
+            continue
+        inner = _inner_list(value, size)
+        out[key] = [[inner[i] for i in order]]
+
+    if result.get("metadatas") is not None:
</code_context>
<issue_to_address>
**Requested embeddings become null**

When a caller requests embeddings and Chroma returns them as NumPy arrays, `rerank_query_result` passes Chroma’s NumPy-array embeddings to `_inner_list`, which accepts only Python lists. It substitutes `None` entries, so callers receive null embeddings instead of the requested vectors.

Make `_inner_list` handle Chroma’s NumPy-array results as well as Python lists.

Also at `amrita_plugin_memory/decay.py:115`.
</issue_to_address>

Sourcery assessment

Needs a human reviewer. 9 findings to address first, and with the default settings, a scheduled task permanently deletes low-importance memories older than the retention threshold, and higher-importance memories are persistently demoted first. Reverting the code stops future sweeps but cannot restore memories already purged; the retention policy is also the decision itself and applies broadly from deployment.

Blocking findings: amrita_plugin_memory/rethinking/tools.py:5, amrita_plugin_memory/vector.py:498, amrita_plugin_memory/vector.py:501, amrita_plugin_memory/rethinking/tools.py:447, amrita_plugin_memory/vector.py:429, and 4 more


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

Comment thread amrita_plugin_memory/rethinking/tools.py
Comment thread amrita_plugin_memory/vector.py Outdated
Comment thread amrita_plugin_memory/vector.py Outdated
Comment thread amrita_plugin_memory/rethinking/tools.py Outdated
Comment thread amrita_plugin_memory/vector.py Outdated
Comment thread amrita_plugin_memory/rethinking/tools.py Outdated
Comment thread amrita_plugin_memory/rethinking/runner.py
Comment thread amrita_plugin_memory/rethinking/runner.py Outdated
Comment thread amrita_plugin_memory/decay.py
- sweep_decay:改为「只读分页扫描 + 按分区持锁复核后应用」。既不会漏掉
  sweep_batch_limit 之外的记录,也不会用过期快照覆盖扫描期间的并发编辑
- duplicate_helper:候选簇改为在完整过滤集上聚类、再按当前分片筛选,
  跨分片的相似记忆不再漏判;提示词补一条「分片遍历期间先别删改」
- decay.tier_days:short/long 写反时按大小重排,保证 low ≤ medium ≤ high,
  不再出现 high 比 medium 先过期
- decay._inner_list:兼容 chroma 返回的 numpy embeddings,
  调用方请求 embeddings 时不再被填成 None
- 去掉 subconscious_knowledge_list/_read/_search 的全局注册,
  恢复「不污染全局 ToolsManager」的隔离设计
- cancel_and_reschedule 递增惩罚后立即落盘,重启不再丢已累积的退避
- 新增 permissions.py:超级用户 / lp.admin / memory.knowledge 依次放行;
  LitePerm 不可用时只认 .env 超级用户并告警,不静默放开
- knowledge_list / knowledge_read / knowledge_search 重新对会话侧注册,
  改为 custom_run 入口,先校验权限再调用同一套实现;
  潜意识循环侧走隔离管理器,不受该权限约束
- 未授权时返回错误 JSON,不返回任何知识内容
- README 补上节点说明与授权方式
CI 的 build / check 都挂在 pyright,根因是 uv.lock 仍是旧解析结果
(amrita-core 0.13.9 / amrita 1.9.0,已不满足 pyproject 的 >=1.0.0 / >=1.10),
CI 每次 uv sync 都会重新解析到 amrita-core 1.3.0,而 LLMConfig 在这中间换了 API。

- runner: enable_compaction 已由 llm.context_strategy 取代,bool 开关映射为 compact/none
- runner: 惩罚清零后的落盘失败改为告警,不再静默丢退避
- vector: query_notes 去掉每次检索前全量取 id 计数的 O(N) 调用,直接用 overfetch 放大
- vector: get_all_notes 支持按 ids 取指定记忆
- vector/decay: rerank 入参放宽为 Mapping[str, Any],返回处 cast 回 QueryResult
- tools: duplicate_helper 的 offset 分页改为 next_cursor 游标,遍历期间删改前几片不再漏看
- tools: 正文改为只取当前分片,仅按长度排序时才全量
- uv.lock: 重新解析锁定

@JohnRichard4096 JohnRichard4096 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@JohnRichard4096
JohnRichard4096 merged commit 07c8028 into main Oct 14, 2026
3 checks passed
@JohnRichard4096
JohnRichard4096 deleted the feat/temporal-memory-decay branch October 14, 2026 14:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants