Adapter Contractv1.0

Conversational Memory Adapter Spec

Adapter contract for conversational memory systems. These systems store and retrieve information from multi-turn dialogue, maintaining context across sessions.

Required methods

def setup(self) -> None

Initialize the memory system. Called once before any benchmark items. Must be idempotent.

def reset(self) -> None

Clear all stored memories. Called between benchmark items to ensure isolation.

def teardown(self) -> None

Clean up resources. Called once after all benchmark items complete.

def ingest(self, turns: List[Dict[str, Any]]) -> None

Store a conversation history. Each turn has 'role' (user/assistant) and 'content' keys.

def recall(self, query: str) -> str

Retrieve relevant memories for a query. Returns the raw text response from the memory system.

Required capabilities

-Multi-turn conversation ingestion
-Natural language query recall
-Session isolation (reset between items)

Applicable benchmarks

Benchmark	Alignment
LongMemEval	core
LoCoMo	core
Truth Arbitration	core
Memory Poisoning	core
Reliability	core
Knowledge Retrieval	adjacent
Budget Curves	adjacent

Example implementation

python

"""Example conversational memory adapter."""
from typing import Any, Dict, List, Optional
from benchd_harness.adapters.base import BaseAdapter


class MyMemoryAdapter(BaseAdapter):

    @property
    def name(self) -> str:
        return "my-memory"

    @property
    def version(self) -> Optional[str]:
        return "1.0.0"

    def setup(self) -> None:
        # Initialize your memory system
        self._client = MyMemoryClient()

    def reset(self) -> None:
        # Clear all memories for isolation
        self._client.clear()

    def teardown(self) -> None:
        self._client.close()

    def ingest(self, turns: List[Dict[str, Any]]) -> None:
        for turn in turns:
            self._client.add_message(
                role=turn["role"],
                content=turn["content"],
            )

    def recall(self, query: str) -> str:
        return self._client.search(query)

Quick start

Install

bash

pip install benchd-harness

Validate & test