No description
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
DylanDark f42605d7ac
Some checks are pending
Quality / quality (push) Waiting to run
Quality / PostgreSQL (development subset) (push) Blocked by required conditions
Quality / PostgreSQL (full merge validation) (push) Blocked by required conditions
Implement governed memory projection prerequisites for restricted-live runtime (#298)
## Outcome

This PR closes the two architectural prerequisites PR #296 identified as
blocking the restricted-live confirmed-memory path, without weakening
the Context Broker or privacy/disclosure boundary and without reviving
the bypass PR #296 removed. The confirmed-memory read into
`GovernedConversationRequestBuilder` remains fail-closed. No protected
plaintext is persisted, and no keys, secrets, or live secret resolution
are introduced.

## Architecture choice

**1. Canonical claim-to-information-object association.** A
`DurableClaim` carries only `semantic_type_version_id`; no committed
table maps a claim to an `InformationObjectId`. This PR adds a durable,
reviewed association rather than synthesizing an ad hoc runtime mapping:

- `memory.claim_information_objects` table (Alembic
`phase_10a_memory_projection`, `down_revision =
phase_10a_turn_reply_continuity`).
- One claim maps to exactly one canonical information object, enforced
by a database uniqueness constraint on `durable_claim_id`.
- `ClaimInformationObject` domain record + `ClaimInformationObjectId`.
- `ClaimInformationObjectRepository` port +
`PostgreSQLClaimInformationObjectRepository` adapter, wired through
`PostgreSQLUnitOfWork`.

**2. Governed protected-value resolution.** A claim's opaque
`value_reference` is not the remembered value. This PR adds a narrow,
inward-owned protected-value store rather than copying content into
`DurableClaim`:

- `vault.protected_values` table keyed by the opaque
`protected_value_reference` (equal to `DurableClaim.value_reference`).
- `ProtectedValue` domain record persists only a ciphertext-only,
envelope-encryption-ready envelope — `ciphertext`, `wrapped_dek`,
`kek_reference` — plus content-minimized integrity metadata
(`content_digest`, `content_size_bytes`). No protected plaintext is
stored, consistent with ADR 0004 and
`docs/architecture/protected-value-vault.md`.
- `ProtectedValue.__repr__` redacts ciphertext and wrapped key material,
so protected material cannot leak to logs, errors, configuration, or
telemetry.
- `ProtectedValueRepository` port + adapter, wired through
`PostgreSQLUnitOfWork`.

**3. Binding before rendering.** `project_confirmed_memory` resolves a
protected record only after verifying it is
cryptographically/semantically bound to the exact claim that references
it: opaque reference, owner actor, privacy classification, value kind,
digest, and size must all match. Cross-actor reference splicing and
metadata/digest mismatches fail closed. Because the protected record is
ciphertext-only, no segment is rendered until governed decryption (the
vault KEK) is separately approved;
`ConfirmedMemoryProjectionResult.bound_claims` records the bound claims
for audit.

**4. Composition.** `project_confirmed_memory` composes the existing
`project_context` (Context Broker) and
`memory_retrieval_service.retrieve_claims` (disclosure ladder).
Remembered content reaches a `ContinuitySegment` only after the
caller-supplied authorized projection, disclosure filtering, and
protected-value binding permit it. Actor scoping alone is never
authorization.

**5. Migration naming.** The revision is `phase_10a_memory_projection`,
not a `phase_5*` head. The repository guardrail "Migration head
unchanged after Phase 5" freezes Phase 5 migrations, and this is
restricted-live (Phase 10A) work. The migration is packaging-only here;
it is never applied to a retained/live database in this issue.

## What changed

- `src/steve_os/domain/` — `ClaimInformationObject`,
`ClaimInformationObjectId`, ciphertext-only `ProtectedValue`; exported
through `domain/__init__.py`.
- `src/steve_os/ports/memory_claims.py` — narrow
`ClaimEvidenceRepository`, `ClaimRelationshipRepository`,
`ClaimRetractionRepository`, `ClaimInformationObjectRepository`,
`ProtectedValueRepository`; extended `MemoryClaimUnitOfWork`.
- `src/steve_os/adapters/postgresql/memory_claims.py` — new tables +
repositories + list/get-by-claim read methods.
- `src/steve_os/adapters/postgresql/unit_of_work.py` —
`claim_information_objects` and `protected_values` properties.
- `src/steve_os/application/confirmed_memory_projection.py` —
`project_confirmed_memory`, `ConfirmedMemoryProjectionResult`, and
`bind_protected_value_to_claim`.
- `src/steve_os/application/memory_retrieval_service.py` — extend
`retrieve_claims` to exclude superseded claims and surface
`is_superseded` in `UncertaintyMetadata`.
-
`src/steve_os/adapters/postgresql/migrations/versions/phase_10a_memory_projection.py`
— new packaging-only revision.
- `src/steve_os/runtime/one_user_runtime_adapters.py` —
`EmptyConfirmedMemoryReader` docstring updated for ciphertext-only store
and the write-path orchestration gap.
- `tests/support/postgresql.py` — `PACKAGED_HEAD` updated to
`phase_10a_memory_projection`.

## Tests

- `tests/unit/test_confirmed_memory_projection.py` — deterministic,
offline: an eligible claim is retrieved and its protected record is
bound but not rendered (fail-closed, no vault KEK); unprojected and
privacy-denied claims produce no segments; missing protected value;
retracted/superseded/contradicted/expired claims excluded; another actor
cannot retrieve owner memory; `ProtectedValue` repr redacts
ciphertext/wrapped key material; binding rejects cross-actor reference
splicing and digest/privacy/value-kind/size/reference mismatches.
- `tests/integration/test_postgresql_memory_projection.py` — disposable
PostgreSQL: claim is durably associated with an information object
through the canonical path; eligible memory and its ciphertext-only
protected envelope resolve after a genuinely fresh engine/factory
construction over committed state (writer engine disposed before a new
reader engine is built); denied privacy prevents remembered content.
- `tests/unit/test_memory_retrieval_service.py` — supersession
exclusion.

## Validation

All repository quality gates pass locally:

- `ruff check .` and `ruff format --check .` — clean.
- `mypy src tests tools` — success.
- `validate-repository` — passed (architecture, canon, current state,
automation, queue, agent files, prompt files).
- `uv lock --check` — clean.
- `alembic heads` — `phase_10a_memory_projection (head)`.
- `pytest tests/unit` — 4908 passed, 2 skipped.
- `STEVE_OS_RUN_POSTGRES_TESTS=1 pytest -m postgresql` — 383 passed.

## Residual gaps (reported, not worked around)

1. `project_confirmed_memory` correctly requires a caller-supplied
`ContextProjectionInput` carrying established authorization and privacy
decisions. The runtime `ConfirmedMemoryReader.confirmed_memory(actor_id,
conversation_id)` contract carries only actor and conversation identity,
which is not sufficient authorization to assemble that projection.
`EmptyConfirmedMemoryReader` therefore stays fail-closed with an updated
docstring. Wiring memory into `GovernedConversationRequestBuilder`
requires a separate governance decision about which privacy class may be
exposed to the response model and how established disclosure decisions
assemble an authorized projection — that decision is out of scope here
and must be approved separately.

2. The `write_candidate()` path does not yet atomically create the
claim-to-information-object association or the protected-value envelope.
Conversational memory write/read continuity is therefore not closed by
these read prerequisites alone; that write orchestration is an explicit
residual integration gap.

---

_This PR was drafted by an AI agent (OpenHands) on behalf of the issue
author._

---------

Co-authored-by: openhands <openhands@all-hands.dev>
2026-08-17 07:35:57 -07:00
.github phase-3b5-governance-disclosure-repositories: Add governance disclosure repositories (#94) 2026-08-04 20:23:48 -07:00
canon Complete canon authority and receipt chains 2026-07-27 04:21:04 -07:00
config [Phase 10A A.19] Gate A acceptance package (#293) 2026-08-16 21:29:43 -07:00
deploy [Phase 10A A.18] Complete restricted-live collection runtime (#294) 2026-08-16 20:48:36 -07:00
docs [Phase 10A A.19] Gate A acceptance package (#293) 2026-08-16 21:29:43 -07:00
src/steve_os Implement governed memory projection prerequisites for restricted-live runtime (#298) 2026-08-17 07:35:57 -07:00
tests Implement governed memory projection prerequisites for restricted-live runtime (#298) 2026-08-17 07:35:57 -07:00
tools [Phase 10A A.18] Complete restricted-live collection runtime (#294) 2026-08-16 20:48:36 -07:00
.gitignore Add Phase 1 Python project foundation 2026-07-27 04:42:38 -07:00
.python-version Add Phase 1 Python project foundation 2026-07-27 04:42:38 -07:00
AGENTS.md Rename the Python and repository namespace to Steve OS (#62) 2026-08-02 21:31:12 -07:00
pyproject.toml [Phase 10A A.19] Gate A acceptance package (#293) 2026-08-16 21:29:43 -07:00
README.md [DIRECT] Phase 10A Gate A.1 provider and credential readiness (#211) 2026-08-13 03:55:17 -07:00
uv.lock Phase 10A Gate A.13 bounded production HTTPS transports (#253) 2026-08-14 09:06:44 -07:00

Current state (2026-08-13): Steve OS is merged and repository-complete through the Minimum Useful Steward (MUS) Phase 7-9 implementation tranche, and the Phase 10A planning architecture/gate sequence is canonical. The product owner has authorized Phase 10A Gate A repository live-readiness implementation as offline repository work only; Gate A.1 provider-binding/credential-reference readiness is the current bounded slice. Nothing in that authorization deploys Steve, creates credentials, enables providers, applies retained migrations, provisions infrastructure, or permits live traffic. No operational gate is authorized.

Steve OS

Steve OS is the complete product and system users experience. Core Steve is its persistent identity-bearing and continuity-bearing center. Deterministic platform services, domain systems, Channel Edges, Connectors, runtime composition, the Control Room, infrastructure, and future TRON boundaries are parts of Steve OS rather than separate Steves.

Models may help Steve reason, classify, interpret, or compose language. They are not Steve, and model output is evidence rather than authority. Deterministic policy, privacy, provenance, authorization, correction, and effect boundaries remain outside model control.

Minimum Steve

The current repository contains the safe foundation for the Minimum Useful Steward:

  • natural conversation through a provider-neutral conversation stack;
  • durable conversational continuity and governed memory;
  • append-oriented correction, contradiction, supersession, and retraction of memory claims;
  • generic collection semantics with comics and vinyl as reference profiles;
  • safe bounded image evidence intake;
  • photo-assisted collection evidence that remains confirmation-required and non-authoritative;
  • bounded messaging Channel Edge contracts with replay, route, privacy, and transmission controls;
  • read-only collection metadata and market-evidence contracts;
  • minimum runtime composition with truthful degraded/not-ready states; and
  • a thin browser Control Room that has zero authority of its own.

This is repository capability, not a live service. Phase 10A Gate A repository live-readiness is now the active implementation milestone; infrastructure, staging, and restricted-live activation remain separate later gates.

Repository status

Area Status
Phase 0-2 foundation, persistence, evidence, and durable work complete
Phase 3 identity, contextual authorization, and privacy complete
Phase 4 Core Steve conversation and continuity complete
Phase 5 governed memory and collection semantics complete
Phase 6 broad task/action/schedule system planned, generally post-MUS
Phase 7 safe visual evidence complete at repository level
Phase 8 Channel Edge and read-only collection evidence complete at repository level
Phase 9 runtime composition and zero-authority Control Room complete at repository level
Phase 7-9 cumulative security/privacy closure complete at repository level
Phase 10A restricted-live MUS Gate A repository implementation in progress; operational gates not activated

The canonical sequence and status live in the roadmap. Current product-owner decisions live in approved decisions.

Start here

  1. Development roadmap
  2. Approved decisions
  3. Architecture blueprint
  4. System boundaries
  5. Approval gates
  6. Local development
  7. Repository guardrails
  8. Minimum Useful Steward ADR
  9. Browser Control Room ADR
  10. Phase 7-9 cumulative review

Historical schema reference anchors

These schema notes remain linked from the README because migration verification treats their discoverability as a durable repository contract:

Repository shape

src/steve_os/
    core_steve/   # Steve identity, personality, voice, and conversational continuity
    domain/       # shared deterministic domain primitives
    application/  # bounded orchestration and policy-respecting services
    ports/        # inward-owned capability contracts
    adapters/     # bounded implementations and provider edges
    runtime/      # composition and readiness contracts
    control_room/ # zero-authority browser presentation/service boundary
canon/            # deterministic fictional test canon
docs/             # architecture, ADRs, phase records, and governance
config/           # repository and development guardrails
tests/            # unit, integration, adversarial, and disposable-PostgreSQL evidence
tools/            # repository development/validation helpers

Development

Steve OS is a Python project managed with uv. The repository's committed configuration is authoritative for exact commands and versions. The usual local workflow is:

uv sync --locked --dev
uv run python -m steve_os.repository_validation
uv run pytest

Pull requests are expected to remain bounded, reviewable, and green under the repository's locked Quality workflow. Real providers, credentials, retained databases, migrations against retained targets, infrastructure, deployment, enablement, shadow operation, and live operation remain distinct actions rather than side effects of merging code.

Permanent safety and architecture rules

  • Evidence is not authority.
  • Transport identity is not authentication or permission.
  • Privacy/disclosure permission must be deterministic, minimum-necessary, and bound to the exact context being disclosed.
  • Models, interpreters, metadata sources, market evidence, and the Control Room cannot self-authorize.
  • Work, edition/issue/pressing/variant, and physical-copy identities remain distinct.
  • New knowledge should usually create data; new capabilities create code.
  • Missing mandatory runtime dependencies fail closed. Optional dependencies report truthful degradation.
  • The Control Room is presentation and request collection only; authoritative state and effects live behind Steve OS interfaces.
  • Repository completion never silently becomes operational authority.

Historical verification anchors

The repository keeps detailed historical records under docs/ and in Git history. These anchors are retained here because older cumulative validators verify that living documentation still reconciles correctly:

  • Phase 3A through Phase 3D are merged and complete. Phase 3C.8 is historical merged work. Phase 3D cumulative review merged through PR #113 at 52c7a01325dd3ef7834a092f4f7c88de93984cf8.
  • Phase 3E is activated through governance PR #114 and merged through PR #117 at f4f54026bb54bc45194f54f371cad68e031402c9. Phase 3 is complete.
  • Phase 4 reconciliation and queue planning is historical completed work. Corrective repairs 4B.3a and 4D.1a merged at 0434e30150fef3608bd4e212f9bc3cf6a125ce3f and 05de3a4d42d242954ea561e050db3c8ea24dd5ef; Phase 4E merged through PR #162 at 8c04c97d9ef17992e2bb1075215e9c9e376b16b8.
  • Historical pre-Phase-5 governance recorded that Phase 5 production implementation remains unauthorized. That statement is retained only as a test/reconciliation anchor and is superseded by Phase 5 repository completion.
  • Phase 5C.1 proposal-flooding repair merged through PR #185 at 976973cf05dd418036b1b3153601b439b923e0eb; Phase 5 is complete at the repository level.
  • MUS Phase 7-9 delivery culminated in PR #204 at d9e04fc056a572ddb67b325647d1678784748f3d, followed by the post-repair cumulative closure recorded in the current repository.

These historical facts do not create present operational permission. No operational gate is authorized.