Skip to content

feat(learning): add Azure decision connectors - #314

Open
alliscode wants to merge 15 commits into
build/hub/newfrom
alliscode-azure-decision-connectors
Open

alliscode wants to merge 15 commits into
build/hub/newfrom
alliscode-azure-decision-connectors

Conversation

@alliscode

Copy link
Copy Markdown
Contributor

Why

Replica needs a practical Azure reference for the customer-operated decision runtime. The runtime must keep private training data, model artifacts, credentials, and infrastructure charges inside the customer's Azure subscription while preserving the provider-neutral lifecycle merged in #297 and #298.

What changes

This adds Azure adapters for private immutable Blob artifacts, durable PostgreSQL job and promotion state, bounded GPU job execution, generation-protected inference, managed identity credentials, and redacted operational signals.

flowchart LR
  Hub["Replica Hub"] --> Runtime["Customer decision runtime"]
  Runtime --> Blob["Private Azure Blob storage"]
  Runtime --> Postgres["Existing Replica PostgreSQL"]
  Runtime --> Broker["Fixed-template training broker"]
  Broker --> GPU["Scale-to-zero GPU job"]
  Runtime --> Endpoint["Customer inference endpoint"]
Loading

Terraform adds the private decision container, split workload identities, an exact non-delete Blob data role, broker and inference grants, retention policy, and deployment coordinates. A separate withdrawal identity permanently deletes selected Blob versions; runtime and training identities cannot delete them.

The connector and infrastructure tests use mocks and require no cloud credentials. A real Azure account is still required to verify regional GPU quota, the customer-provisioned job and broker, RBAC propagation, private connectivity, measured cost, cold starts, permanent withdrawal deletion, restore, outage, quota-exhaustion, and teardown behavior. This pull request does not claim a live Azure deployment pass.

Ben Thomas and others added 15 commits November 19, 2026 13:58
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@alliscode
alliscode requested review from a team and nshki as code owners November 19, 2026 22:53
Copilot AI adaptive review requested due to automatic review settings November 19, 2026 22:53

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved critical flaws allow Hub to modify decision-runtime state, permit filesystem writes outside the private cache, and let expired workers update jobs.

15 open issues
What changed in this PR

Adds a customer-operated Azure reference bundle for the decision runtime, building on the provider-neutral lifecycle, budget, and generation contracts from #297 and #298.

Changes:

  • Adds Azure storage, PostgreSQL state, training, inference, identity, and telemetry connectors with mocked contract coverage.
  • Passes inference generations and cooperative cancellation signals to workers.
  • Adds optional Azure infrastructure and operating guidance; live cloud verification remains outstanding.
File Description
packages/​learning/​environments/​brand-check/​tests/​test_runtime.py Adds worker cancellation and timeout coverage.
packages/​learning/​environments/​brand-check/​tests/​test_azure_connectors.py Adds mocked Azure connector and conformance coverage.
packages/​learning/​environments/​brand-check/​runtime.py Propagates inference generations and worker cancellation.
packages/​learning/​environments/​brand-check/​README.md Documents Azure adapters and operational boundaries.
packages/​learning/​environments/​brand-check/​azure_connectors.py Implements the Azure connector bundle.
packages/​learning/​environments/​brand-check/​__init__.py Exposes Azure APIs through lazy exports.
infrastructure/​README.md Documents provisioning and required cloud verification.
infrastructure/​azure/​variables.tf Adds decision-runtime configuration inputs.
infrastructure/​azure/​tests/​deployment.tftest.hcl Checks decision storage, access grants, and identity wiring.
infrastructure/​azure/​storage.tf Adds decision storage, retention, and permanent-delete support.
infrastructure/​azure/​roles.tf Defines decision identities and scoped access grants.
infrastructure/​azure/​outputs.tf Exports storage, job, and identity coordinates.
infrastructure/​azure/​application.tf Adds decision identities and configuration to Hub.

Review effort: Max (adaptive)

Comment on lines +80 to +82
var.enable_decision_runtime ? [
azurerm_user_assigned_identity.decision_runtime[0].id,
azurerm_user_assigned_identity.decision_inference[0].id,
delete_retention_policy {
days = 14
days = 14
permanent_delete_enabled = true


AZURE_POSTGRES_SCHEMA = """
CREATE TABLE IF NOT EXISTS replica_decision_jobs (
SET record = %s::jsonb, version = version + 1
WHERE scope_key = %s AND job_id = %s AND version = %s
AND lease_token = %s AND lease_owner_kind = %s
AND lease_owner_id = %s AND lease_expires_at > %s
Comment on lines +1493 to +1495
directory = self._cache_root / candidate_id
directory.mkdir(parents=True, exist_ok=True)
path = directory / name
Comment on lines +1655 to +1669
try:
self.cancellation_recovery_hook(execution_id, context)
except Exception as error:
raise ConnectorError(
"azure_training_cancellation_unrecorded",
"Azure training cancellation could not be scheduled for recovery.",
category="internal",
) from error
for attempt in range(3):
try:
self.transport.cancel(execution_id, bearer_token=token)
return
except Exception:
if attempt < 2:
self.sleep(0.1 * (attempt + 1))
Comment on lines +1765 to +1769
dict(
request,
endpoint=self.endpoint_name,
expectedGeneration=expected_generation,
),
Comment on lines +2104 to +2119
if (
context.budget_mode == "hard"
and context.max_credits is not None
and sum(
item.quantity
for item in usage
if item.kind == "training_credits" and item.unit == "credits"
)
> context.max_credits
):
raise ConnectorError(
"credit_budget_exceeded",
"Azure reported usage above the hard credit budget.",
category="integrity",
)
return training
Comment on lines +824 to +830
answers = self.connector.predict(
adapter,
dict(request, expectedGeneration=1),
context=context,
credential=credential,
)
return dict(answers, operationId=context.operation_id)
Comment on lines +971 to +978
for name in tuple(blobs.blobs):
blobs.upload_blob(
name,
blobs.blobs[name]["data"],
overwrite=True,
metadata=blobs.blobs[name]["metadata"],
)
blobs.versions[name].insert(0, {"id": "already-deleted"})
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants