Security Hardening

Overview

OpenTela is a decentralized network where untrusted operators can run nodes. The security model addresses four concerns:

1. Binary integrity — Is this node running an official, unmodified binary?
2. Operator identity — Does the node operator actually control the wallet they claim?
3. Abuse prevention — How do we limit request floods and resource exhaustion?
4. Transport confidentiality — Is traffic between nodes encrypted?

Each layer is independently configurable and backward-compatible with older nodes that pre-date these features.

1. Build Attestation

How it works

During a release build, the CI pipeline signs the string version|commitHash with a maintainer Ed25519 private key. The hex-encoded signature is injected into the binary via ldflags (-X main.buildSig=...). The corresponding public key is compiled into every binary.

At runtime:

• On startup, the node packages its version, commit hash, and signature into a BuildAttestation field on its CRDT peer record.
• When any node receives a peer record, it verifies the signature against the embedded maintainer public key.
• The result is stored in Peer.SignedBuild (a locally-computed boolean, not trusted from the network).

Configuration

security:
  require_signed_binary: false   # default: accept unsigned peers with a warning

When true, peers without a valid build attestation are rejected from the node table entirely — they cannot participate in routing.

When false (default), unsigned peers are accepted but Peer.SignedBuild is set to false, and a warning is logged. This allows gradual rollout without breaking existing deployments.

Setup (one-time)

cd src

# Generate a maintainer Ed25519 keypair
go run ./internal/attestation/cmd/buildsign keygen
# Output:
#   public:  <hex>
#   private: <hex>

1. Put the public key in internal/attestation/attestation.go → maintainerPubKeyHex constant.
2. Store the private key as a GitHub Actions secret named BUILD_SIGN_KEY.

The release workflow (.github/workflows/release.yml) will automatically sign builds when BUILD_SIGN_KEY is set.

Files

2. Node Trust Model

Trust levels

How identity attestation works

On startup, if the node has a wallet configured, it creates an IdentityAttestation:

{
  "peer_id": "QmABC...",
  "wallet_pubkey": "5xyz...",
  "timestamp": 1741564800,
  "signature": "<base64 Ed25519 signature over the JSON of the above three fields>"
}

This is published in the CRDT peer record. When another node receives this record, it:

1. Reconstructs the signed payload from {peer_id, wallet_pubkey, timestamp}.
2. Decodes the wallet_pubkey (base58) into an Ed25519 public key.
3. Verifies the signature.
4. If valid, sets TrustLevel = 1. Then checks if the wallet is in trusted_wallets or matches the local wallet for elevation to level 2.

Trust-aware routing

Clients can request a minimum trust level via the X-Otela-Trust HTTP header:

X-Otela-Trust: 1

The head node filters candidate workers to only include peers at or above the requested trust level. If no peers qualify, it returns 503 Service Unavailable with a descriptive error — it does not silently fall back to untrusted peers.

Default (no header or X-Otela-Trust: 0): all connected peers are eligible, regardless of trust level.

Configuration

# Wallets you explicitly trust (grants trust level 2 to matching peers)
trusted_wallets:
  - "5YourFriendsPubkey..."
  - "7AnotherOperator..."

# Your own wallet (self-trust: peers with the same wallet get level 2)
wallet:
  account: "YourOwnPubkey..."

Files

3. Worker Access Control

Overview

Node operators can control which peers are allowed to forward requests to their worker node. This is enforced as Gin middleware on the /_service endpoints — the internal routes that receive forwarded requests from head nodes via libp2p.

Policies

Configuration

security:
  access_control:
    policy: "any"              # "any" | "self" | "whitelist" | "blacklist"
    whitelist:                 # used when policy = "whitelist"
      - "5WalletPubkey1..."
      - "7WalletPubkey2..."
    blacklist:                 # used when policy = "blacklist"
      - "5BannedWallet..."

How it works

1. When a forwarded request arrives at /_service, the middleware extracts the remote libp2p peer ID from Request.RemoteAddr.
2. It looks up the peer in the node table to find their wallet address (Peer.Owner).
3. It checks the wallet against the configured policy.
4. If denied, returns 403 Forbidden with a descriptive error. If allowed, the request proceeds to ServiceForwardHandler.

Examples

Only serve requests from your own head nodes:

security:
  access_control:
    policy: "self"

Serve requests from a consortium of known operators:

security:
  access_control:
    policy: "whitelist"
    whitelist:
      - "5PartnerA..."
      - "7PartnerB..."
      - "9PartnerC..."

Block a known bad actor:

security:
  access_control:
    policy: "blacklist"
    blacklist:
      - "5MaliciousWallet..."

Files

4. API Key Authentication (Bearer Tokens)

Problem

In the standard flow (Client → Head node → Worker), the client's identity is lost — the worker only sees the head node's libp2p peer ID. Clients who don't run their own node have no way to prove their wallet identity.

Solution

A separate FastAPI auth service issues API keys bound to wallet addresses. The flow is:

1. Client proves wallet ownership → Auth server issues a bearer token
2. Client includes "Authorization: Bearer <token>" in requests to the head node
3. Head node calls auth server to verify token → gets wallet pubkey
4. Head node injects "X-Otela-Client-Wallet: <wallet>" when forwarding to worker
5. Worker access control checks the end-user's wallet (not just the head node's)

Auth server

Located in auth/. Managed with uv.

cd auth
uv sync                                    # install deps
uv run uvicorn auth.server:app --port 8090 # start server
uv run pytest test_server.py -v            # run tests

Endpoints:

Key creation requires the client to sign a challenge string (otela-auth:<wallet>:<timestamp>) with their Ed25519 wallet key. This prevents someone from creating keys for wallets they don't control.

Tokens are SHA-256 hashed before storage — the raw token is returned only once at creation time.

CLI

cd auth

# Create a key (signs challenge automatically with your keypair)
uv run python -m auth.cli create --keypair ~/.config/opentela/accounts/<pubkey>/keypair.json

# List your keys
uv run python -m auth.cli list --wallet <pubkey>

# Revoke a key
uv run python -m auth.cli revoke --wallet <pubkey> --key-id okey_abc123

# Verify a token
uv run python -m auth.cli verify --token otela_...

Head node configuration

security:
  auth_url: "http://localhost:8090"   # URL of the auth server

When auth_url is set, the head node extracts the Authorization: Bearer <token> header from client requests, verifies it against the auth server, and injects X-Otela-Client-Wallet into the forwarded request.

Verified wallets are cached for 60 seconds to avoid hitting the auth server on every request.

Files

5. Rate Limiting

How it works

An in-memory per-IP token-bucket rate limiter sits as Gin middleware before all HTTP handlers. Each client IP gets an independent bucket. Stale entries (no requests for 3 minutes) are automatically evicted.

When the limit is exceeded, the server returns:

HTTP 429 Too Many Requests
Retry-After: 1
{"error": "rate limit exceeded"}

Configuration

security:
  rate_limit:
    enabled: true              # default: false
    requests_per_second: 100   # default: 100
    burst: 200                 # default: 200

Disabled by default. When enabled, the middleware is active for all endpoints (health, CRDT, P2P forwarding, service forwarding).

Files

6. Transport Encryption

Current state

libp2p is configured with two security transports:

• TLS 1.3 (/tls/1.0.0) — preferred
• Noise (/noise) — fallback

Both are registered in host.go. libp2p automatically negotiates the strongest mutually-supported protocol on every connection. All inter-node traffic is encrypted by default.

The negotiated security protocol is logged on each connection event:

Connected to peer: QmABC...  security=/tls/1.0.0  Total connections: 5

PSK (Private Network)

Code for pre-shared key network isolation exists in host.go but is currently commented out. When enabled, only nodes sharing the same PSK can form a network. This is a separate concern from build attestation — PSK isolates the network, while build attestation verifies binary integrity.

Caveats and Limitations

Build attestation is software-only

A modified binary can report a fake BuildAttestation with a valid signature copied from a legitimate build of the same version. Build attestation prevents:

• Accidental use of unofficial or development builds in production
• Naive tampering (modifying the binary without updating the attestation)

It does not prevent a determined attacker who patches the binary to replay a legitimate attestation. True binary integrity requires hardware-backed remote attestation (Intel TDX, AMD SEV-SNP), which is not yet implemented.

Identity attestation proves key ownership, not identity

IdentityAttestation proves that the node operator controls a particular Ed25519 private key. It does not prove:

• Who the operator is (that requires KYC, which is not yet implemented)
• That the operator is operating the node honestly
• That the node's hardware or software has not been tampered with

A malicious operator with a valid wallet can still run a compromised node that passes identity attestation at trust level 1.

Trust level is computed locally

TrustLevel is not a network-wide consensus value. Each node computes it independently based on its own trusted_wallets config. This means:

• The same peer may have TrustLevel = 2 on one head node and TrustLevel = 1 on another, depending on their respective trust lists.
• A head node's trust decisions are only as good as its own configuration.

Auth server is a centralized component

The API key auth service is a single centralized server. This is a deliberate trade-off — key management is simpler and more maintainable as a centralized service than as a decentralized protocol. However:

• If the auth server is down, head nodes cannot verify new bearer tokens (cached tokens continue to work for 60 seconds).
• The auth server's database is the source of truth for API keys. It should be backed up and access-controlled.
• The X-Otela-Client-Wallet header is set by the head node after verification. A malicious head node could inject an arbitrary wallet. Workers should only trust this header from head nodes they trust (combine with worker access control).

Access control depends on wallet resolution

The worker access control middleware identifies callers by looking up the remote libp2p peer ID in the node table and reading Peer.Owner. This has limitations:

• If the caller peer hasn't propagated its record via CRDT yet (e.g., a brand-new node), its wallet is unknown and resolves to "". Under whitelist policy, this means new peers are denied until their record propagates. Under blacklist, they are allowed (since "" is not in the blacklist).
• The Owner field is self-reported. A malicious head node could claim any wallet address. Access control is therefore only meaningful when combined with identity attestation (trust level >= 1), which cryptographically proves wallet ownership.
• Access control is enforced on the worker's /_service endpoints only. It does not prevent a peer from connecting at the libp2p layer or appearing in the node table — it only blocks request forwarding.

Rate limiting is per-node, not network-wide

Each node enforces rate limits independently. A distributed attacker could spread requests across multiple head nodes to circumvent per-node limits. Network-wide rate limiting would require coordination (e.g., shared state via CRDT), which is not implemented.

Rate limiting is also IP-based only. Clients behind a shared NAT or proxy will share the same bucket. Per-wallet rate limiting is a planned future enhancement.

SignedBuild and TrustLevel are in the JSON peer record

Both signed_build and trust_level are present in the serialized JSON peer record that propagates via CRDT. However, receiving nodes always recompute these values locally from the raw attestation data — they are never trusted from the network. A malicious node setting "signed_build": true or "trust_level": 3 in its record has no effect on other nodes.

Backward compatibility

• Nodes running versions before these security features will have BuildAttestation = nil, IdentityAttestation = nil, and TrustLevel = 0.
• They can still join the network and serve requests unless security.require_signed_binary is set to true.
• Different signed versions (e.g., v1.0.0 and v2.0.0) can communicate freely — the attestation verifies the maintainer's signature, not a specific version hash.
• The X-Otela-Trust header is opt-in. Clients that don't send it get the previous behavior (all peers eligible).

No revocation mechanism

There is currently no way to revoke a compromised maintainer signing key or a compromised wallet. If the maintainer key is leaked:

• All binaries signed with it remain "valid" until the public key constant is rotated and a new release is cut.
• Old nodes will still accept the compromised key until they upgrade.

Similarly, there is no wallet blocklist. A compromised wallet at trust level 2 (in someone's trusted_wallets) stays trusted until manually removed from the config.