Arija A. 3e7e9c6711

Signed-off-by: Arija A. <ari@ari.lt>

2025-07-27 02:00:02 +03:00

16 KiB

Raw Permalink Blame History

DeployD Protocol Specification (dproto), version 0

Abstract
Conventions and Definitions
Protocol Overview
Packet Encoding
COMMAND Packet
Keepalive: PING / PING_REPLY
Access Control: ALLOWED / READY
Logging: LOG / LOGS_END
EXIT Packet
ERROR Packet
Token Generation Algorithm
Proof-of-Work Algorithm
Security Considerations
Authors

1. Abstract

This document specifies dproto - a (near-)stateless protocol with a goal for cryptographic continuous deployment, DeployD's encrypted, authenticated, small, binary, little-endian, streaming protocol. It defines every packet format; enforces exact timeout values, strict length fields, and precise error-handling behaviours; and describes the proof-of-work handshake, command semantics, log streaming, and token generation with full determinism.

2. Conventions and Definitions

bool :Exactly 1 octet. Permitted values: 0x00 (false), 0x01 (true). Any other value MUST be interpreted as false.
Endianness: All multibyte integers are encoded in little-endian order.
Timeouts: Both read and write timeouts SHALL be more or equal to 5 seconds. The client and the server's connection immediately terminates on exhaustion.
Connection: A single SSL/TLS session over TCP, reused for the entire exchange.
Streaming: Data may be sent in continuous back-to-back packets; boundaries are defined solely by the packet length fields.

3. Protocol Overview

3.1 Connection Establishment

Client performs standard SSL/TLS handshake.
Upon handshake completion, the client MUST immediately read the first packet of up to 256 bytes.

3.2 Server Information Exchange

Packet

struct {
    uint8_t version;                      /* MUST be 0x00 (for this version) */
    uint8_t server_info_len;              /* MUST be in [1; 255] */
    uint8_t server_info[server_info_len];
};

Requirements
- server_info_len SHALL be above or equal to 1, but not more than 255.
- Server MUST send exactly 1 + server_info_len octets.
- server_info contains raw data without any terminator.
- If fewer or more bytes arrive, the client MUST optionally initiate the EXIT packet and drop the connection.
- If the client does not support the given version, the client MUST optionally initiate the EXIT packet and drop the connection.
- The server MUST send the maximum supported version, with no gaps in support of lower versions. Dproto guarantees backwards compatibility.
- The client SHOULD choose the highest supported version, even if it is before version. Universal behaviour is guaranteed.

3.3 Proof-of-Work Challenge

Packet

struct {
    uint8_t type = 0x10; /* PING */
    /* or */
    uint8_t type = 0x13; /* READY */
};

(client may only send PING or READY during PoW stage)

Challenge Packet

struct {
    uint8_t challenge[16]; /* 16 cryptographically secure random bytes */
    uint8_t difficulty;    /* MUST be above or equal to 1 */
    uint8_t ones;          /* MUST be above or equal to 1 */
};

Rules
- Server sends exactly 16bytes for challenge, plus 2 bytes for difficulty and ones. These constitute PoW parameters.
- Client may issue up to 64 PINGs, at most one per second. This is to avoid locking a worker/socket for too long and avoid spam.
- Each PING elicits a PING_REPLY; unmatched PINGs trigger error code 0x2004.
- Excess PINGs trigger error code=0x3000.
- The purpose of PoW is to avoid DoS/DDoS attacks by providing a robust layer against spam, hence why difficulty and ones MUST be at least 1.

3.4 Proof-of-Work Solution

Client computes nonce in range of [0; 18446744073709551615] satisfying the challenge and difficulty.

Client sends:

struct {
    uint8_t type = 0x13; /* READY */
    uint8_t nonce[8];    /* 64-bit little-endian */
};

The nonce search algorithm is defined in § 12.

Server verifies:
- If valid: replies with ALLOWED (type=0x12).
- If invalid: replies with ERROR (code=0x3001), optionally followed by EXIT, and the connection is dropped.
- On any other errors a different ERROR may be issued.

3.5 Packet Phase

After ALLOWED, client may send up to 64 packets in same connection. This is to avoid locking a worker/socket for too long and avoid spam.
No interleaving proof-of-work messages allowed once in packet phase.

3.6 Termination

Any side may send:

struct { uint8_t type = 0x30; } /* EXIT */

Upon sending or receiving EXIT, both sides MUST immediately close the connection; in-flight data may flush, in which case the server MUST not process further, and the client MAY choose to process specific packets, however, this is not a guarantee.
The EXIT packet is optional, an "implicit" shutdown is done after a timeout or disconnection.
At any point the server or the client can send a PING packet which MUST be followed by a PING_REPLY packet.

3.7 Chart

4. Packet Encoding

4.1 Generic Packet Structure

struct {
    uint8_t type;
    uint8_t *data; /* Length derived from packet type definition */
};

4.2 Token Types

Name	Value	Direction	Content
COMMAND	`0x00`	c2s	See §5
PING	`0x10`	bid.	No payload
PING_REPLY	`0x11`	bid.	No payload
ALLOWED	`0x12`	s2c	No payload
READY	`0x13`	bid.	As specified in context
LOG	`0x20`	s2c	See §8
LOGS_END	`0x21`	s2c	No payload
EXIT	`0x30`	bid.	No payload
ERROR	`0xFF`	bid.	See §10

Note: "c2s" refers to "client to server", "bid." to "bidirectional (both c2s and s2c)", and "s2c" to "server to client", for brevity.

4.3 Error Codes and Formats

struct {
    uint8_t  type = 0xFF;
    uint16_t msg_len;      /* MUST be above or equal to 1, in bytes */
    uint16_t error_code;   /* See table */
    uint8_t  msg[msg_len]; /* UTF-8 text; no NULL terminator */
};

Name	Value	Description
Internal	`0x0000`	Internal error
Type	`0x0001`	Unexpected packet type
Status	`0x0002`	Invalid status field
AuthToken	`0x1000`	Authentication token invalid
AuthKey	`0x1001`	Authentication key invalid
ProtoPacketTooShort	`0x2000`	Packet too short
ProtoDomainInvalid	`0x2001`	Invalid domain format
ProtoPacketTooLong	`0x2002`	Packet too long
ProtoDomainNotFound	`0x2003`	Domain not found
ProtoPacketInvalid	`0x2004`	Malformed or invalid packet
PowTooManyPings	`0x3000`	Too many PINGs during PoW
PowBadSolution	`0x3001`	Invalid PoW solution
DeployError	`0x4000`	Deployment execution error
InvalidCommand	`0x4001`	Invalid COMMAND sub-type

5. `COMMAND` Packet

struct {
    uint8_t  type    = 0x00;
    uint8_t  command;            /* See table */
    bool     is_unsafe;          /* 0x00 or 0x01 */
    uint64_t id;                 /* Pre-shared semi-secret */
    uint8_t  domain_len;         /* MUST be more or equal to 1 */
    uint8_t  domain[domain_len]; /* Raw bytes, no terminator */
    uint8_t  key[32];            /* Pre-shared secret */
    uint8_t  token[16];          /* From §11 */
};

Command	Value	Description
Trigger	`0x00`	Trigger action
Teardown	`0x01`	Tear down deployment
Deploy	`0x02`	Initiate deployment
Rollback	`0x03`	Roll back to prior version
Cleanup	`0x04`	Clean up artefacts
Restart	`0x05`	Restart service
Sysadmin	`0x06`	System administration tasks
Logs	`0x07`	Request latest deployment logs

After COMMAND, server streams zero or more LOG packets, then LOGS_END.
Commands lock the domain, and new deploys WILL be waited on if triggered when locked.
- The domain lock established by a COMMAND packet of type Deploy SHALL remain active for the entire duration of the associated deployment process. This lock MUST persist even if the initiating client disconnects, and it SHALL only be released upon successful or failed completion of the deployment.
All secrets MUST be generated out-of-band.

6. Keepalive: `PING` / `PING_REPLY`

struct { uint8_t type = 0x10; }; /* PING */
struct { uint8_t type = 0x11; }; /* PING_REPLY */

No payload.
Every PING MUST be followed by corresponding PING_REPLY by the receiver before any other message type.

7. Access Control: `ALLOWED` / `READY`

struct { uint8_t type = 0x12; }; /* ALLOWED */
struct { uint8_t type = 0x13; }; /* READY */

READY carries no payload except when used to send the 8-byte PoW nonce as specified in §3.4.

8. Logging: `LOG` / `LOGS_END`

struct {
    uint8_t  type        = 0x20;
    uint16_t chunk_size;         /* MUST be more or equal to 1 */
    uint8_t  chunk[chunk_size];  /* Opaque */
};
struct { uint8_t type = 0x21; }; /* LOGS_END */

chunk_size is exact byte count; logs are raw UTF-8 or binary data.
All LOG packets MUST be sent in order.

9. `EXIT` Packet

struct { uint8_t type = 0x30; };

No payload. Both peers MUST close the SSL connection immediately.

10. `ERROR` Packet

Defined in §4.3. Error packets may appear at any stage. On receiving ERROR, the receiver SHOULD cease current stage; on critical errors, server may follow with EXIT.

11. Token Generation Algorithm

Inputs:
- 64-bit UNIX epoch reference T (seconds).
- 24-byte URL-safe secret (characters A-Z, a-z, 0-9, _, -).

Compute:

delta   = now() - T
counter = floor(delta / 300)

Derive:
- Initialize BLAKE2s with key=secret, digest_size=16.
- Update with counter as 8-octet little-endian.
- Finalize to obtain 16-octet token.
Assuming:
- All secrets are provisioned out-of-band.
- The clock is in-sync.

Example (Python):

import hashlib
import struct
import time


def gen_token(secret: bytes, timestamp: int) -> bytes:
    """Generate a token"""

    return hashlib.blake2s(
        struct.pack("<Q", int((time.time() - timestamp) / 300)),  # data
        digest_size=16,
        key=secret,
    ).digest()

The server MAY accept a counter drift +-1, but it is NOT guaranteed.

12. Proof-of-Work Algorithm

Inputs:
- 16-byte challenge (opaque, secret, server-generated).
- 8-bit difficulty: number of required leading zero bits.
- 8-bit ones: number of required XOR-passing bytes.
Valid solution:

A nonce is considered valid if:
- The 32-byte BLAKE2s digest of the ASCII decimal nonce (keyed with the challenge) has at least difficulty leading 0 bits.
- At least ones out of the first 32 digest bytes satisfy: XOR with the challenge byte (modulo 16) contains more or equal to 6 ones (in binary)

Compute:

For a given nonce:

nonce_string = string(nonce)  # e.g., "12345"
hash = BLAKE2s(nonce_string, key=challenge)

Then verify:

has_leading_zero_bits(hash, difficulty) == true
AND
has_xor_ones(hash, ones, challenge)     == true

Bit-Checking Logic:
- Leading-zero bits are checked by verifying that:
  - The full zero bytes match.
  - The remaining prefix bits (if any) are zero via masking.
- XOR byte-pair test passes if:
  - XOR of challenge[i % 16] and digest[i % 32] contains more or equal to 6 one-bits, where i is in range [0;31]
  - Repeat until ones such bytes are found (or fail early)

Example (Python):

import hashlib


def has_leading_zero_bits(h: bytes, difficulty: int) -> bool:
    """Check bit prefix requirements"""

    full_bytes: int = difficulty // 8
    remainder_bits: int = difficulty % 8

    if h[:full_bytes] != b"\x00" * full_bytes:
        return False

    if remainder_bits > 0:
        next_byte: int = h[full_bytes]
        mask: int = 0xFF << (8 - remainder_bits) & 0xFF

        if (next_byte & mask) != 0:
            return False

    return True


def count_ones(num: int) -> int:
    """Fast bit-counting using Hacker's Delight method"""

    num = (num - ((num >> 1) & 0x55)) & 0xFF
    num = ((num & 0x33) + ((num >> 2) & 0x33)) & 0xFF
    return (num + (num >> 4)) & 0x0F


def has_xor_ones(h: bytes, ones: int, challenge: bytes) -> bool:
    """Check XOR requirements"""

    ok_ones: int = 0

    for idx in range(32):
        if count_ones(challenge[idx % 16] ^ h[idx % 32]) >= 6:
            ok_ones += 1

        if ok_ones == ones:
            return True

    return False


def solve_pow(
    difficulty: int, ones: int, challenge: bytes, nonce: int = 0, batch: int = 2**64 - 1
) -> t.Tuple[int, bool]:
    """Solve proof of work"""

    for _ in range(batch):
        if nonce > 2**64 - 1:
            return nonce, False

        h: bytes = hashlib.blake2s(str(nonce).encode("ascii"), key=challenge).digest()

        if has_leading_zero_bits(h, difficulty) and has_xor_ones(h, ones, challenge):
            return nonce, True

        nonce += 1

    return nonce, False

13. Security Considerations

Replay Protection: Unique PoW nonces, time-bounded tokens.
Confidentiality & Integrity: TLS mandatory; BLAKE2s keyed hashing.
DoS Mitigation: Modern and dynamic PoW challenge; ping rate limits.
Strict Parsing: All length fields and value ranges are enforced exactly.
Authentication Failures: Explicit error codes for invalid keys or tokens.
Timeouts: 5s read/write socket timeouts to prevent stalling.
Version Pinning: Unknown protocol versions are rejected.
Domain Locking: Prevents race conditions during client registration.
Immutable Logs: No log mutation; entire stream SHOULD be read.
Silent Failure on Malformed Packets: Unless required, no responses.
One-time Use Nonces: Prevent precomputation or reuse.
Out-of-Band Trust Model: Secrets MUST be securely provisioned.

14. Authors

Arija A. <ari@ari.lt> - Author of dproto.

This protocol is licensed under AGPL-3.0-only and is part of the DeployD project which you can find at https://git.ari.lt/ari/deployd.

More information about the license can be found at https://git.ari.lt/ari/deployd/raw/branch/main/LICENSE.

16 KiB Raw Permalink Blame History