Apache Kafka

Overview#

The Kafka connector proxies produce and consume calls to Apache Kafka (self-hosted or Confluent Cloud). Message values are wrapped in a HybridCryptoEnvelope before they reach the broker. SASL/SCRAM or mTLS credentials are sealed in QuantaVault. Consumer groups receive signed envelopes they can verify using the QuantaSeal SDK.

Prerequisites#

• 1A Kafka cluster (Apache Kafka 2.8+ or Confluent Cloud)
• 2Kafka credentials: SASL/SCRAM username + password, or mTLS client certificate
• 3Bootstrap server addresses
• 4A QuantaSeal API key

Configuration#

Follow these steps to connect Apache Kafka to QuantaSeal. You can configure integrations via the Admin Console or directly via the API.

1. 1

   Create a Kafka service user with produce/consume ACLs on the target topics.

2. 2

   Seal credentials: POST /api/v2/vault/seal with credential_type: kafka_credentials and values: {username, password, bootstrap_servers, security_protocol}.

3. 3

   Create the integration: POST /api/v2/integrations with system_type: kafka.

Authentication Types#

Use kafka_credentials for SASL/SCRAM (username + password + bootstrap_servers). Use mtls_cert for mTLS authentication with a client certificate and private key PEM.

All credential types are sealed in QuantaVault with ML-KEM-768 + AES-256-GCM and wrapped by your tenant AWS KMS CMK before storage. See the Vault API reference for the full list of credential types and seal/unseal endpoints.

Available Operations#

QuantaSeal enforces a default-deny operation policy. Only operations listed in your integration's allowed_operations array will be permitted. Add operations when creating or updating the integration.

Code Example#

Every proxy call returns a HybridCryptoEnvelope - the response is ML-KEM-768 key-encapsulated, AES-256-GCM encrypted, and signed with ML-DSA-65 + HMAC-SHA-512. Verify both signatures before trusting the decrypted payload.

curl -X POST https://api.quantaseal.io/api/v2/proxy/outbound \
  -H "X-API-Key: qs_live_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "integration_id": "int_01HZ9X2K4MNPQRKAFKA000001",
    "operation": "produce",
    "payload": {
      "topic": "customer-events",
      "key": "cust_4821",
      "value": "{"event": "profile.updated", "customer_id": "cust_4821"}",
      "headers": {"source": "qs-proxy"}
    }
  }'

# Response - HybridCryptoEnvelope
{
  "success": true,
  "encrypted": {
    "ciphertext_kem":  "<base64 - 1088 bytes ML-KEM-768>",
    "ciphertext_data": "<base64 - AES-256-GCM encrypted Kafka produce result>",
    "nonce":           "<base64 - 12 bytes>",
    "tenant_id":       "ten_01HZ9X2K4MNPQR5STUVWXYZ00",
    "algorithm":       "ML-KEM-768"
  },
  "signature": {
    "pqc_signature":  "<base64 - ~3309 bytes ML-DSA-65>",
    "hmac_signature": "<base64 - 64 bytes HMAC-SHA-512>",
    "tenant_id":      "ten_01HZ9X2K4MNPQR5STUVWXYZ00",
    "algorithm":      "ML-DSA-65+HMAC-SHA-512"
  },
  "audit_event_id": "aud_01HZ9XABCDEF"
}

Troubleshooting#