AWS Provider

The AWS provider is a production-ready implementation of the cloud provider abstraction, using AWS DynamoDB for storage, Secrets Manager for configuration and secrets, and CloudWatch for metrics.

Prerequisites

AWS Account Requirements

Active AWS account with appropriate permissions
IAM user or role with necessary permissions (see IAM Permissions)
Access to the AWS regions where you want to deploy

Required AWS Services

DynamoDB - Component and API key storage
AWS Secrets Manager - Configuration and API key secrets
CloudWatch (optional) - Metrics publishing

Quick Start

1. Basic Configuration

Add to your application.yml:

hub:
  aws:
    region: us-east-1
    dynamodb:
      prefix: zenoo-hub
      createTables: true
    secrets:
      prefix: zenoo-hub

2. AWS Credentials

The Hub uses the AWS SDK default credential chain. You can provide credentials via: Environment variables:

export AWS_ACCESS_KEY_ID=your-access-key
export AWS_SECRET_ACCESS_KEY=your-secret-key
export AWS_REGION=us-east-1

IAM Role (recommended for EC2/ECS/EKS):

# No explicit credentials needed - uses instance/pod role
hub:
  aws:
    region: us-east-1

Configuration file (not recommended for production):

hub:
  aws:
    accessKey: ${AWS_ACCESS_KEY_ID}
    secretKey: ${AWS_SECRET_ACCESS_KEY}
    region: us-east-1

3. Start the Hub

The Hub will automatically:

Create required DynamoDB tables (if createTables: true)
Set up TTL for the sharables table
Configure multi-region replication (if configured)

DynamoDB Tables

The AWS provider creates and manages three DynamoDB tables:

Components Table

Table Name: {prefix}-components Stores Hub component definitions with versioning support. Schema:

Partition Key: componentName (String) - Component name
Sort Key: revision (Number) - Component revision
Attributes:
- definition (String) - Component DSL definition
- metadata (Map) - Component metadata
- dependencies (List) - Component dependencies
- connectors (List) - Required connectors
- version (Number) - Optimistic locking version

Features:

Point-in-time recovery enabled
Versioned revisions for rollback capability
Optimistic locking for concurrent updates
Global table support for multi-region

Configuration:

hub:
  aws:
    dynamodb:
      prefix: my-hub  # Table name: my-hub-components

API Key Lookup Table

Table Name: {prefix}-api-key-lookup Maintains a lookup index for API keys associated with components. Schema:

Partition Key: componentName (String) - Component name
Attributes:
- secrets (Set<String>) - API key secret names
- exposed (Boolean) - Whether component is exposed
- exposedFunctions (List<String>) - Exposed function names
- version (Number) - Optimistic locking version

Use Cases:

Fast lookup of API keys for a component
Determining if a component is publicly exposed
Managing component access control

Sharables Table

Table Name: {prefix}-sharables Stores temporary sharable tokens (e.g., for magic links, temporary access). Schema:

Partition Key: token (String) - Unique token
Attributes:
- payload (String) - Sharable payload (JSON)
- expiresAt (Number) - Expiration timestamp
- expired (Boolean) - Expiration flag
- ttl (Number) - DynamoDB TTL attribute

Features:

Automatic cleanup via DynamoDB TTL
High-performance token retrieval
Configurable expiration

Configuration:

hub:
  cloud:
    provider:
      sharable:
        defaultTtl: 24h  # Default TTL for sharables

AWS Secrets Manager

The AWS provider stores sensitive configuration and API keys in AWS Secrets Manager.

Component Configuration Secrets

Naming Convention: {prefix}/component-config/{configId.key}/{configId.version} Example: zenoo-hub/component-config/payment-api/1.0 Content Structure:

{
  "api": {
    "endpoint": "https://api.example.com",
    "timeout": 30000
  },
  "credentials": {
    "username": "api-user",
    "password": "secret-password"
  }
}

Features:

Automatic versioning with AWSCURRENT and AWSPREVIOUS stage labels
Multi-region replication
Encryption at rest with AWS KMS
Audit trail via CloudTrail

API Key Secrets

Naming Convention: {prefix}/api-keys/{keyName} Example: zenoo-hub/api-keys/client-api-key Content Structure:

{
  "apiKey": "secret-key-value",
  "name": "client-api-key",
  "description": "API key for client application",
  "permissions": [
    {
      "componentId": {
        "name": "payment-workflow",
        "revision": "1.0"
      },
      "permission": "EXECUTE"
    }
  ]
}

Features:

Secure storage with encryption
Permission-based access control
Component-level permissions
Automatic rotation support (via AWS)

Configuration Reference

AWS Core Configuration

hub:
  aws:
    # AWS credentials (uses default chain if not specified)
    accessKey: ${AWS_ACCESS_KEY_ID:}
    secretKey: ${AWS_SECRET_ACCESS_KEY:}

    # AWS region
    region: us-east-1  # Required

DynamoDB Configuration

hub:
  aws:
    dynamodb:
      # DynamoDB endpoint (for LocalStack or custom)
      endpoint: http://localhost:4566  # Optional, omit for production

      # Table name prefix
      prefix: zenoo-hub  # Required

      # Automatically create tables on startup
      createTables: true  # Default: true

      # Resource tags
      tags:
        Environment: production
        Team: platform
        CostCenter: engineering

      # Multi-region replication
      replicas:
        - us-west-2
        - eu-west-1

      # Retry strategy
      retryStrategy:
        requestTimeout: 500ms    # Request timeout
        maxRetries: 10           # Max retry attempts
        backoff: 100ms           # Backoff duration

Secrets Manager Configuration

hub:
  aws:
    secrets:
      # Secret name prefix
      prefix: zenoo-hub  # Required

      # Enable caching
      cache: true  # Default: true

      # Cache TTL
      cacheTtl: 300s  # Default: 5 minutes

      # Force delete without recovery window
      forceDelete: false  # Default: false (30-day recovery)

      # Resource tags
      tags:
        Environment: production
        DataClassification: HighlyRestricted

      # Multi-region replication
      replicas:
        - us-west-2
        - eu-west-1

CloudWatch Metrics Configuration

hub:
  aws:
    dynamodb:
      metrics:
        enabled: true           # Enable CloudWatch metrics
        namespace: ZenooHub     # CloudWatch namespace

Multi-Region Deployment

The AWS provider supports multi-region replication for both DynamoDB and Secrets Manager.

DynamoDB Global Tables

hub:
  aws:
    region: us-east-1  # Primary region
    dynamodb:
      prefix: zenoo-hub
      replicas:
        - us-west-2    # Replica in US West
        - eu-west-1    # Replica in Europe

Features:

Active-active replication
Automatic conflict resolution
Low-latency local reads
Cross-region failover

Requirements:

Same AWS account in all regions
DynamoDB streams enabled (automatic)
IAM permissions for global table creation

Secrets Manager Replication

hub:
  aws:
    secrets:
      prefix: zenoo-hub
      replicas:
        - us-west-2
        - eu-west-1

Features:

Automatic secret replication
Independent KMS keys per region
Automatic rotation support
Regional failover

IAM Permissions

Minimum Required Permissions

Create an IAM policy with the following permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:PutItem",
        "dynamodb:UpdateItem",
        "dynamodb:DeleteItem",
        "dynamodb:Query",
        "dynamodb:Scan",
        "dynamodb:BatchGetItem",
        "dynamodb:BatchWriteItem"
      ],
      "Resource": [
        "arn:aws:dynamodb:*:*:table/zenoo-hub-*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:CreateTable",
        "dynamodb:DescribeTable",
        "dynamodb:UpdateTable",
        "dynamodb:UpdateTimeToLive",
        "dynamodb:DescribeTimeToLive"
      ],
      "Resource": [
        "arn:aws:dynamodb:*:*:table/zenoo-hub-*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:RequestedRegion": ["us-east-1", "us-west-2"]
        }
      }
    },
    {
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue",
        "secretsmanager:DescribeSecret",
        "secretsmanager:ListSecretVersionIds"
      ],
      "Resource": [
        "arn:aws:secretsmanager:*:*:secret:zenoo-hub/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "secretsmanager:CreateSecret",
        "secretsmanager:UpdateSecret",
        "secretsmanager:DeleteSecret",
        "secretsmanager:TagResource"
      ],
      "Resource": [
        "arn:aws:secretsmanager:*:*:secret:zenoo-hub/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "cloudwatch:PutMetricData"
      ],
      "Resource": "*",
      "Condition": {
        "StringEquals": {
          "cloudwatch:namespace": "ZenooHub"
        }
      }
    }
  ]
}

Recommended Additional Permissions

For production deployments, also grant:

kms:Decrypt, kms:Encrypt - For KMS encryption
logs:CreateLogGroup, logs:CreateLogStream, logs:PutLogEvents - For CloudWatch Logs
sts:AssumeRole - For cross-account access (if needed)

Performance Tuning

DynamoDB Capacity

Choose between on-demand and provisioned capacity: On-Demand (Recommended for variable workloads):

hub:
  aws:
    dynamodb:
      billingMode: PAY_PER_REQUEST  # Default

Provisioned (For predictable workloads):

hub:
  aws:
    dynamodb:
      billingMode: PROVISIONED
      throughput:
        read: 100   # Read capacity units
        write: 50   # Write capacity units

Connection Pooling

The AWS provider uses the AWS SDK’s built-in connection pooling. For high-throughput scenarios:

# Spring Boot application properties
spring:
  threads:
    virtual:
      enabled: true  # Use virtual threads (Java 21+)

Caching Strategy

Enable Secrets Manager caching to reduce API calls:

hub:
  aws:
    secrets:
      cache: true
      cacheTtl: 300s  # Cache secrets for 5 minutes

Trade-offs:

Reduces Secrets Manager API calls (cost savings)
Improves latency
Secrets changes take up to cacheTtl to propagate

Security Best Practices

1. Use IAM Roles Instead of Access Keys

Recommended:

# On EC2/ECS/EKS - use instance/pod IAM role
hub:
  aws:
    region: us-east-1
    # No credentials needed

Avoid:

# Don't hardcode credentials
hub:
  aws:
    accessKey: AKIAIOSFODNN7EXAMPLE
    secretKey: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

2. Enable Encryption at Rest

DynamoDB tables are encrypted by default. For additional security, use customer-managed KMS keys:

hub:
  aws:
    dynamodb:
      kmsKeyId: arn:aws:kms:us-east-1:123456789012:key/12345678-1234-1234-1234-123456789012

3. Use VPC Endpoints

For enhanced security, access DynamoDB and Secrets Manager via VPC endpoints:

Eliminates internet gateway traffic
Reduces data transfer costs
Improved security posture

4. Enable CloudTrail Audit Logging

Monitor all API calls for compliance:

hub:
  aws:
    dynamodb:
      audit:
        enabled: true
        cloudTrailArn: arn:aws:cloudtrail:us-east-1:123456789012:trail/my-trail

5. Tag Resources Appropriately

Use tags for cost allocation and access control:

hub:
  aws:
    dynamodb:
      tags:
        Environment: production
        CostCenter: engineering
        DataClassification: HighlyRestricted
        Compliance: PCI-DSS

Monitoring and Metrics

CloudWatch Metrics

The AWS provider publishes the following metrics: DynamoDB Metrics:

DynamoDB.Operation.Latency - Operation latency by operation type
DynamoDB.Operation.Errors - Error count by operation type
DynamoDB.ThrottledRequests - Number of throttled requests

Secrets Manager Metrics:

SecretsManager.CacheHitRate - Cache hit percentage
SecretsManager.AccessCount - Secret access count

Custom Dimensions:

TableName - DynamoDB table name
Operation - Operation type (GetItem, PutItem, etc.)
SecretName - Secret name

CloudWatch Alarms

Set up alarms for critical metrics:

# Example alarm configuration
alarms:
  - name: HighDynamoDBLatency
    metric: DynamoDB.Operation.Latency
    threshold: 1000  # ms
    evaluation_periods: 3

  - name: DynamoDBThrottling
    metric: DynamoDB.ThrottledRequests
    threshold: 10
    evaluation_periods: 2

Troubleshooting

Tables Not Created

Symptom: Hub starts but tables don’t exist Solutions:

Check createTables: true in configuration
Verify IAM permissions include dynamodb:CreateTable
Check CloudWatch Logs for errors
Verify region is correct

Secret Not Found

Symptom: SecretNotFoundException when retrieving configuration Solutions:

Verify secret name format: {prefix}/component-config/{key}/{version}
Check IAM permissions include secretsmanager:GetSecretValue
Ensure secret exists in the correct region
Check secret version/stage labels

DynamoDB Throttling

Symptom: ProvisionedThroughputExceededException Solutions:

Switch to on-demand billing mode
Increase provisioned capacity
Implement exponential backoff (already configured)
Review access patterns for optimization

High Latency

Symptom: Slow component retrieval Solutions:

Enable DynamoDB DAX (caching layer)
Use VPC endpoints to reduce network latency
Choose regions closer to your deployment
Review DynamoDB table design

Connectivity Issues

Symptom: Cannot connect to DynamoDB/Secrets Manager Solutions:

Check VPC security groups and NACLs
Verify VPC endpoints are configured correctly
Ensure DNS resolution is working
Check IAM permissions

Testing with LocalStack

For local development and testing, use LocalStack to emulate AWS services:

1. Start LocalStack

docker run -d \
  -p 4566:4566 \
  -e SERVICES=dynamodb,secretsmanager \
  localstack/localstack

2. Configure Hub for LocalStack

hub:
  aws:
    region: us-east-1
    accessKey: test
    secretKey: test
    dynamodb:
      endpoint: http://localhost:4566
      prefix: test-hub
      createTables: true
    secrets:
      endpoint: http://localhost:4566
      prefix: test-hub

3. Run Tests

./gradlew :backend-integration-tests:test

Cost Optimization

DynamoDB Costs

On-Demand Pricing:

Pay per request (recommended for variable workloads)
No capacity planning required
Cost scales with usage

Provisioned Pricing:

Reserve capacity (cheaper for predictable workloads)
Auto-scaling available
Pay for reserved capacity even if unused

Optimization Tips:

Use on-demand for development/staging
Use provisioned with auto-scaling for production
Archive old components to S3 with lifecycle policies
Use TTL for automatic data expiration (sharables)

Secrets Manager Costs

Charged per secret per month + API calls

Use caching to reduce API calls:

hub:
  aws:
    secrets:
      cache: true
      cacheTtl: 600s  # 10-minute cache

Consolidate secrets where appropriate

Data Transfer Costs

Use VPC endpoints to eliminate internet gateway charges
Deploy in same region as other services
Use multi-region replication judiciously

Migration from Old Configuration

If upgrading from an older Hub version, your existing configuration continues to work: Old Configuration (Still Supported):

hub:
  aws:
    dynamodb:
      prefix: my-hub
    secrets:
      prefix: my-hub

New Configuration (Recommended):

hub:
  cloud:
    provider:
      type: aws  # Explicit provider selection
  aws:
    dynamodb:
      prefix: my-hub
    secrets:
      prefix: my-hub

Read the complete migration guide

​AWS Provider

​Prerequisites

​AWS Account Requirements

​Required AWS Services

​Quick Start

​1. Basic Configuration

​2. AWS Credentials

​3. Start the Hub

​DynamoDB Tables

​Components Table

​API Key Lookup Table

​Sharables Table

​AWS Secrets Manager

​Component Configuration Secrets

​API Key Secrets

​Configuration Reference

​AWS Core Configuration

​DynamoDB Configuration

​Secrets Manager Configuration

​CloudWatch Metrics Configuration

​Multi-Region Deployment

​DynamoDB Global Tables

​Secrets Manager Replication

​IAM Permissions

​Minimum Required Permissions

​Recommended Additional Permissions

​Performance Tuning

​DynamoDB Capacity

​Connection Pooling

​Caching Strategy

​Security Best Practices

​1. Use IAM Roles Instead of Access Keys

​2. Enable Encryption at Rest

​3. Use VPC Endpoints

​4. Enable CloudTrail Audit Logging

​5. Tag Resources Appropriately

​Monitoring and Metrics

​CloudWatch Metrics

​CloudWatch Alarms

​Troubleshooting

​Tables Not Created

​Secret Not Found

​DynamoDB Throttling

​High Latency

​Connectivity Issues

​Testing with LocalStack

​1. Start LocalStack

​2. Configure Hub for LocalStack

​3. Run Tests

​Cost Optimization

​DynamoDB Costs

​Secrets Manager Costs

​Data Transfer Costs

​Migration from Old Configuration

​See Also

AWS Provider

Prerequisites

AWS Account Requirements

Required AWS Services

Quick Start

1. Basic Configuration

2. AWS Credentials

3. Start the Hub

DynamoDB Tables

Components Table

API Key Lookup Table

Sharables Table

AWS Secrets Manager

Component Configuration Secrets

API Key Secrets

Configuration Reference

AWS Core Configuration

DynamoDB Configuration

Secrets Manager Configuration

CloudWatch Metrics Configuration

Multi-Region Deployment

DynamoDB Global Tables

Secrets Manager Replication

IAM Permissions

Minimum Required Permissions

Recommended Additional Permissions

Performance Tuning

DynamoDB Capacity

Connection Pooling

Caching Strategy

Security Best Practices

1. Use IAM Roles Instead of Access Keys

2. Enable Encryption at Rest

3. Use VPC Endpoints

4. Enable CloudTrail Audit Logging

5. Tag Resources Appropriately

Monitoring and Metrics

CloudWatch Metrics

CloudWatch Alarms

Troubleshooting

Tables Not Created

Secret Not Found

DynamoDB Throttling

High Latency

Connectivity Issues

Testing with LocalStack

1. Start LocalStack

2. Configure Hub for LocalStack

3. Run Tests

Cost Optimization

DynamoDB Costs

Secrets Manager Costs

Data Transfer Costs

Migration from Old Configuration

See Also