> ## Documentation Index
> Fetch the complete documentation index at: https://platform.docs.zenoo.com/llms.txt
> Use this file to discover all available pages before exploring further.

# AWS Provider

> Complete guide to deploying Zenoo Hub on AWS using DynamoDB, Secrets Manager, and CloudWatch

# AWS Provider

The AWS provider is a production-ready implementation of the cloud provider abstraction, using AWS DynamoDB for storage, Secrets Manager for configuration and secrets, and CloudWatch for metrics.

## Prerequisites

### AWS Account Requirements

* Active AWS account with appropriate permissions
* IAM user or role with necessary permissions (see [IAM Permissions](#iam-permissions))
* Access to the AWS regions where you want to deploy

### Required AWS Services

* **DynamoDB** - Component and API key storage
* **AWS Secrets Manager** - Configuration and API key secrets
* **CloudWatch** (optional) - Metrics publishing

## Quick Start

### 1. Basic Configuration

Add to your `application.yml`:

```yaml theme={null}
hub:
  aws:
    region: us-east-1
    dynamodb:
      prefix: zenoo-hub
      createTables: true
    secrets:
      prefix: zenoo-hub
```

### 2. AWS Credentials

The Hub uses the AWS SDK default credential chain. You can provide credentials via:

**Environment variables:**

```bash theme={null}
export AWS_ACCESS_KEY_ID=your-access-key
export AWS_SECRET_ACCESS_KEY=your-secret-key
export AWS_REGION=us-east-1
```

**IAM Role (recommended for EC2/ECS/EKS):**

```yaml theme={null}
# No explicit credentials needed - uses instance/pod role
hub:
  aws:
    region: us-east-1
```

**Configuration file (not recommended for production):**

```yaml theme={null}
hub:
  aws:
    accessKey: ${AWS_ACCESS_KEY_ID}
    secretKey: ${AWS_SECRET_ACCESS_KEY}
    region: us-east-1
```

### 3. Start the Hub

The Hub will automatically:

* Create required DynamoDB tables (if `createTables: true`)
* Set up TTL for the sharables table
* Configure multi-region replication (if configured)

## DynamoDB Tables

The AWS provider creates and manages three DynamoDB tables:

### Components Table

**Table Name:** `{prefix}-components`

Stores Hub component definitions with versioning support.

**Schema:**

* **Partition Key:** `componentName` (String) - Component name
* **Sort Key:** `revision` (Number) - Component revision
* **Attributes:**
  * `definition` (String) - Component DSL definition
  * `metadata` (Map) - Component metadata
  * `dependencies` (List) - Component dependencies
  * `connectors` (List) - Required connectors
  * `version` (Number) - Optimistic locking version

**Features:**

* Point-in-time recovery enabled
* Versioned revisions for rollback capability
* Optimistic locking for concurrent updates
* Global table support for multi-region

**Configuration:**

```yaml theme={null}
hub:
  aws:
    dynamodb:
      prefix: my-hub  # Table name: my-hub-components
```

### API Key Lookup Table

**Table Name:** `{prefix}-api-key-lookup`

Maintains a lookup index for API keys associated with components.

**Schema:**

* **Partition Key:** `componentName` (String) - Component name
* **Attributes:**
  * `secrets` (`Set<String>`) - API key secret names
  * `exposed` (Boolean) - Whether component is exposed
  * `exposedFunctions` (`List<String>`) - Exposed function names
  * `version` (Number) - Optimistic locking version

**Use Cases:**

* Fast lookup of API keys for a component
* Determining if a component is publicly exposed
* Managing component access control

### Sharables Table

**Table Name:** `{prefix}-sharables`

Stores temporary sharable tokens (e.g., for magic links, temporary access).

**Schema:**

* **Partition Key:** `token` (String) - Unique token
* **Attributes:**
  * `payload` (String) - Sharable payload (JSON)
  * `expiresAt` (Number) - Expiration timestamp
  * `expired` (Boolean) - Expiration flag
  * `ttl` (Number) - DynamoDB TTL attribute

**Features:**

* Automatic cleanup via DynamoDB TTL
* High-performance token retrieval
* Configurable expiration

**Configuration:**

```yaml theme={null}
hub:
  cloud:
    provider:
      sharable:
        defaultTtl: 24h  # Default TTL for sharables
```

## AWS Secrets Manager

The AWS provider stores sensitive configuration and API keys in AWS Secrets Manager.

### Component Configuration Secrets

**Naming Convention:** `{prefix}/component-config/{configId.key}/{configId.version}`

**Example:** `zenoo-hub/component-config/payment-api/1.0`

**Content Structure:**

```json theme={null}
{
  "api": {
    "endpoint": "https://api.example.com",
    "timeout": 30000
  },
  "credentials": {
    "username": "api-user",
    "password": "secret-password"
  }
}
```

**Features:**

* Automatic versioning with AWSCURRENT and AWSPREVIOUS stage labels
* Multi-region replication
* Encryption at rest with AWS KMS
* Audit trail via CloudTrail

### API Key Secrets

**Naming Convention:** `{prefix}/api-keys/{keyName}`

**Example:** `zenoo-hub/api-keys/client-api-key`

**Content Structure:**

```json theme={null}
{
  "apiKey": "secret-key-value",
  "name": "client-api-key",
  "description": "API key for client application",
  "permissions": [
    {
      "componentId": {
        "name": "payment-workflow",
        "revision": "1.0"
      },
      "permission": "EXECUTE"
    }
  ]
}
```

**Features:**

* Secure storage with encryption
* Permission-based access control
* Component-level permissions
* Automatic rotation support (via AWS)

## Configuration Reference

### AWS Core Configuration

```yaml theme={null}
hub:
  aws:
    # AWS credentials (uses default chain if not specified)
    accessKey: ${AWS_ACCESS_KEY_ID:}
    secretKey: ${AWS_SECRET_ACCESS_KEY:}

    # AWS region
    region: us-east-1  # Required
```

### DynamoDB Configuration

```yaml theme={null}
hub:
  aws:
    dynamodb:
      # DynamoDB endpoint (for LocalStack or custom)
      endpoint: http://localhost:4566  # Optional, omit for production

      # Table name prefix
      prefix: zenoo-hub  # Required

      # Automatically create tables on startup
      createTables: true  # Default: true

      # Resource tags
      tags:
        Environment: production
        Team: platform
        CostCenter: engineering

      # Multi-region replication
      replicas:
        - us-west-2
        - eu-west-1

      # Retry strategy
      retryStrategy:
        requestTimeout: 500ms    # Request timeout
        maxRetries: 10           # Max retry attempts
        backoff: 100ms           # Backoff duration
```

### Secrets Manager Configuration

```yaml theme={null}
hub:
  aws:
    secrets:
      # Secret name prefix
      prefix: zenoo-hub  # Required

      # Enable caching
      cache: true  # Default: true

      # Cache TTL
      cacheTtl: 300s  # Default: 5 minutes

      # Force delete without recovery window
      forceDelete: false  # Default: false (30-day recovery)

      # Resource tags
      tags:
        Environment: production
        DataClassification: HighlyRestricted

      # Multi-region replication
      replicas:
        - us-west-2
        - eu-west-1
```

### CloudWatch Metrics Configuration

```yaml theme={null}
hub:
  aws:
    dynamodb:
      metrics:
        enabled: true           # Enable CloudWatch metrics
        namespace: ZenooHub     # CloudWatch namespace
```

## Multi-Region Deployment

The AWS provider supports multi-region replication for both DynamoDB and Secrets Manager.

### DynamoDB Global Tables

```yaml theme={null}
hub:
  aws:
    region: us-east-1  # Primary region
    dynamodb:
      prefix: zenoo-hub
      replicas:
        - us-west-2    # Replica in US West
        - eu-west-1    # Replica in Europe
```

**Features:**

* Active-active replication
* Automatic conflict resolution
* Low-latency local reads
* Cross-region failover

**Requirements:**

* Same AWS account in all regions
* DynamoDB streams enabled (automatic)
* IAM permissions for global table creation

### Secrets Manager Replication

```yaml theme={null}
hub:
  aws:
    secrets:
      prefix: zenoo-hub
      replicas:
        - us-west-2
        - eu-west-1
```

**Features:**

* Automatic secret replication
* Independent KMS keys per region
* Automatic rotation support
* Regional failover

## IAM Permissions

### Minimum Required Permissions

Create an IAM policy with the following permissions:

```json theme={null}
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:PutItem",
        "dynamodb:UpdateItem",
        "dynamodb:DeleteItem",
        "dynamodb:Query",
        "dynamodb:Scan",
        "dynamodb:BatchGetItem",
        "dynamodb:BatchWriteItem"
      ],
      "Resource": [
        "arn:aws:dynamodb:*:*:table/zenoo-hub-*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:CreateTable",
        "dynamodb:DescribeTable",
        "dynamodb:UpdateTable",
        "dynamodb:UpdateTimeToLive",
        "dynamodb:DescribeTimeToLive"
      ],
      "Resource": [
        "arn:aws:dynamodb:*:*:table/zenoo-hub-*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:RequestedRegion": ["us-east-1", "us-west-2"]
        }
      }
    },
    {
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue",
        "secretsmanager:DescribeSecret",
        "secretsmanager:ListSecretVersionIds"
      ],
      "Resource": [
        "arn:aws:secretsmanager:*:*:secret:zenoo-hub/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "secretsmanager:CreateSecret",
        "secretsmanager:UpdateSecret",
        "secretsmanager:DeleteSecret",
        "secretsmanager:TagResource"
      ],
      "Resource": [
        "arn:aws:secretsmanager:*:*:secret:zenoo-hub/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "cloudwatch:PutMetricData"
      ],
      "Resource": "*",
      "Condition": {
        "StringEquals": {
          "cloudwatch:namespace": "ZenooHub"
        }
      }
    }
  ]
}
```

### Recommended Additional Permissions

For production deployments, also grant:

* `kms:Decrypt`, `kms:Encrypt` - For KMS encryption
* `logs:CreateLogGroup`, `logs:CreateLogStream`, `logs:PutLogEvents` - For CloudWatch Logs
* `sts:AssumeRole` - For cross-account access (if needed)

## Performance Tuning

### DynamoDB Capacity

Choose between on-demand and provisioned capacity:

**On-Demand (Recommended for variable workloads):**

```yaml theme={null}
hub:
  aws:
    dynamodb:
      billingMode: PAY_PER_REQUEST  # Default
```

**Provisioned (For predictable workloads):**

```yaml theme={null}
hub:
  aws:
    dynamodb:
      billingMode: PROVISIONED
      throughput:
        read: 100   # Read capacity units
        write: 50   # Write capacity units
```

### Connection Pooling

The AWS provider uses the AWS SDK's built-in connection pooling. For high-throughput scenarios:

```yaml theme={null}
# Spring Boot application properties
spring:
  threads:
    virtual:
      enabled: true  # Use virtual threads (Java 21+)
```

### Caching Strategy

Enable Secrets Manager caching to reduce API calls:

```yaml theme={null}
hub:
  aws:
    secrets:
      cache: true
      cacheTtl: 300s  # Cache secrets for 5 minutes
```

**Trade-offs:**

* Reduces Secrets Manager API calls (cost savings)
* Improves latency
* Secrets changes take up to cacheTtl to propagate

## Security Best Practices

### 1. Use IAM Roles Instead of Access Keys

**Recommended:**

```yaml theme={null}
# On EC2/ECS/EKS - use instance/pod IAM role
hub:
  aws:
    region: us-east-1
    # No credentials needed
```

**Avoid:**

```yaml theme={null}
# Don't hardcode credentials
hub:
  aws:
    accessKey: AKIAIOSFODNN7EXAMPLE
    secretKey: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
```

### 2. Enable Encryption at Rest

DynamoDB tables are encrypted by default. For additional security, use customer-managed KMS keys:

```yaml theme={null}
hub:
  aws:
    dynamodb:
      kmsKeyId: arn:aws:kms:us-east-1:123456789012:key/12345678-1234-1234-1234-123456789012
```

### 3. Use VPC Endpoints

For enhanced security, access DynamoDB and Secrets Manager via VPC endpoints:

* Eliminates internet gateway traffic
* Reduces data transfer costs
* Improved security posture

### 4. Enable CloudTrail Audit Logging

Monitor all API calls for compliance:

```yaml theme={null}
hub:
  aws:
    dynamodb:
      audit:
        enabled: true
        cloudTrailArn: arn:aws:cloudtrail:us-east-1:123456789012:trail/my-trail
```

### 5. Tag Resources Appropriately

Use tags for cost allocation and access control:

```yaml theme={null}
hub:
  aws:
    dynamodb:
      tags:
        Environment: production
        CostCenter: engineering
        DataClassification: HighlyRestricted
        Compliance: PCI-DSS
```

## Monitoring and Metrics

### CloudWatch Metrics

The AWS provider publishes the following metrics:

**DynamoDB Metrics:**

* `DynamoDB.Operation.Latency` - Operation latency by operation type
* `DynamoDB.Operation.Errors` - Error count by operation type
* `DynamoDB.ThrottledRequests` - Number of throttled requests

**Secrets Manager Metrics:**

* `SecretsManager.CacheHitRate` - Cache hit percentage
* `SecretsManager.AccessCount` - Secret access count

**Custom Dimensions:**

* `TableName` - DynamoDB table name
* `Operation` - Operation type (GetItem, PutItem, etc.)
* `SecretName` - Secret name

### CloudWatch Alarms

Set up alarms for critical metrics:

```yaml theme={null}
# Example alarm configuration
alarms:
  - name: HighDynamoDBLatency
    metric: DynamoDB.Operation.Latency
    threshold: 1000  # ms
    evaluation_periods: 3

  - name: DynamoDBThrottling
    metric: DynamoDB.ThrottledRequests
    threshold: 10
    evaluation_periods: 2
```

## Troubleshooting

### Tables Not Created

**Symptom:** Hub starts but tables don't exist

**Solutions:**

1. Check `createTables: true` in configuration
2. Verify IAM permissions include `dynamodb:CreateTable`
3. Check CloudWatch Logs for errors
4. Verify region is correct

### Secret Not Found

**Symptom:** `SecretNotFoundException` when retrieving configuration

**Solutions:**

1. Verify secret name format: `{prefix}/component-config/{key}/{version}`
2. Check IAM permissions include `secretsmanager:GetSecretValue`
3. Ensure secret exists in the correct region
4. Check secret version/stage labels

### DynamoDB Throttling

**Symptom:** `ProvisionedThroughputExceededException`

**Solutions:**

1. Switch to on-demand billing mode
2. Increase provisioned capacity
3. Implement exponential backoff (already configured)
4. Review access patterns for optimization

### High Latency

**Symptom:** Slow component retrieval

**Solutions:**

1. Enable DynamoDB DAX (caching layer)
2. Use VPC endpoints to reduce network latency
3. Choose regions closer to your deployment
4. Review DynamoDB table design

### Connectivity Issues

**Symptom:** Cannot connect to DynamoDB/Secrets Manager

**Solutions:**

1. Check VPC security groups and NACLs
2. Verify VPC endpoints are configured correctly
3. Ensure DNS resolution is working
4. Check IAM permissions

## Testing with LocalStack

For local development and testing, use LocalStack to emulate AWS services:

### 1. Start LocalStack

```bash theme={null}
docker run -d \
  -p 4566:4566 \
  -e SERVICES=dynamodb,secretsmanager \
  localstack/localstack
```

### 2. Configure Hub for LocalStack

```yaml theme={null}
hub:
  aws:
    region: us-east-1
    accessKey: test
    secretKey: test
    dynamodb:
      endpoint: http://localhost:4566
      prefix: test-hub
      createTables: true
    secrets:
      endpoint: http://localhost:4566
      prefix: test-hub
```

### 3. Run Tests

```bash theme={null}
./gradlew :backend-integration-tests:test
```

## Cost Optimization

### DynamoDB Costs

**On-Demand Pricing:**

* Pay per request (recommended for variable workloads)
* No capacity planning required
* Cost scales with usage

**Provisioned Pricing:**

* Reserve capacity (cheaper for predictable workloads)
* Auto-scaling available
* Pay for reserved capacity even if unused

**Optimization Tips:**

* Use on-demand for development/staging
* Use provisioned with auto-scaling for production
* Archive old components to S3 with lifecycle policies
* Use TTL for automatic data expiration (sharables)

### Secrets Manager Costs

* Charged per secret per month + API calls
* Use caching to reduce API calls:
  ```yaml theme={null}
  hub:
    aws:
      secrets:
        cache: true
        cacheTtl: 600s  # 10-minute cache
  ```
* Consolidate secrets where appropriate

### Data Transfer Costs

* Use VPC endpoints to eliminate internet gateway charges
* Deploy in same region as other services
* Use multi-region replication judiciously

## Migration from Old Configuration

If upgrading from an older Hub version, your existing configuration continues to work:

**Old Configuration (Still Supported):**

```yaml theme={null}
hub:
  aws:
    dynamodb:
      prefix: my-hub
    secrets:
      prefix: my-hub
```

**New Configuration (Recommended):**

```yaml theme={null}
hub:
  cloud:
    provider:
      type: aws  # Explicit provider selection
  aws:
    dynamodb:
      prefix: my-hub
    secrets:
      prefix: my-hub
```

[Read the complete migration guide](/technical-specification/hub-backend/cloud-providers/migration-guide)

## See Also

* [Cloud Provider Architecture](/technical-specification/hub-backend/cloud-providers/architecture)
* [Configuration Reference](/technical-specification/hub-backend/cloud-providers/configuration-reference)
* [Local Development Guide](/technical-specification/hub-backend/cloud-providers/local-development)
* [Migration Guide](/technical-specification/hub-backend/cloud-providers/migration-guide)
* [AWS MSK Configuration](/technical-specification/hub-backend/aws-msk)
* [AWS Parameter Store](/technical-specification/hub-backend/parameter-store)