Local Provider

The local provider is a lightweight, in-memory implementation of the cloud provider abstraction designed for local development and testing. It requires zero external dependencies and provides instant startup with no configuration overhead.

Overview

The local provider stores all data in memory using thread-safe ConcurrentHashMap structures. All data is lost when the application stops, making it ideal for:

Local Development - Quick iteration without AWS setup
Integration Testing - Fast, isolated test execution
CI/CD Pipelines - No external service dependencies
Prototyping - Rapid experimentation
Learning - Simple to understand and debug

Not suitable for:

Production deployments
Data persistence across restarts
Multi-instance deployments (no shared state)
Large datasets (memory-bound)

Quick Start

1. Add Dependency

Add to your build.gradle:

dependencies {
    implementation project(':backend-spring-boot-starter')
    implementation project(':cloud-provider-local')
}

2. Configure Provider

Add to your application.yml:

hub:
  cloud:
    provider:
      type: local  # Activate local provider

That’s it! No credentials, no external services, no additional setup required.

3. Start the Hub

The Hub will log:

Local Cloud Provider auto-configuration enabled
Using in-memory storage for all cloud provider operations
Note: All data will be lost on application restart

4. Optional Configuration

Configure cleanup and logging (defaults work for most cases):

hub:
  local:
    cleanupEnabled: true       # Enable automatic cleanup of expired sharables
    cleanupInterval: 5m        # Cleanup interval
    verboseLogging: false      # Detailed logging for debugging

Architecture

Storage Design

The local provider uses Java’s ConcurrentHashMap for thread-safe in-memory storage:

LocalComponentStore:
  components: ConcurrentHashMap<String, StringComponent>
    Key: "componentName:revision"
    Value: StringComponent (definition, metadata, dependencies)

  latestRevisions: ConcurrentHashMap<String, String>
    Key: "componentName"
    Value: "latestRevision"

LocalApiKeyStore:
  apiKeyRecords: ConcurrentHashMap<String, ApiKeyRecord>
    Key: "componentName"
    Value: ApiKeyRecord (secrets, exposed, exposedFunctions)

LocalApiKeySecretStorage:
  secrets: ConcurrentHashMap<String, ApiKeySecret>
    Key: "secretName"
    Value: ApiKeySecret (encrypted key, permissions)

LocalSharableStore:
  sharables: ConcurrentHashMap<String, SharableRecord>
    Key: "token"
    Value: SharableRecord (payload, expiration, reusable)

LocalComponentConfigStorage:
  configs: ConcurrentHashMap<String, String>
    Key: "configId.key:configId.version"
    Value: JSON configuration string

LocalComponentTokenService:
  (Coordinates ApiKeyStore and ApiKeySecretStorage)

Thread Safety

All storage implementations use ConcurrentHashMap for thread-safe concurrent access:

No external locking required
Safe for multi-threaded request handling
Atomic operations for consistency
Read-write locks for complex operations (API key records)

Reactive Support

All operations return Project Reactor Mono<T> for consistency with the cloud provider abstraction:

public Mono<StringComponent> get(ComponentId id) {
    return Mono.fromCallable(() -> {
        // Lookup in ConcurrentHashMap
        return components.get(toKey(id));
    });
}

Storage Components

Component Store

Purpose: Store Hub component definitions with versioning Features:

CRUD operations for components
Revision-based versioning
LATEST pseudo-revision support
Metadata and dependency tracking

Key Format: componentName:revision Example:

ComponentId id = ComponentId.builder()
    .name("payment-workflow")
    .revision("1.0")
    .build();

componentStore.store(StringComponent.builder()
    .id(id)
    .definition("workflow { ... }")
    .build());

// Retrieve by exact revision
Mono<StringComponent> component = componentStore.get(id);

// Retrieve latest
ComponentId latestId = id.toBuilder().revision("LATEST").build();
Mono<StringComponent> latest = componentStore.get(latestId);

API Key Store

Purpose: Manage API key associations with components Features:

Associate API keys with components
Track exposed components and functions
Fast lookup by component name
Bidirectional sync with ApiKeySecretStorage

Example:

apiKeyStore.associate(componentId, "client-api-key", true, Set.of("processPayment"));

// Lookup by component
Mono<Set<String>> secrets = apiKeyStore.getSecrets(componentId);

// Check if exposed
Mono<Boolean> isExposed = apiKeyStore.isExposed(componentId);

API Key Secret Storage

Purpose: Store encrypted API key secrets Features:

Secure secret storage
Permission-based access control
Component-level permissions
Bidirectional sync with ApiKeyStore

Example:

ApiKeySecret secret = ApiKeySecret.builder()
    .name("client-api-key")
    .apiKey("secret-key-value")
    .permissions(List.of(
        Permission.builder()
            .componentId(componentId)
            .permission(PermissionType.EXECUTE)
            .build()
    ))
    .build();

apiKeySecretStorage.create(secret);

// Retrieve by component
Mono<ComponentApiKeyLookup> lookup = apiKeySecretStorage.getByComponentId(componentId);

Sharable Store

Purpose: Store temporary tokens (magic links, temporary access) Features:

TTL-based expiration
Lazy expiration (checked on access)
Optional scheduled cleanup
Reusable and one-time tokens

Expiration Strategy:

Lazy Expiration: Checked on get() - expired records are deleted on access
Scheduled Cleanup: Background task removes expired tokens (configurable interval)

Example:

SharableRecord sharable = SharableRecord.builder()
    .token("magic-link-token")
    .payload("{\"userId\":\"123\"}")
    .expiration(Instant.now().plus(Duration.ofHours(24)))
    .reusable(false)
    .build();

sharableStore.store(sharable);

// Retrieve (automatically checks expiration)
Mono<SharableRecord> record = sharableStore.get("magic-link-token");

// Mark as expired
sharableStore.expire("magic-link-token");

Component Config Storage

Purpose: Store component configuration with versioning Features:

Configuration versioning
LATEST version support
JSON storage format
In-memory caching

Key Format: configId.key:configId.version Example:

ConfigId configId = ConfigId.builder()
    .key("payment-api")
    .version("1.0")
    .build();

String config = "{\"endpoint\":\"https://api.example.com\"}";

configStorage.store(configId, config);

// Retrieve by version
Mono<String> config = configStorage.get(configId);

// Retrieve latest
ConfigId latestId = configId.toBuilder().version("LATEST").build();
Mono<String> latestConfig = configStorage.get(latestId);

Component Token Service

Purpose: Validate component access tokens Features:

Token-based authentication
Component permission validation
Coordinates ApiKeyStore and ApiKeySecretStorage

Example:

ComponentTokenServiceResult result = tokenService.validate("api-key-token");

if (result.isValid()) {
    ComponentId componentId = result.getComponentId();
    // Process request
}

Configuration Reference

Core Configuration

hub:
  cloud:
    provider:
      type: local  # Required: Activate local provider

Local Provider Configuration

hub:
  local:
    # Enable automatic cleanup of expired sharables
    # Default: true
    cleanupEnabled: true

    # Cleanup interval (supports: ms, s, m, h, d)
    # Default: 5m
    cleanupInterval: 5m

    # Enable verbose logging for debugging
    # Default: false
    verboseLogging: false

    # Simulated delay for testing (optional)
    # Default: 0ms (no delay)
    simulatedDelay: 10ms

Sharable Expiration Configuration

hub:
  cloud:
    provider:
      sharable:
        # Default TTL for sharables
        # Default: 24h
        defaultTtl: 24h

Complete Example Configuration

application-local.yml:

spring:
  profiles:
    include:
      - uploader
      - redis-file-storage
      - create-topics

hub:
  cloud:
    provider:
      type: local
      sharable:
        defaultTtl: 24h

  local:
    cleanupEnabled: true
    cleanupInterval: 5m
    verboseLogging: false

# Other Hub configuration...
kafka:
  bootstrap-servers: localhost:9092
redis:
  host: localhost
  port: 6379

Usage Scenarios

Scenario 1: Pure Local Development

Use Case: Developing without any external services Configuration:

hub:
  cloud:
    provider:
      type: local

Benefits:

Instant startup
No AWS credentials needed
No external service dependencies
Easy to reset (just restart)
Zero cost

Limitations:

Data lost on restart
No persistence
Single-instance only

Scenario 2: Integration Testing

Use Case: Fast, isolated integration tests Configuration:

# backend-integration-tests/src/main/resources/application.yml
hub:
  cloud:
    provider:
      type: local

  local:
    cleanupEnabled: true
    cleanupInterval: 1m  # Faster cleanup for tests

Benefits:

Fast test execution
No test data pollution
Deterministic behavior
No external service flakiness
Parallel test execution

Best Practices:

Use @DirtiesContext to reset state between tests
Clear stores manually if needed: localComponentStore.clear()
Use fast cleanup intervals

Scenario 3: CI/CD Pipeline

Use Case: Automated testing in CI/CD Configuration:

hub:
  cloud:
    provider:
      type: local

Benefits:

No secrets management in CI
Fast pipeline execution
No external dependencies
Consistent behavior
Cost-free

Scenario 4: Learning and Prototyping

Use Case: Learning Hub DSL and component development Configuration:

hub:
  cloud:
    provider:
      type: local

  local:
    verboseLogging: true  # See what's happening

Benefits:

Simple to understand
Easy to debug
No setup overhead
Experiment freely

Switching Between Providers

Local -> AWS

To switch from local to AWS provider: Before (Local):

hub:
  cloud:
    provider:
      type: local

After (AWS):

hub:
  cloud:
    provider:
      type: aws  # Or omit (AWS is default)

  aws:
    region: us-east-1
    dynamodb:
      prefix: my-hub
      createTables: true
    secrets:
      prefix: my-hub

Note: All data in local storage will be lost. Components and configuration must be recreated in AWS.

Using Profiles for Easy Switching

application-local.yml:

hub:
  cloud:
    provider:
      type: local

application-aws.yml:

hub:
  cloud:
    provider:
      type: aws
  aws:
    region: us-east-1
    dynamodb:
      prefix: my-hub

Activate:

# Local development
./gradlew bootRun --args='--spring.profiles.active=local'

# AWS deployment
./gradlew bootRun --args='--spring.profiles.active=aws'

Using Environment Variables

# Local
export HUB_CLOUD_PROVIDER_TYPE=local
./gradlew bootRun

# AWS
export HUB_CLOUD_PROVIDER_TYPE=aws
export AWS_REGION=us-east-1
./gradlew bootRun

Local vs LocalStack vs AWS

Feature	Local Provider	LocalStack	AWS
Setup	Zero config	Docker setup	AWS account + IAM
Startup	Instant	~10 seconds	Instant (tables exist)
Dependencies	None	Docker, LocalStack	AWS credentials
Persistence	None	Optional	Full persistence
Cost	Free	Free	Pay per use
AWS Accuracy	N/A	~90%	100%
Multi-instance	No	Yes (with config)	Yes
Use Case	Quick dev, testing	AWS behavior testing	Production

When to Use Local Provider

Quick prototyping and experimentation
Unit and integration testing
CI/CD pipelines (no external deps)
Learning Hub DSL
No need for persistence

When to Use LocalStack

Testing AWS-specific features (DynamoDB queries, TTL)
Validating AWS configuration
Multi-region testing
Testing with AWS SDK clients
Need persistence between restarts

When to Use AWS

Production deployments
Staging environments
Need for data persistence
Multi-instance deployments
Compliance requirements

Features

Thread-Safe Concurrent Access

All storage implementations use ConcurrentHashMap for safe concurrent access:

Multiple threads can read/write simultaneously
No external synchronization required
Atomic operations guarantee consistency

Reactive Programming Model

All operations return Mono<T> for consistency:

Integrates with Spring WebFlux
Non-blocking I/O support
Composable async operations

Example:

componentStore.get(componentId)
    .flatMap(component -> {
        // Process component
        return processComponent(component);
    })
    .subscribe();

Automatic Cleanup

Expired sharables are automatically removed:

Lazy Cleanup: On access, expired items are deleted
Scheduled Cleanup: Background task runs at configured interval

Configuration:

hub:
  local:
    cleanupEnabled: true
    cleanupInterval: 5m

Manual Cleanup:

int removed = localSharableStore.cleanupExpired();
log.info("Cleaned up {} expired sharables", removed);

Versioning Support

Component and configuration versioning with LATEST support: Store specific version:

ComponentId v1 = ComponentId.of("payment", "1.0");
componentStore.store(component1);

ComponentId v2 = ComponentId.of("payment", "2.0");
componentStore.store(component2);

Retrieve latest:

ComponentId latest = ComponentId.of("payment", "LATEST");
Mono<StringComponent> latestComponent = componentStore.get(latest);

Bidirectional Sync

ApiKeyStore and ApiKeySecretStorage maintain bidirectional sync: When creating a secret:

Secret is stored in ApiKeySecretStorage
Component association is created in ApiKeyStore

When deleting a secret:

Secret is removed from ApiKeySecretStorage
Component association is removed from ApiKeyStore

This ensures consistency between key storage and component lookup.

Limitations

1. No Persistence

All data is lost on application restart:

Components must be redeployed
Configuration must be recreated
API keys must be regenerated
Sharables are lost

Workaround: Use initialization scripts to populate data on startup.

2. Single Instance Only

No shared state between multiple Hub instances:

Each instance has its own in-memory storage
Components registered in one instance are not visible to others
API keys are not shared

Workaround: Use AWS provider or LocalStack for multi-instance deployments.

3. Memory-Bound

Storage is limited by available heap memory:

Large component definitions consume memory
Many API keys increase memory usage
Long-running instances may accumulate sharables

Workaround:

Enable automatic cleanup
Reduce cleanup interval
Monitor heap usage
Restart periodically in development

4. No Audit Trail

No logging of storage operations:

Cannot track who created/modified components
No history of changes
No compliance audit support

Workaround: Use verbose logging for debugging.

5. No Cross-Region Support

Single JVM, single region only:

No replication
No failover
No multi-region deployment

Workaround: Use AWS provider with global tables for multi-region.

Testing

Unit Tests

Use local provider for fast unit tests:

@SpringBootTest
@TestPropertySource(properties = [
    "hub.cloud.provider.type=local"
])
class MyComponentTest {

    @Autowired
    LocalComponentStore componentStore

    @Test
    void testComponentStorage() {
        ComponentId id = ComponentId.of("test-component", "1.0")
        StringComponent component = StringComponent.builder()
            .id(id)
            .definition("workflow { }")
            .build()

        componentStore.store(component).block()

        StringComponent retrieved = componentStore.get(id).block()
        assertThat(retrieved.getDefinition()).isEqualTo("workflow { }")
    }
}

Integration Tests

Clear storage between tests for isolation:

@SpringBootTest
@TestPropertySource(properties = [
    "hub.cloud.provider.type=local",
    "hub.local.cleanupInterval=1s"
])
class MyIntegrationTest {

    @Autowired
    LocalComponentStore componentStore

    @Autowired
    LocalSharableStore sharableStore

    @BeforeEach
    void setUp() {
        // Clear all storage
        componentStore.clear()
        sharableStore.clear()
    }

    @Test
    void testWorkflow() {
        // Test with clean state
    }
}

Test Utilities

Local stores provide test utilities:

// Check storage size
assertThat(componentStore.size()).isEqualTo(5);

// Clear storage
componentStore.clear();
sharableStore.clear();

// Manual cleanup
int removed = sharableStore.cleanupExpired();
assertThat(removed).isEqualTo(3);

Troubleshooting

Components Not Found

Symptom: Component exists but get() returns empty Causes:

Component stored with different revision
Using wrong component name
Storage was cleared

Solutions:

Check component ID matches exactly: ComponentId.of("name", "revision")
Use LATEST if unsure: ComponentId.of("name", "LATEST")
Enable verbose logging: hub.local.verboseLogging: true
Check if storage was cleared between tests

Sharables Expired Too Quickly

Symptom: Tokens expire before expected Causes:

Cleanup interval too aggressive
Expiration set incorrectly
Time zone issues

Solutions:

Increase cleanup interval: hub.local.cleanupInterval: 10m
Check expiration setting: hub.cloud.provider.sharable.defaultTtl: 24h
Use absolute expiration times in tests
Disable cleanup in tests: hub.local.cleanupEnabled: false

Memory Issues

Symptom: OutOfMemoryError after extended runtime Causes:

Too many components stored
Sharables not being cleaned up
Large component definitions
Memory leak in application code

Solutions:

Enable cleanup: hub.local.cleanupEnabled: true
Reduce cleanup interval: hub.local.cleanupInterval: 1m
Clear storage periodically: componentStore.clear()
Increase heap size: -Xmx2g
Monitor heap usage with JMX
Profile with VisualVM or JProfiler

Data Lost Between Tests

Symptom: Test fails because data from previous test is missing Causes:

Using @DirtiesContext between tests
Storage cleared in @BeforeEach
Different Spring context per test

Solutions:

Share Spring context: Use same test configuration
Don’t clear storage if data should persist
Re-populate required data in each test
Use @SpringBootTest with consistent properties

API Keys Not Working

Symptom: API key validation fails Causes:

Secret not created in ApiKeySecretStorage
Association not created in ApiKeyStore
Wrong secret name
Bidirectional sync failed

Solutions:

Use ApiKeySecretStorage.create() (handles sync automatically)
Check secret exists: apiKeySecretStorage.get("key-name")
Check association: apiKeyStore.getSecrets(componentId)
Enable verbose logging to see sync operations

Advanced Usage

Custom Initialization

Populate storage on startup:

@Component
public class LocalStorageInitializer implements ApplicationListener<ApplicationReadyEvent> {

    @Autowired
    private LocalComponentStore componentStore;

    @Autowired
    private LocalApiKeySecretStorage secretStorage;

    @Override
    public void onApplicationEvent(ApplicationReadyEvent event) {
        // Only initialize for local provider
        if (isLocalProvider()) {
            initializeComponents();
            initializeApiKeys();
        }
    }

    private void initializeComponents() {
        ComponentId id = ComponentId.of("default-workflow", "1.0");
        StringComponent component = StringComponent.builder()
            .id(id)
            .definition(loadFromResource("default-workflow.groovy"))
            .build();
        componentStore.store(component).block();
    }

    private void initializeApiKeys() {
        ApiKeySecret secret = ApiKeySecret.builder()
            .name("admin-key")
            .apiKey(UUID.randomUUID().toString())
            .permissions(List.of(/* ... */))
            .build();
        secretStorage.create(secret).block();
    }
}

Monitoring Storage Size

Track storage size for capacity planning:

@Component
@ConditionalOnProperty(name = "hub.cloud.provider.type", havingValue = "local")
public class LocalStorageMonitor {

    @Autowired
    private LocalComponentStore componentStore;

    @Autowired
    private LocalSharableStore sharableStore;

    @Scheduled(fixedRate = 60000)
    public void logStorageSize() {
        log.info("Component store size: {}", componentStore.size());
        log.info("Sharable store size: {}", sharableStore.size());
    }
}

Exporting Data

Export data for backup or migration:

public class LocalDataExporter {

    public void export(LocalComponentStore componentStore, Path outputPath) {
        List<StringComponent> components = componentStore.findAll();
        String json = objectMapper.writeValueAsString(components);
        Files.writeString(outputPath, json);
    }

    public void restore(LocalComponentStore componentStore, Path inputPath) {
        String json = Files.readString(inputPath);
        List<StringComponent> components = objectMapper.readValue(json,
            new TypeReference<List<StringComponent>>() {});
        components.forEach(c -> componentStore.store(c).block());
    }
}

​Local Provider

​Overview

​Quick Start

​1. Add Dependency

​2. Configure Provider

​3. Start the Hub

​4. Optional Configuration

​Architecture

​Storage Design

​Thread Safety

​Reactive Support

​Storage Components

​Component Store

​API Key Store

​API Key Secret Storage

​Sharable Store

​Component Config Storage

​Component Token Service

​Configuration Reference

​Core Configuration

​Local Provider Configuration

​Sharable Expiration Configuration

​Complete Example Configuration

​Usage Scenarios

​Scenario 1: Pure Local Development

​Scenario 2: Integration Testing

​Scenario 3: CI/CD Pipeline

​Scenario 4: Learning and Prototyping

​Switching Between Providers

​Local -> AWS

​Using Profiles for Easy Switching

​Using Environment Variables

​Local vs LocalStack vs AWS

​When to Use Local Provider

​When to Use LocalStack

​When to Use AWS

​Features

​Thread-Safe Concurrent Access

​Reactive Programming Model

​Automatic Cleanup

​Versioning Support

​Bidirectional Sync

​Limitations

​1. No Persistence

​2. Single Instance Only

​3. Memory-Bound

​4. No Audit Trail

​5. No Cross-Region Support

​Testing

​Unit Tests

​Integration Tests

​Test Utilities

​Troubleshooting

​Components Not Found

​Sharables Expired Too Quickly

​Memory Issues

​Data Lost Between Tests

​API Keys Not Working

​Advanced Usage

​Custom Initialization

​Monitoring Storage Size

​Exporting Data

​See Also

Local Provider

Overview

Quick Start

1. Add Dependency

2. Configure Provider

3. Start the Hub

4. Optional Configuration

Architecture

Storage Design

Thread Safety

Reactive Support

Storage Components

Component Store

API Key Store

API Key Secret Storage

Sharable Store

Component Config Storage

Component Token Service

Configuration Reference

Core Configuration

Local Provider Configuration

Sharable Expiration Configuration

Complete Example Configuration

Usage Scenarios

Scenario 1: Pure Local Development

Scenario 2: Integration Testing

Scenario 3: CI/CD Pipeline

Scenario 4: Learning and Prototyping

Switching Between Providers

Local -> AWS

Using Profiles for Easy Switching

Using Environment Variables

Local vs LocalStack vs AWS

When to Use Local Provider

When to Use LocalStack

When to Use AWS

Features

Thread-Safe Concurrent Access

Reactive Programming Model

Automatic Cleanup

Versioning Support

Bidirectional Sync

Limitations

1. No Persistence

2. Single Instance Only

3. Memory-Bound

4. No Audit Trail

5. No Cross-Region Support

Testing

Unit Tests

Integration Tests

Test Utilities

Troubleshooting

Components Not Found

Sharables Expired Too Quickly

Memory Issues

Data Lost Between Tests

API Keys Not Working

Advanced Usage

Custom Initialization

Monitoring Storage Size

Exporting Data

See Also