Early AWS Serverless Patterns (2015-2020)

Context: The Wild West of Serverless

Between 2015 and 2020, building on AWS Lambda required creative workarounds for limitations that no longer exist or are easily solved by managed services today.

• Hard Timeouts: Lambda execution time was capped at 5 minutes (until Oct 2018).
• No Step Functions: AWS Step Functions didn’t exist until late 2016, and adoption took time.
• Limited Destinations: Direct asynchronous invocations had less visibility.

This note documents two specific patterns used to solve real-world problems during this era.

Pattern 1: The “Lambda Chain” (Timeout Evasion)

Problem: Processing large datasets or long-running tasks often exceeded the 5-minute Lambda timeout window. Solution: Decompose the task into small chunks and use SNS (Simple Notification Service) as a loop mechanism to pass context and keep the “serverless” process alive.

Logic

1. Lambda A starts processing a job.
2. It tracks its own execution time.
3. Before hitting the timeout (e.g., at 4m 30s), it stops processing.
4. It saves the current cursor/state (e.g., “Processed item #500”) to the payload.
5. It publishes this payload to an SNS Topic.
6. The SNS Topic triggers Lambda A (or a worker Lambda) again.
7. The new instance reads the cursor and resumes work from item #501.

sequenceDiagram
    participant Trigger
    participant Lambda as Worker Lambda
    participant SNS as SNS Loop Topic

    Trigger->>Lambda: Start Job (Items 1-10000)
    activate Lambda
    loop Processing
        Lambda->>Lambda: Process Item N...
        Lambda->>Lambda: Check Time Remaining
    end
    
    Lambda->>SNS: Publish { cursor: 500, jobId: "abc" }
    Note right of Lambda: Approaching Timeout!
    deactivate Lambda

    SNS->>Lambda: Invoke with { cursor: 500 }
    activate Lambda
    Lambda->>Lambda: Resume from Item 501...
    deactivate Lambda

Pattern 2: Intelligent Push Notification Router

Problem: A user might be logged in on 10+ devices (phones, tablets). Sending a push notification to all of them simultaneously creates a “notification storm” for the user and wastes resources. Solution: A “Smart Router” pattern using DynamoDB to track device activity and prioritize the active device.

Logic

1. Device Registry (DynamoDB): Every time a device interacts with the API, update its last_active_at timestamp in DynamoDB.
2. Notification Trigger: Backend decides to alert the user.
3. Active Check: Lambda queries DynamoDB for all user devices, sorted by activity.
4. Priority Send: Send the notification only to the most recently active device.
5. Fan-out (Optional): If delivery fails or if critical, fan out to the rest.

Architecture

flowchart LR
    Event[Backend Event] --> Notifier[Notifier Lambda]
    
    subgraph "State Store"
        DDB[(DynamoDB<br/>UserDevices)]
    end
    
    subgraph "Routing Logic"
        Notifier -->|1. Query Devices| DDB
        DDB -->|2. Return List| Notifier
        Notifier -->|3. Filter| Logic{Is Active?}
    end
    
    Logic -- Yes (Active Device) --> PhoneA[📱 Active Phone]
    Logic -- No (Inactive) --> SNS[SNS Fanout]
    SNS --> Tablet[Tablet]
    SNS --> OldPhone[Old Phone]

Scenario: The “1000 Logins” Edge Case

In a scenario where a user had 1000 logged-in sessions (e.g., test accounts or extreme power users), fetching all records and iterating was slow.

• Optimization: We introduced a Global Secondary Index (GSI) on last_active_at to efficiently fetch only the top 1 active device (Limit: 1) instead of scanning all 1000 records.

Pattern 3: The “Cold Start Warmer” (CloudWatch Hack)

Problem: Java and C# Lambdas, and even Node.js inside VPCs, suffered from massive “cold starts” (5-10 seconds latency) when scaling from zero. Solution: We created a “Warmer Lambda” (or a CloudWatch Event Rule) that pinged our critical functions every 5-15 minutes to keep the containers “warm” and prevent AWS from reclaiming the execution environment.

graph LR
    CW["CloudWatch Event<br/>(Every 5 mins)"] --> Warmer[Warmer Lambda]
    Warmer -->|"'Ping' (Async)"| API[API Lambda]
    Warmer -->|"'Ping' (Async)"| Worker[Worker Lambda]
    
    style CW fill:#f9f,stroke:#333

Pattern 4: The Monolithic Lambda (“Lambdalith”)

Problem: CloudFormation had a hard limit of 200 resources. Deploying hundreds of micro-functions blew past this limit and made deployment logic complex. Solution: We used libraries like aws-serverless-express to wrap an entire Express.js or Flask application inside one single Lambda function.

• Pros: familiar dev experience, single deployment artifact, fast local testing.
• Cons: huge bundle size, slower cold starts, violated the “single responsibility” principle.

Pattern 5: S3 Event Pipeline (“The River”)

Problem: Orchestrating complex workflows was hard before Step Functions. Solution: We used S3 buckets as state checkpoints. Each processing step saved its output to a specific bucket, which triggered the next Lambda.

• Flow: Upload -> Bucket A -> Resize Lambda -> Bucket B -> Compress Lambda -> Bucket C.
• Downside: Hard to debug “silent failures” if a Lambda failed to write to the next bucket. Usage of S3 for purely signaling (not storage) was costly/slow.

Pattern 6: DynamoDB Stream Aggregator

Problem: Scan and Query on DynamoDB were expensive and slow. Showing a “Total Users” count on a dashboard was architecturally difficult. Solution: We enabled DynamoDB Streams on the main table and connected a “Counter Lambda”.

• Logic:
  • INSERT event → Increment counter in StatsTable.
  • REMOVE event → Decrement counter in StatsTable.
• Result: O(1) read for the total count, maintained asynchronously.

Pattern 7: SNS Scatter-Gather

Problem: Need to perform parallel processing (e.g., fetch data from 3 APIs) and wait for all to finish. Solution:

1. Scatter: Main Lambda publishes message to SNS Topic.
2. Process: 3 different Lambdas subscribed to that topic run in parallel.
3. Gather: Each worker writes results to a DynamoDB table with a shared CorrelationId.
4. Check: A separate stream or poller checks if “3 of 3” results are present.

Pattern 8: KMS-Encrypted Env Vars (“The Secrets Hack”)

Problem: AWS Secrets Manager didn’t exist (launched 2018), and Parameter Store (Systems Manager) was rate-limited or unknown to many. Solution: We stored secrets (API keys, DB passwords) directly in Lambda Environment Variables, but encrypted them with KMS.

• Runtime: The Lambda code had to include a customized boilerplate to decrypt process.env.DB_PASSWORD_ENCRYPTED using the AWS SDK KMS.decrypt() call on cold start.

Evolution: How we do it today

These patterns were necessary innovations at the time, but modern AWS features have simplified them:

Pattern	Historic Solution (2015-2020)	Modern Solution (2025+)
Long Tasks	Lambda → SNS → Lambda Loop	AWS Step Functions (Map state / Standard workflows)
Context Passing	JSON payload in SNS	Step Functions State Machine Data
Smart Push	Custom Lambda + DynamoDB Logic	Amazon Pinpoint / User Segments
Timeouts	5 minutes hard limit	15 min limit + Fargate for longer tasks
Cold Starts	CloudWatch 5-min Ping	Provisioned Concurrency / SnapStart
Deployment	”Lambdalith” (Express.js shim)	Granular Lambdas + SAM/CDK nested stacks
Orchestration	S3 Chains (“The River”)	Step Functions / EventBridge Pipes
Aggregation	DDB Streams + Lambda Counters	DDB Streams (still valid) or Timestream
Fan-out	SNS Scatter-Gather	EventBridge (Rule Targets)
Secrets	KMS Encrypted Env Vars	AWS Secrets Manager / Parameter Store