As our platform evolved, so did our needs around scalability, security, and operational efficiency. Kong served as our go-to API gateway for years—flexible, reliable, and cost-effective—but as we leaned more heavily into AWS-native services, migrating to AWS API Gateway became a clear move.
Why Migrate?
Kong did the job well, but a few key reasons pushed us towards AWS. It provided a more seamless integration with AWS Lambda, IAM, VPCs, and AWS's native monitoring (who would’ve thought). And let’s be honest, maintaining separate Kong clusters, plugins, and configurations across multiple environments for many different services became overly cumbersome. Transitioning would reduce overall operational hassle, to say the least.
Planning the Migration
Migrating an API gateway isn’t trivial. Here’s how we tackled it: taking inventory, feature mapping, utilizing IaC (Infrastructure-as-Code), and doing some incremental cutover. We documented all existing Kong routes, plugins, and upstream services and matched Kong functionalities with AWS API Gateway counterparts. It wasn’t bad at all. After that, it was a no-brainer to use Terraform to utilize an openapi standard in defining our gateway resources, paving the way to eventual adoption of AWS CDK (no more serverless [I have nothing against serverless]). Finally, we migrated services one domain at a time since you always gotta be careful. Can’t just transition something with high traffic and turn it up to 100% on a completely different path. Madness!
How We Did It
Before the migration, our Kong setup involved:

A cluster of EC2 instances behind an AWS ELB, serving as a single API traffic entry point.

ELB distributing traffic to handle fault tolerance and scaling.

Kong clusters hosted in dedicated VPCs, ensuring private, secure API traffic.

Kong handling all proxy routing and traffic logic.


Limitations with Kong:

Heavy operational overhead due to EC2 cluster management, scaling, patching, and scattered configurations.

Transition to AWS API Gateway:
We streamlined our infrastructure with:

Removal of EC2 and ELB dependencies (with ELB temporarily in use during the transition).

Internal API Gateway for private APIs.

AWS API Gateway authorizers for secure header-based routing.


Why This Matters
The migration significantly boosted efficiency, security, and scalability. Here's what improved:

Automatic Scaling: Less manual intervention and better response to traffic spikes.

Simpler Routing: Declarative routing with clearer configurations.

Enhanced Security: AWS-native security and comprehensive monitoring.

Lower Maintenance: Freed up engineering resources from routine tasks.


Lessons Learned
Here's what we learned along the way:

Inventory Matters: Knowing exactly what you have in Kong is crucial.

Automate Everything: Infrastructure as Code (IaC) minimizes errors and speeds up deployments.

Test Frequently: Constantly validate migration steps in staging before going live.

Documentation Is Key: Clear documentation speeds up onboarding and troubleshooting.


Conclusion
Migrating from Kong to AWS API Gateway modernized our API infrastructure, enhanced our security posture, and allowed our engineers to focus on more impactful work. Despite the complexity, the benefits clearly outweighed the challenges.
If you're eyeing a similar move—start small, automate relentlessly, and leverage AWS’s managed services for the smoothest experience IF you are already quite cloud-native.


Migrating from Kong to AWS API Gateway


This isn’t a copy‑paste tutorial. It’s a behind‑the‑scenes look at why I chose a specific shape—hexagonal micro‑orchestrator on SNS → SQS—for a cross‑channel notification pipeline, and the trade‑offs I wrestled with along the way.

Problem Statement
Our platform publishes domain action events whenever users have opted in to notifications through certain channels. We needed to convert those raw events into user‑specific notifications (email, SMS, push) without coupling every producer to every delivery channel.

Goal: 1‑second p95 latency, zero duplicate messages, and the ability to bolt on new event types in a week (not a quarter, lol).

Solution Overview


flowchart TD

    %% ───────── Notification pipeline ─────────
    API["API Suite"]
    SNS_TOPIC["AWS SNS Topic"]
    ORCH["Notification Orchestrator"]
    SQS_QUEUE["SQS Notification Queue"]
    NSVC["Notification Service"]
    SNS_DELIVERY["AWS SNS (delivery)"]

    API -->|Publish notification| SNS_TOPIC
    SNS_TOPIC -->|Subscription payload| ORCH

    %% self-loop to show internal enrichment
    ORCH -->|Enrich &amp; fetch user prefs| ORCH

    ORCH -->|Destination-specific payload| SQS_QUEUE
    SQS_QUEUE -->|Consume notification| NSVC
    NSVC -->|Send via channel| SNS_DELIVERY

The orchestrator is a AWS Spring Boot microservice deployed as a container (ECS/Fargate or Kubernetes) that enriches each event with user preferences, then sends it out to an SQS queue to be consumed and fanned out by the notification (micro)service.
Key Architectural Decisions




DecisionWhy I picked itWhat I gave up



SNS → SQS → NotificationService fan outBuilt‑in reliability & inexpensive at our scale; dead‑simple IAM; millisecond latencyregional only

FIFO SQS Queue to Notification microserviceAllows notification service to delegate how to handle the destinations. That is not an orchestrator’s job.Speedy, but may congest due to a little more things the notification service has to do at once. Possibly slows down throughput

Hexagonal (Ports & Adapters)Keeps core “enrich & route” pure; swap SQS for Kafka with a new adapterMore boilerplate wrappers; devs need pattern literacy

Keycloak for user profilesAlready our source of truth; avoids another DBAdds network hop; Keycloak SDK is verbose

Kotlin + Spring for adaptersFirst‑class coroutines, familiar to the teamCold‑start heavier than Node Lambda

Single Orchestrator vs. per‑channel LamdasCentralised logic; easier A/B testingRisk of god‑service—needs guardrails


Trade‑offs & Alternatives Considered
EventBridge Pipes
Pros: Managed transform stage, fewer lines of code.Cons: At announcement time lacked FIFO target; enrichment limited to Lambda only.
SaaS Routers (Knock, Courier)
Pros: Off‑the‑shelf routing & templates.Cons: Per‑message cost × our volume → non‑starter; vendor lock‑in.
Full ESB (Kafka + ksqlDB)
Pros: Decoupled replay, SQL‑style transforms.Cons: Operability tax; overkill for < 10 K msg/min.
DB Triggers + Event Sourcing
Pros: Bypasses need for SNS, low latency consumer api callsCons: overkill for <10k/min; reconciliation overhead
Lessons Learned

Validate cardinality early. Encapsulating a notification, with a unique account id and embedded destinations allowed us to not bloat our infra with more SQS queues to handle

Central isn’t evil—just measured. One orchestrator is fine if it’s ruthlessly scoped to enrichment + routing.

Observability before prod. With notifications coming from both the consumer and scheduled events, distributed tracing becomes more necessary. This was setup early since the need for modularity and intention when starting out with microsrevices.


Conclusion
A notification pipeline looks deceptively simple until edge‑cases—missing preferences, duplicate events/notifications, out‑of‑order SMS—bite you. By leaning on AWS primitives and a hexagonal core, I get the flexibility of an old‑school ESB without the bloat—and I can add any consumer with a small PR.

Next up (if there’s interest): updating the api suite to save notification subscription history as an audit trail.

Appendix — Minimal Code & Diagram Fragments




PortResponsibilityAdapter Example



UserManagementPortFetch user profileKeycloakUserManagementServicePortAdapter

NotificationPreferencesRepositoryPortLoad notif prefsNotificationPreferencesRepositoryPortAdapter

NotificationServicePortEmit enhanced requestSqsNotificationServicePortAdapter




Building a notification orchestrator: a decision log

I like to write stories to better explain problems I have or concepts I want to share. I still want the message to be clear, concise, and easy to digest, so I’ll give you a glossary to refer back to when reading the story if it it doesn’t click the first time.
Prologue: Because configuring shit is honestly 90% of it
Alright, let's get one thing straight. Setting up cloud infrastructure, wrestling with configurations, deciphering cryptic error messages – sometimes it feels less like software engineering and more like trying to assemble IKEA furniture in the dark during an earthquake while someone shouts jargon at you. I've read the manuals, I've stared at the diagrams, and often, my brain just goes... static.
See, my mind doesn't always click with purely technical terms dropped like anvils from Mount Olympus. It likes stories. It needs analogies, maybe slightly weird ones, to make things stick. I figure, if the core concept makes sense, the fancy names will follow (or at least be Google-able). If I have to imagine mobsters using a secret pigeon network to understand AWS SNS, then so be it.
So, we're taking a detour from the usual dry tech talk. We're heading to Mama Rosa's Pizza in New York City. Why? Because sometimes, explaining secure, asynchronous messaging feels a lot like navigating the delicate communication needs of a family... business.
Is this the official AWS documentation? Absolutely not. Is it slightly ridiculous? Probably. But maybe, just maybe, translating configuration hell and debugging cloud services into the language of pizza, pigeons, and wiseguys will actually make it clearer. At the very least, it might be more fun than reading another spec sheet.
Grab a metaphorical slice, keep your wits about you, and let's dive into how John tried to keep The Boss happy without ending up wearing concrete shoes.
The scene
Mama Rosa’s Pizza, NYC. Best slice in the tristate? Eh. Also, the nerve center for this neighborhood’s… let’s call it “import/export consulting”. I’m John, I sling dough, wipe tables, and sometimes, I make the computers do stuff The Boss wants.

💡
Before we get lost in this beautiful pizza parlor story, I want to note that there, all the way down to the bottom, is a glossary of terms. If you need any help with understanding what something is—I’m not going to assume I can walk through complex concepts like James Baldwin—do not hesitate to take a peek.


The problem: Getting the word out (quietly)
Imagine this: The Boss walks in, smelling of garlic and authority. “Johnny”, he says, polishing his pinky ring. “We need a new way. Tony’s in the kitchen finishing the… baconeggandcheese… he needs to tell Sal out front with the delivery car. Now. No phones, no pagers, too risky. Make it happen.”
Right. Baconeggandcheese. So, I, ol John, need a secure, instant message from the Kitchen Crew (one system) to the Delivery Crew (another system). Untraceable.
My bright idea? We set up a private Pigeon Coop Network. It’s slick. The kitchen sends a coded message: extra garlic on #12 to a specific Handler at the coop. Only the Lofts registered with that guy — like Sal’s lookout post — get the pigeon. Fast, reliable, no digital trail. What could go wrong?
The rule: Don’t test on live pigeons
Now, The Boss has one rule thicker than a snicker: you don’t mess up a live run. Sending a test “baconeggandcheese” signal before the actual baconeggandcheese is ready? That’s a one-way ticket to swimming lessons over near Dumbo.

💡
Before you say anything, ‘cause I have no doubt in my mind you feel like saying something,YES, we also sell baconeggandcheese. Can’t just be a one-trick pony, right? Don’t think about it. Just accept it.


So, now I’m looking for a practice area.
Where the hell am I going to find a practice area?
I requisition the dusty back room and build a Model Pizza Kitchen & Fake Street Corner. It’s a perfect replica—fake ovens, cardboard delivery car, the works. It even has its own Model Pigeon Coop (I call it model because it’s not real) that works just like the real one, but it’s totally self-contained. No messages get out.
To make setup and teardown faster (The Boss hates clutter), I use my Pop-Up Crime Scene Kit. I made it myself (or at least I tell myself I did). It’s this neat box—pop it open, boom, instant model kitchen and fake street corner. Close it, poof, it’s gone. Keeps the real operation clean.
My first attempts: ay ay ay
I rig up the kit, get the model kitchen running, set up a fake Delivery Crew receiver, and try sending a test message. “PINEAPPLE BELONGS ON PIZZA”—something guaranteed not to be real code.

💡
Carl (the author) speaking. I’m from Hawaii, New York, and California. I enjoy pineapple on pizza.


Result: nada. Sal’s cardboard cutout counterpart hears nothing. The pigeons ain’t flying. Shit. Why? Grab an espresso, Johnny, it’s gonna be a long night.

Stumble #1: Using the Wrong Page in the Codebook (AWS Configuration & awslocal)

See, the Family has an Official Ledger & Codebook (our AWS config & credentials files) locked in the back office safe. It's got all the secret addresses and access codes for the real Pigeon Coop Network. But the Model Pigeon Coop in the back room uses different, temporary codes (like access_key_id = test). My message sender device (application.yaml) was still reading from the real ledger page! Of course it didn't work. I also needed to use the right command words (awslocal or specific endpoint settings) to signal I was talking to the model coop, not the real deal. Details, Johnny, details!


Stumble #2: Assholes & The Secret Handshake (CSRF Protection)

Okay, fixed the codebook issue. Now? The fake Delivery Crew receiver shouts YA THOUGHT! (401 Unauthorized). Turns out, Sal's real lookout post has security – a Secret Handshake (CSRF Protection). It stops rival crews (hackers) from sending fake messages ("Deliver the calzones... to precinct 7") trying to mess us up. Any message needs the handshake (a CSRF Token) to be accepted. My messages from the Model Kitchen? They didn't know the handshake! I had to have a word with the "security" on the fake lookout post (SecurityConfig): "Yo, messages from this specific model kitchen are cool. It's me, Johnny, testing. Let 'em through without the handshake for now, okay? Just these practice runs."


Stumble #3: Shouting Inside the Walk-In Freezer (localhost Trap)

This one almost had me pulling my hair out. Everything seemed right, but still no message. Then it hit me: The Model Kitchen (LocalStack running in my Pop-Up Kit/Testcontainers) is like its own little walk-in freezer. When I told my sender inside the freezer to send a message to the receiver at this_freezer:8080 (localhost:8080), it was just echoing off the frozen salami! It wasn't pointing to the receiver's location outside the freezer, on the main floor plan of the back room. I needed the address that navigated out of the freezer to the correct spot. (My Pop-Up Kit had a special way to figure out this address). It's like yelling directions inside a box – nobody outside can hear you right.



Making the sauce right: The working setup for CLI
Finally, clarity. Like a perfect tomato sauce, it needed the right ingredients, in the right order:

Get your configs straight: In the back office safe (~/.aws/), create two clear entries in the Ledger & Codebook: one for the real Pigeon Network, one for the Model Pigeon Coop (localstack). Make sure the model one lists the back room's address (endpoint_url = http://localhost:4566) and the practice codes (test/test). Triple-check that your message sender device (application.yaml) is set to use the localstack profile from the codebook for testing.
 # ~/.aws/config --> Page for the Model Coop
 [profile localstack]
 region=us-east-1 # Doesn't matter much for model, but keep it consistent
 output=json
 endpoint_url = http://localhost:4566 # Address of the Model Coop

 # ~/.aws/credentials --> Practice Access Codes
 [localstack]
 aws_access_key_id = test
 aws_secret_access_key = test

 # application.yaml --> Sender Device Instructions
 spring:
   cloud:
     aws:
       sns:
         region: us-east-1 # Match the profile


Set up a handler in the model coop (create topic): Give the order, specifying the localstack profile. Remember the Handler's official designation (TopicArn).
 Bash
 # Order: "New Handler for baconeggandcheese signals, Model Coop Only"
 aws sns create-topic --name cannoli-signals --profile localstack


Register Sal’s fake loft(subscribe to topic): Tell the "baconeggandcheese" handler where the fake Delivery Crew lookout post (notification_endpoint) is located. Use the address reachable from the Model Kitchen's perspective. Wait for the confirmation pigeon and signal back.
 # Order: "Fake Sal Loft listens to baconeggandcheese Handler"
 aws sns subscribe \
     --topic-arn <TopicArn_from_step_2> \
     --protocol http \
     --notification-endpoint <Address_of_Fake_Sal_Loft> \
     --profile localstack


Send a Test Pigeon (Publish Message): Let's see if this bird flies.
 # Order: "Send 'Test Batch #1' via baconeggandcheese Handler"
 aws sns publish \
     --topic-arn <TopicArn_from_step_2> \
     --message '{"type": "Test#1", "status": "ReadyForPickup...NotReally"}' \
     --profile localstack



AYE! The test message flew through the Model Pigeon Coop, got past the (correctly instructed) security on the fake lookout post, arrived at the right fake address, and the cardboard Sal got the word.
I wiped the sweat off my brow. The system worked. No real pigeons were harmed (or alerted), no sleep schedules with the fishes were arranged. Now, to hook it up to the real network... carefully. The Boss just nodded, a hint of baconeggandcheese still on his lapel. Another day at Mama Rosa's.

💡
Note: If using testcontainers, your notification-endpoint hostname will be host.docker.internal.


Mama Rosa’s glossary of terms
Think of this as the cheat sheet John probably wishes he had. It maps the story elements back to the actual technical concepts being wrestled with.




NameActualDescription



Mama Rosa’s Pizza / The Family BusinessOverall application or systemThe overall application or system you’re building.

The BossPerson demanding resultsThe person who demands results without knowing the how.

JohnDeveloperYou (or maybe not), the dev, trying to make this stuff work under pressure.

Kitchen Crew / KitchenOpsMessage senderThe specific part of your system sending the message or event (e.g. a microservice that completes a task).

Delivery Crew / DeliveryDispatch / Sal / Sal’s Lookout PostMessage receiverThe specific part of your system receiving the message or event (e.g. another microservice that needs to react. A subscriber).

“baconeggandcheese” / coded messageMessage or eventWhatever you’re sending between the services/components in your system (event notification or payload or whatever you got in your back pocket).

Pigeon Coop NetworkAWS Simple Notification Service (AWS SNS)The managed cloud service used for pub-sub messaging.

Handler (at the coop)SNS TopicSpecific channels or mailing lists within SNS; you publish messages to a Topic.

Lofts / Sal’s (Fake) LoftSNS SubscriptionDefines an endpoint (like your receiving microservice) that is registered to receive messages sent to a specific SNS Topic.

Model Pizza Kitchen & Fake Street CornerLocalStackA tool that runs on your local machine and mimics various AWS cloud services (like SNS) so you can test your code without going broke.

Model Pigeon CoopSNS service as emulated by LocalStackBehaves like the real SNS but it’s running entirely in your local testing environment.

Pop-up Crime Scene KitTestcontainersA testing library that lets you programmatically start, manage, and stop temporary software environments (like LocalStack) inside Docker containers, making automated testing much easier and cleaner. This is not needed to run your stuff locally. This is helpful for TESTING. USING THAT JUNIT. BING BAM BADDA BOOM.

Official Ledger & Codebook (in the back office safe)AWS Configuration & Credentials filesThese store your secret access keys and configuration details (like region, output format, and custom endpoint URLs) needed to connect to AWS services.

Model Coop's Page/Entry (in the Ledger)AWS ProfileA specific AWS Profile defined within your config/credentials files, set up to connect to your local LocalStack instance. Makes it easy so you don’t have to add a bunch of flags to your CLI. LocalStack even provides a awslocal, so you don’t even need to have any flags. Crazy, huh?

Message Sender DeviceApplication's Code and Configuration (e.g., settings in application.yaml, environment variables, code that initializes the AWS client)This is what actually uses the credentials and configuration to interact with SNS (or LocalStack

Rival Gangs / AssholesMalicious actorsHackers or unintended external requests trying to interact with your system.

Secret HandshakeCSRF Protection/TokenA security measure to ensure requests come from a trusted source.

Security (on the lookout post)Web Security FrameworkYour application's Web Security Framework (like Spring Security, or middleware in other frameworks). This enforces rules like checking for the valid Secret Handshake (CSRF token) on incoming requests.

Walk-In Freezer / Back Room Address (localhost issue)Docker networking issueThe common localhost networking confusion when using Docker. Inside a container (like the one Testcontainers runs for LocalStack), localhost refers to the container itself, not the machine hosting the container. Connecting to services running on the host from inside the container requires specific addresses (like host.docker.internal or correctly configured Docker networking).

Orders (AWS CLI Commands)AWS Command Line Interface (CLI) commandsStandard AWS Command Line Interface (CLI) commands (aws sns create-topic, aws sns subscribe, aws sns publish, etc.). These are used to manually interact with and manage AWS services (or, when configured with the right profile/endpoint, to interact with LocalStack).




Tag: AWS

Migrating from Kong to AWS API Gateway

Building a notification orchestrator: a decision log

The pigeon coop & the back room: A wiseguy's guide to not screwing up the message