CDK Constructs for connecting AWS Lambda to Tailscale

30 Jan 2025

This blog explains how to connect AWS Lambda functions to a Tailscale network.

Tailscale is a zero-config VPN built on Wireguard, designed to simplify secure networking. It seamlessly connects cloud, on-premises, and personal devices without the typical VPN complexities.

Existing solutions often require Docker or lack a clean integration as a CDK construct. To address these gaps, two new CDK constructs have been created:

Why are we here?

After evaluating multiple options, none fully met the requirements. The ideal solution needed to:

Avoid Docker at runtime
Leverage Lambda Extensions
Remain language-agnostic
Be simple to use and maintain

Tailscale Official Documentation

The Tailscale official documentation relies on an AWS OS-only base image (Amazon Linux 2), which does not support the Lambda Runtime Interface Client. As a result, events from sources like SQS or API Gateway are not passed and processed by the Lambda handler function. The Lambda Runtime Interface Client, which is language-specific, is required for this functionality.

Using Docker images for each language would have been a workaround, but we aimed for a language-agnostic solution that minimized Docker usage due to its overhead.

AWS Blog Post - Serverless Webhook Forwarder

The AWS blog post, Building a secure webhook forwarder using an AWS Lambda extension and Tailscale, validated the use of Lambda Extensions as an alternative to Docker. This approach allows Lambdas to process events normally without additional complexity.

However, the project in the blog is overly complicated and unmaintained. Much of the complexity stems from configuring OpenSSL to sign AWS API requests using Sigv4, which could have been simplified since CURL natively supports AWS request signing.

Below is an example of how the CDK Lambda Extension uses CURL’s AWS Sigv4 capabilities to retrieve a secret from AWS Secrets Manager:

# Get the Tailscale API Key from the Secrets Manager secret
RESPONSE=$(curl -sX POST "https://secretsmanager.${AWS_REGION}.amazonaws.com" \
           --user "${AWS_ACCESS_KEY_ID}:${AWS_SECRET_ACCESS_KEY}" \
           --aws-sigv4 "aws:amz:${AWS_REGION}:secretsmanager" \
           --header "x-amz-security-token: ${AWS_SESSION_TOKEN}" \
           --header "X-Amz-Target: secretsmanager.GetSecretValue" \
           --header "Content-Type: application/x-amz-json-1.1" \
            --data "{
                \"SecretId\": \"${TS_SECRET_API_KEY}\"
            }")
TS_KEY=$(echo "$RESPONSE" | grep -o '"SecretString":"[^"]*"' | sed 's/"SecretString":"\(.*\)"/\1/')

Corey Quinn’s Solution

Corey Quinn’s blog, Corey Writes Open-Source Code for Lambda and Tailscale, demonstrated the feasibility of handling Tailscale connectivity via a Lambda Extension. However, as Corey himself noted, the solution is not neatly packaged or intended for long-term support.

I’m afraid I come to you this morning with terrible news: I’ve been writing code again.

I don’t really want to get people in the habit of trusting random things I put out there that are incredibly important to their security posture. Hence, I’ve put the code on GitHub, distilled the creation process down to a handful of “make” commands, and left the rest for you folks.

That said, his code repo has 150 ⭐️s and predates the AWS blog post. Much of the code for the two new CDK constructs is based on his initial implementation.

Rails Lambda Tailscale Extension

Another AWS Hero, Ken Collins, created a similar project called tailscale-extension, also based on Corey’s work. However, this solution is Docker-based and exposes the Tailscale API Key as an environment variable. Although the extension is language-agnostic, its association with the “rails-lambda” organization might deter potential users.

The Solution

To address these challenges, two CDK constructs were developed: the Tailscale Lambda Extension and the Tailscale Lambda Proxy. The Proxy builds on the Extension, providing a Lambda function with a Function URL (FURL) that forwards HTTP requests to your Tailscale network.

CDK TailScale Lambda Extension

Repo: https://github.com/rehanvdm/tailscale-lambda-extension

Refer to the README for a complete example and code walkthrough. The snippet below demonstrates how to attach the extension to a Lambda function:

import * as cdk from 'aws-cdk-lib';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import { NodejsFunction } from 'aws-cdk-lib/aws-lambda-nodejs';
import * as secretsmanager from 'aws-cdk-lib/aws-secretsmanager';

import { TailscaleLambdaExtension } from 'tailscale-lambda-extension';

export class MyStack extends cdk.Stack {

  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // Define the layer
    const tailscaleExtension = new TailscaleLambdaExtension(this, 'TailscaleExtension');

    // Add the layer to your Lambda function
    const myLambda = new NodejsFunction(this, 'MyFunction', {
      runtime: lambda.Runtime.NODEJS_20_X,
      entry: "/path/to/my/file.ts",
      handler: 'index.handler',
      layers: [tailscaleExtension.layer],
      environment: {
        TS_SECRET_API_KEY: "tailscale-api-key",
        TS_HOSTNAME: "my-lambda",
      }
    });

    // Give the Lambda and thus the Extension permission to read the Tailscale API Key Secret from Secrets Manager 
    const tsApiKeySecret = secretsmanager.Secret.fromSecretNameV2(this, "tailscale-api-key", "tailscale-api-key");
    tsApiKeySecret.grantRead(myLambda);
  }
  
}

The Extension can be used with any Lambda runtime by adding it as a Lambda Layer. It starts the Tailscale daemon and connects to the Tailscale network, exposing a SOCKS5 proxy on port 1055 (socks://localhost:1055) for the main Lambda runtime process to use. Examples of this can be found in the repository’s README.

The diagram below illustrates the sequence of events during a cold start of a Lambda function using the Extension: Where:

t1 - The cold start time introduced by the Extension to start the Tailscale process.
t2 - Your Lambda function’s cold start time.
t3 - The time taken to make a request to the Tailscale network through the SOCKS5 proxy.

Be aware of the following limitations:

The IP address of the Tailscale target must be used.
The Layer adds approximately 50MB to the Lambda package size, primarily due to the Tailscale binaries.
Expect a 5-10 second increase in cold start time due to the Tailscale process initialization.

CDK Tailscale Lambda Proxy

Repo: https://github.com/rehanvdm/tailscale-lambda-proxy

The Proxy Lambda leverages the Tailscale Lambda Extension CDK construct.

It is recommended to use the Proxy Lambda to simplify Tailscale connectivity and reduce cold starts by reusing the same Lambda function for all Tailscale traffic.

Use the Extension directly if:

You have a single Lambda or service that needs to connect to your Tailscale network.
You are comfortable with the Lambda having mixed responsibilities, such as connecting to Tailscale and running business logic.

Use the Proxy (recommended) if:

You have multiple Lambdas or services requiring Tailscale connectivity. The Proxy Lambda creates a “pool of warm connections” to the Tailscale network, ready for use by other Lambdas.
You want to separate responsibilities by dedicating a Lambda to Tailscale connectivity.
Authentication to the Tailscale network is handled at the IAM level, granting access to the Proxy Lambda’s Function URL (FURL) instead of directly to the Tailscale API Secret Manager.

Consider the following scenario:

t1 - Service A (Lambda function) needs to connect to a device on the Tailscale network.
t1 + 30 seconds - Service B also connects to a device on the Tailscale network.
t1 + 60 seconds - Service C also connects to a device on the Tailscale network.

Without the Proxy, each function:

Pays the Lambda Extension cold start time.
Requires access to the same Tailscale Secrets Manager.
Combines Tailscale connectivity with business logic.
Repeats the same Tailscale connection code.

With the Proxy, only the first Lambda pays the cold start penalty, while others reuse the same connection. This approach centralizes Tailscale connectivity and eliminates mixed responsibilities.

Refer to the README for a complete example and code walkthrough. The snippet below shows how to create the Proxy, pass its URL to another Lambda function, and grant invocation permissions:

import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { NodejsFunction } from "aws-cdk-lib/aws-lambda-nodejs";
import * as lambda from "aws-cdk-lib/aws-lambda";
import * as secretsmanager from "aws-cdk-lib/aws-secretsmanager";

import { TailscaleLambdaProxy } from "tailscale-lambda-proxy";

export class MyStack extends cdk.Stack {

  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const tailscaleProxy = new TailscaleLambdaProxy(this, "tailscale-proxy", {
      tsSecretApiKey: secretsmanager.Secret.fromSecretNameV2(this, "tailscale-api-key", "tailscale-api-key"),
      tsHostname: "lambda-test",
    });

    const caller = new NodejsFunction(this, "tailscale-caller", {
      functionName: "tailscale-caller",
      runtime: lambda.Runtime.NODEJS_20_X,
      timeout: cdk.Duration.seconds(30),
      entry: "lib/lambda/tailscale-caller/index.ts",
      environment: {
        TS_PROXY_URL: tailscaleProxy.lambdaFunctionUrl.url,
      }
    });
    tailscaleProxy.lambdaFunctionUrl.grantInvokeUrl(caller); // Important! Allow the caller to invoke the proxy
  }
}

Conclusion

Connecting AWS Lambda functions to a Tailscale network doesn’t have to be complicated. With the Tailscale Lambda Extension and Proxy CDK Constructs, you can simplify secure networking without having to rely on Docker and serverfull infrastructure.