Unify Lambda Runtime using Runtime API #5306

dfangl · 2022-01-19T14:16:08Z

In this PR, we will create a re-implementation of the Lambda executors, to properly support multiple architectures as well as new runtimes, with the use of the Lambda Runtime API (https://docs.aws.amazon.com/lambda/latest/dg/runtimes-api.html).

This is a draft PR for the progress.

To activate the new provider add PROVIDER_OVERRIDE_LAMBDA=asf (note that it is currently not feature complete).

This PR will automatically download the specified release of https://github.com/localstack/lambda-runtime-init as "Runtime Init".

Configuration options:

LAMBDA_PREBUILD_IMAGES=1 (defaults to 0) toggles the building of a lambda version specific container image that includes the code and init before any invoke happens instead of copying them into the container before each invoke. This is actually fairly simillar to the AWS behavior since they also have some initial "optimization" of Lambdas when creating a function. Unfortunately it involves a significant overhead in initial total execution time (from create to an invoke result), so for many single-use lambdas this heavily reduces performance.

sequenceDiagram
    participant LS as LocalStack Endpoint
    participant RAPID as Runtime Init
    participant RIC as Runtime Interface Client
    LS-)RAPID: Start Container
    RAPID-)RIC: Start RIC
    alt Init success
        RIC->>RAPID: next()
        RAPID->>LS: Done
    else
        RIC->>RAPID: InitError
        RAPID->>LS: DoneError
    end
    loop per invocation
        LS-)RAPID: Invoke(id)

        RAPID-->>+RIC: next: Invocation(id)
        alt Invocation Success
            RIC->>RAPID: InvocationResult(id)
        else
            RIC->>RAPID: InvocationError(id)
        end
        par Rapid to LocalStack
            alt Invocation Success
                RAPID->>LS: Response(id)
            else
                RAPID->>LS: ErrorResponse(id)
            end
        and RIC to Rapid
            RIC->>-RAPID: next()
        end
    end

flowchart TD
    API[Lambda API]
    SERVICE[Lambda Service]
    REPO[Repository]
    VM[Version Manager]
    ENV[Environment]
    EXEC[Executor]
    EXEC-CONN[Executor Connection]
    subgraph CONTAINER
        RAPID
    end
    API --> SERVICE
    SERVICE --> REPO
    SERVICE --> VM
    VM --> ENV
    ENV --> EXEC
    EXEC <--> EXEC-CONN
    EXEC-CONN -- invokes --> RAPID
    RAPID -- result/status --> EXEC-CONN
    EXEC -- starts/stops --> CONTAINER

github-actions · 2022-01-19T14:57:56Z

LocalStack integration with Pro

      3 files ±  0       3 suites ±0 51m 12s ⏱️ - 5m 26s
  950 tests +  7   910 ✔️ +  5 40 💤 +2 0 ❌ ±0
1 163 runs +18 1 095 ✔️ +11 68 💤 +7 0 ❌ ±0

Results for commit be7b4e8. ± Comparison against base commit 7aa69c7.

♻️ This comment has been updated with latest results.

coveralls · 2022-01-19T15:07:06Z

Coverage decreased (-4.0%) to 84.715% when pulling 8b0b0ab on lambda-runtime-unification into 4118650 on master.

thrau

This looks fantastic guys! Really well done 🙌 Clean abstractions and mostly easy to understand. Can't wait to give this a try.

I just have a few really minor nits for this PR. :-)

localstack/config.py

thrau · 2022-04-01T15:26:16Z

localstack/services/awslambda/invocation/executor_endpoint.py

+
+class InvokeSendError(Exception):
+    def __init__(self, invocation_id: str, payload: Optional[bytes]):
+        message = f"Error while trying to send invocation to RAPID for id {invocation_id}. Response: {payload}"


is it possible that we're printing control characters to stderr like this?

Payload as well as invocation ids should be strings, so I see a slim chance of that happening here.

thrau · 2022-04-01T15:26:16Z

localstack/services/awslambda/invocation/executor_endpoint.py

+        executor_endpoint = Flask(f"executor_endpoint_{self.port}")
+
+        @executor_endpoint.route("/invocations/<req_id>/response", methods=["POST"])
+        def invocation_response(req_id: str) -> ResponseReturnValue:
+            result = InvocationResult(req_id, request.data)
+            self.service_endpoint.invocation_result(invoke_id=req_id, invocation_result=result)
+            return Response(status=HTTPStatus.ACCEPTED)


this is a very creative solution! i can see how flask enabled the encapsulation of the HTTP server here. i think we could probably improve the code here by using our own Router implementation together with a werkzeug server. The inline methods are clearly necessary with flask, because the method decorators are bound to the flask app being dynamically created, but we could pull them out into methods of the ExecutorEndpoint with the Router class (or even using flask blueprints)

I would generally like to separate the HTTP endpoint implementation from the Server instance that spawns the HTTP server, for better composability and separation of concerns.

Will be done in a follow-up PR! 👍

thrau · 2022-04-01T15:26:16Z

localstack/services/awslambda/invocation/executor_endpoint.py

+    def do_run(self) -> None:
+        endpoint = self._create_endpoint()
+        LOG.debug("Running executor endpoint API on %s:%s", self.host, self.port)
+        endpoint.run(self.host, self.port)


Since we're starting a new flask app for every lambda, i think this is also a good argument against flask. Using something more low-level like Router directly with werkzeug will likely reduce resource usage.

This is definitely out of scope for this revision though :-)

localstack/services/awslambda/invocation/runtime_executor.py

localstack/services/awslambda/invocation/version_manager.py

…identifier

localstack/services/awslambda/invocation/executor_endpoint.py

localstack/services/awslambda/invocation/runtime_executor.py

whummer

Fantastic set of changes, also can't wait to give this a try and see single-digit millisecond Lambda invocation times!! 🚀 😄

Agree with the comments around Flask usage for the individual executor endpoints, there we maybe have an opportunity for further improvement in future iterations.. 👍 Otherwise only two minor comments, but shouldn't hold back the merge. Great job!

localstack/services/awslambda/invocation/lambda_models.py

whummer · 2022-04-01T19:53:22Z

localstack/services/awslambda/invocation/lambda_service.py

+            qualified_arn = function_version.id.qualified_arn()
+            version_manager = self.lambda_version_managers.get(qualified_arn)
+            if version_manager:
+                raise Exception("Version '%s' already created", qualified_arn)


Suggested change

raise Exception("Version '%s' already created", qualified_arn)

raise Exception(f"Version '{qualified_arn}' already created")

will pick these up in later stages 👍

whummer · 2022-04-01T19:53:22Z

localstack/services/awslambda/invocation/version_manager.py

+
+    def start(self) -> None:
+        try:
+            invocation_thread = Thread(target=self.invocation_loop)


nit: we could set invocation_thread.daemon = True, to ensure the thread is terminated on process teardown (probably already covered by the logic around self.shutdown_event, but generally a good pattern to be on the safe side)

localstack/services/generic_proxy.py

dfangl temporarily deployed to localstack-ext-tests Jan 19, 2022 Inactive

dominikschubert assigned dominikschubert and dfangl Jan 19, 2022

dfangl force-pushed the lambda-runtime-unification branch from 0b36615 to bda1d72 Compare Feb 1, 2022

dfangl had a problem deploying to localstack-ext-tests Feb 1, 2022 Failure

dfangl had a problem deploying to localstack-ext-tests Feb 2, 2022 Failure

dominikschubert force-pushed the lambda-runtime-unification branch from 305edca to cc3ca2e Compare Feb 3, 2022

dominikschubert temporarily deployed to localstack-ext-tests Feb 3, 2022 Inactive

dfangl temporarily deployed to localstack-ext-tests Feb 4, 2022 Inactive

alexrashed added the area: asf AWS Server Framework label Feb 8, 2022

dfangl force-pushed the lambda-runtime-unification branch from 51b9b74 to 22f2fca Compare Feb 8, 2022

dfangl temporarily deployed to localstack-ext-tests Feb 8, 2022 Inactive

dfangl temporarily deployed to localstack-ext-tests Feb 9, 2022 Inactive

dominikschubert temporarily deployed to localstack-ext-tests Feb 10, 2022 Inactive

dfangl temporarily deployed to localstack-ext-tests Feb 10, 2022 Inactive

dfangl force-pushed the lambda-runtime-unification branch from 08634c7 to 955aced Compare Feb 18, 2022

dfangl had a problem deploying to localstack-ext-tests Feb 18, 2022 Failure

dfangl force-pushed the lambda-runtime-unification branch from 955aced to bd36f00 Compare Feb 18, 2022

dfangl had a problem deploying to localstack-ext-tests Feb 18, 2022 Failure

dfangl temporarily deployed to localstack-ext-tests Feb 21, 2022 Inactive

dfangl force-pushed the lambda-runtime-unification branch from 1b2bfa3 to da655aa Compare Feb 23, 2022

dfangl had a problem deploying to localstack-ext-tests Feb 23, 2022 Error

dfangl temporarily deployed to localstack-ext-tests Feb 23, 2022 Inactive

dfangl temporarily deployed to localstack-ext-tests Feb 24, 2022 Inactive

dominikschubert and others added 7 commits Apr 1, 2022

skip requestId check for non-asf provider

7e933a7

add lambda asf provider circleci test job

2fbdea5

wip

19653f3

regenerate lambda scaffold without comments

ee23963

actually add shutdownpill to log handler

10db3a7

prune lambda to minimal invoke example

6976c38

fix formatting

bafbcc0

dominikschubert force-pushed the lambda-runtime-unification branch from 680c0e9 to bafbcc0 Compare Apr 1, 2022

dominikschubert temporarily deployed to localstack-ext-tests Apr 1, 2022 Inactive

fix additional linting issues

419e2de

dominikschubert temporarily deployed to localstack-ext-tests Apr 1, 2022 Inactive

mark flaky test as xfail

6c01ac0

dominikschubert temporarily deployed to localstack-ext-tests Apr 1, 2022 Inactive

thrau approved these changes Apr 1, 2022

View changes

fix nits and unintentional disabled code, as well as correct version …

c5a9aec

…identifier

dfangl had a problem deploying to localstack-ext-tests Apr 1, 2022 Failure

dfangl temporarily deployed to localstack-ext-tests Apr 1, 2022 Inactive

dominikschubert reviewed Apr 1, 2022

View changes

localstack/services/awslambda/invocation/executor_endpoint.py Outdated Show resolved Hide resolved

dominikschubert reviewed Apr 1, 2022

View changes

localstack/services/awslambda/invocation/runtime_executor.py Show resolved Hide resolved

dominikschubert reviewed Apr 1, 2022

View changes

localstack/services/awslambda/invocation/runtime_executor.py Outdated Show resolved Hide resolved

change int to str, mistake in fixing nits

be7b4e8

dfangl temporarily deployed to localstack-ext-tests Apr 1, 2022 Inactive

dominikschubert marked this pull request as ready for review Apr 1, 2022

dominikschubert requested a review from alexrashed as a code owner Apr 1, 2022

whummer approved these changes Apr 1, 2022

View changes

dominikschubert merged commit 0b9f90a into master Apr 1, 2022
15 checks passed

viren-nadkarni deleted the lambda-runtime-unification branch Apr 3, 2022

dfangl mentioned this pull request Apr 14, 2022

Allow using amazon-maintained lambda docker image repo #4734

Closed

dfangl mentioned this pull request Aug 22, 2022

Switch to new Lambda provider implementation #6724

Draft

8 tasks

Unify Lambda Runtime using Runtime API #5306

Unify Lambda Runtime using Runtime API #5306

dfangl commented Jan 19, 2022 •

edited by dominikschubert

github-actions bot commented Jan 19, 2022 •

edited

coveralls commented Jan 19, 2022 •

edited

thrau left a comment

thrau Apr 1, 2022

dfangl Apr 1, 2022

thrau Apr 1, 2022 •

edited

dfangl Apr 1, 2022

thrau Apr 1, 2022

whummer left a comment

whummer Apr 1, 2022

dominikschubert Apr 1, 2022

whummer Apr 1, 2022

	raise Exception("Version '%s' already created", qualified_arn)
	raise Exception(f"Version '{qualified_arn}' already created")

Unify Lambda Runtime using Runtime API #5306

Unify Lambda Runtime using Runtime API #5306

Conversation

dfangl commented Jan 19, 2022 • edited by dominikschubert

github-actions bot commented Jan 19, 2022 • edited

LocalStack integration with Pro

coveralls commented Jan 19, 2022 • edited

thrau left a comment

thrau Apr 1, 2022

Choose a reason for hiding this comment

dfangl Apr 1, 2022

Choose a reason for hiding this comment

thrau Apr 1, 2022 • edited

Choose a reason for hiding this comment

dfangl Apr 1, 2022

Choose a reason for hiding this comment

thrau Apr 1, 2022

Choose a reason for hiding this comment

whummer left a comment

whummer Apr 1, 2022

Choose a reason for hiding this comment

dominikschubert Apr 1, 2022

Choose a reason for hiding this comment

whummer Apr 1, 2022

Choose a reason for hiding this comment

dfangl commented Jan 19, 2022 •

edited by dominikschubert

github-actions bot commented Jan 19, 2022 •

edited

coveralls commented Jan 19, 2022 •

edited

thrau Apr 1, 2022 •

edited