CW Logs: Test suite for service internalization #13692

pinzon · 2026-02-03T21:08:14Z

Motivation

As part of the internalization of CloudWatch Logs, this PR introduces a new AWS-validated test suite that consolidates tests from bot the existing LocalStack test suite and the Moto library test suite, organizing them by feature for better maintainability.

Changes

Metric	Count
Total Moto Tests	85
Migrated	73 (~86%)
Not Yet Migrated	12 (~14%)
Additional LocalStack Tests	8

Migration Mapping by Test File

Moto File	LocalStack File	Status
`test_logs/test_logs.py` (log groups)	`test_logs_groups.py`	✅ Migrated
`test_logs/test_logs.py` (streams)	`test_logs_streams.py`	✅ Migrated
`test_logs/test_logs.py` (events)	`test_logs_events.py`	✅ Migrated
`test_logs/test_logs.py` (destinations)	`test_logs_destinations.py`	✅ Migrated
`test_logs/test_logs.py` (resource policies)	`test_logs_resource_policies.py`	✅ Migrated
`test_logs/test_logs.py` (delivery)	`test_logs_delivery.py`	✅ Migrated
`test_logs/test_logs_tags.py`	`test_logs_destinations.py`, `test_logs_groups.py`	✅ Migrated
`test_logs/test_logs_metric_filters.py`	`test_logs_metric_filters.py`	✅ Migrated
`test_logs/test_logs_filter_log_events.py`	`test_logs_filter_events.py`	✅ Migrated
`test_logs/test_integration.py`	`test_logs_subscription_filters.py`	✅ Mostly
`test_logs/test_export_tasks.py`	`test_logs_export_tasks.py`	✅ Migrated (skipped)
`test_logs/test_logs_query/test_boto3.py`	`test_logs_queries.py`	✅ Migrated
`test_logs/test_logs_cloudformation.py`	—	❌ Not Migrated
`test_logs/test_models.py`	—	❌ Not Migrated
`test_logs/test_logs_query/test_query.py`	—	❌ Not Migrated
`test_logs/test_logs_query/test_query_parser.py`	—	❌ Not Migrated

Notes

Assisted by AI 🤖

github-actions · 2026-02-03T21:11:56Z

Test Results (MA/MR) - Preflight, Unit

0 tests 0 ✅ 0s ⏱️
0 suites 0 💤
0 files 0 ❌

Results for commit e37fae4.

github-actions · 2026-02-03T21:12:05Z

Test Results - Preflight, Unit

23 099 tests ±0 21 237 ✅ ±0 6m 17s ⏱️ -4s
1 suites ±0 1 862 💤 ±0
1 files ±0 0 ❌ ±0

Results for commit 589e278. ± Comparison against base commit 2046dff.

♻️ This comment has been updated with latest results.

github-actions · 2026-02-03T22:21:58Z

LocalStack Community integration with Pro

2 files ± 0 2 suites ±0 59m 36s ⏱️ - 58m 32s
2 387 tests - 2 815 2 262 ✅ - 2 589 125 💤 - 226 0 ❌ ±0
2 394 runs - 2 810 2 267 ✅ - 2 584 127 💤 - 226 0 ❌ ±0

Results for commit 76e70f6. ± Comparison against base commit d8ec7bc.

This pull request removes 2932 and adds 117 tests. Note that renamed tests count towards both.

tests.aws.scenario.bookstore.test_bookstore.TestBookstoreApplication ‑ test_lambda_dynamodb
tests.aws.scenario.bookstore.test_bookstore.TestBookstoreApplication ‑ test_opensearch_crud
tests.aws.scenario.bookstore.test_bookstore.TestBookstoreApplication ‑ test_search_books
tests.aws.scenario.bookstore.test_bookstore.TestBookstoreApplication ‑ test_setup
tests.aws.scenario.kinesis_firehose.test_kinesis_firehose.TestKinesisFirehoseScenario ‑ test_kinesis_firehose_s3
tests.aws.scenario.lambda_destination.test_lambda_destination_scenario.TestLambdaDestinationScenario ‑ test_destination_sns
tests.aws.scenario.lambda_destination.test_lambda_destination_scenario.TestLambdaDestinationScenario ‑ test_infra
tests.aws.scenario.loan_broker.test_loan_broker.TestLoanBrokerScenario ‑ test_prefill_dynamodb_table
tests.aws.scenario.loan_broker.test_loan_broker.TestLoanBrokerScenario ‑ test_stepfunctions_input_recipient_list[step_function_input0-SUCCEEDED]
tests.aws.scenario.loan_broker.test_loan_broker.TestLoanBrokerScenario ‑ test_stepfunctions_input_recipient_list[step_function_input1-SUCCEEDED]
…

tests.aws.services.logs.test_logs_delivery.TestDeliveries ‑ test_create_delivery
tests.aws.services.logs.test_logs_delivery.TestDeliveries ‑ test_delete_delivery
tests.aws.services.logs.test_logs_delivery.TestDeliveries ‑ test_describe_deliveries
tests.aws.services.logs.test_logs_delivery.TestDeliveries ‑ test_get_delivery
tests.aws.services.logs.test_logs_delivery.TestDeliveryDestinationPolicies ‑ test_delete_delivery_destination_policy
tests.aws.services.logs.test_logs_delivery.TestDeliveryDestinationPolicies ‑ test_get_delivery_destination_policy
tests.aws.services.logs.test_logs_delivery.TestDeliveryDestinationPolicies ‑ test_put_delivery_destination_policy
tests.aws.services.logs.test_logs_delivery.TestDeliveryDestinations ‑ test_delete_delivery_destination
tests.aws.services.logs.test_logs_delivery.TestDeliveryDestinations ‑ test_delete_delivery_destination_not_found
tests.aws.services.logs.test_logs_delivery.TestDeliveryDestinations ‑ test_describe_delivery_destinations
…

This pull request removes 245 skipped tests and adds 19 skipped tests. Note that renamed tests count towards both.

tests.aws.scenario.kinesis_firehose.test_kinesis_firehose.TestKinesisFirehoseScenario ‑ test_kinesis_firehose_s3
tests.aws.scenario.loan_broker.test_loan_broker.TestLoanBrokerScenario ‑ test_stepfunctions_input_recipient_list[step_function_input4-FAILED]
tests.aws.scenario.mythical_mysfits.test_mythical_misfits.TestMythicalMisfitsScenario ‑ test_deployed_infra_state
tests.aws.scenario.mythical_mysfits.test_mythical_misfits.TestMythicalMisfitsScenario ‑ test_populate_data
tests.aws.scenario.mythical_mysfits.test_mythical_misfits.TestMythicalMisfitsScenario ‑ test_user_clicks_are_stored
tests.aws.services.apigateway.test_apigateway_api.TestApiGatewayApiRestApi ‑ test_get_api_case_insensitive
tests.aws.services.apigateway.test_apigateway_api.TestApiGatewayApiRestApi ‑ test_update_rest_api_concatenation_of_errors
tests.aws.services.apigateway.test_apigateway_api.TestApigatewayIntegration ‑ test_put_integration_request_parameter_bool_type
tests.aws.services.apigateway.test_apigateway_api.TestApigatewayIntegrationResponse ‑ test_integration_response_invalid_responsetemplates
tests.aws.services.apigateway.test_apigateway_basic.TestAPIGateway ‑ test_api_gateway_authorizer_crud
…

tests.aws.services.logs.test_logs_destinations.TestDestinationsTags ‑ test_destination_tags
tests.aws.services.logs.test_logs_events.TestLogsEvents ‑ test_put_log_events_too_new
tests.aws.services.logs.test_logs_events.TestLogsEvents ‑ test_put_log_events_too_old
tests.aws.services.logs.test_logs_export_tasks.TestExportTasks ‑ test_cancel_export_task_not_found
tests.aws.services.logs.test_logs_export_tasks.TestExportTasks ‑ test_create_export_task
tests.aws.services.logs.test_logs_export_tasks.TestExportTasks ‑ test_create_export_task_bucket_not_found
tests.aws.services.logs.test_logs_export_tasks.TestExportTasks ‑ test_create_export_task_log_group_not_found
tests.aws.services.logs.test_logs_export_tasks.TestExportTasks ‑ test_create_export_task_with_prefix
tests.aws.services.logs.test_logs_export_tasks.TestExportTasks ‑ test_describe_export_tasks
tests.aws.services.logs.test_logs_export_tasks.TestExportTasks ‑ test_describe_export_tasks_not_found
…

♻️ This comment has been updated with latest results.

github-actions · 2026-02-09T15:58:48Z

Test Results (amd64) - Acceptance

7 tests ±0 5 ✅ ±0 2m 59s ⏱️ -2s
1 suites ±0 2 💤 ±0
1 files ±0 0 ❌ ±0

Results for commit 589e278. ± Comparison against base commit 2046dff.

♻️ This comment has been updated with latest results.

github-actions · 2026-02-09T16:37:50Z

Test Results (amd64) - Integration, Bootstrap

5 files 5 suites 2h 39m 34s ⏱️
5 718 tests 5 193 ✅ 524 💤 1 ❌
5 724 runs 5 193 ✅ 530 💤 1 ❌

For more details on these failures, see this check.

Results for commit 589e278.

♻️ This comment has been updated with latest results.

steffyP

Great initiative, @pinzon 🥳 🙌

I have a couple of remarks:

in several test classes the fixtures for creating log-group and log-stream are duplicated. Could you consolidate and ideally put them into the fixtures.py?
I spotted a couple of tests (or classes) marked with @pytest.mark.skip(reason="not supported") - could you please elaborate why this is not supported? given that the tests have been ported from moto they should work, shouldn't they?
just my personal opinion (feel free to ignore): i would love to get more hard facts into the PR description, e.g. how many tests have been added/ported from moto?

steffyP · 2026-02-11T14:15:45Z

tests/aws/services/logs/test_logs.py

This file should be deleted if it's now empty :)
also please make sure to get the upstream changes from main, as formatting rules slightly changed (that's causing the merge-conflict).

steffyP · 2026-02-11T14:44:42Z

tests/aws/services/logs/test_logs_metric_filters.py

+    """Tests for metric filter integration with CloudWatch metrics."""
+
+    @markers.aws.validated
+    @pytest.mark.skip(reason="not implemented correctly")


NIT: originally we had this skip note:

@pytest.mark.skip("TODO: failing against community - filters are only in pro -> move test?")

It would be great to add a similar comment here, as it should be implemented in pro.

pinzon · 2026-02-11T21:38:20Z

Thanks @steffyP. I added the report. Removed the extra file and added more details in the skips. I have some theories on why some of them fail but nothing concrete. For example:

The test related with exports fail with a "bucket not found". That happens because logs service searches in memory.
Tests that are related to tags could be failing because some resources are handled by moto and others by LocalStack.

Still, this new test suite made me learn that the new internalized service provider is (#13521) greatly incomplete.

steffyP

Thanks for addressing all suggestions 👍 🙌

Once the pipeline is green (there are currently issues with pro) it should be good to merge :)

steffyP · 2026-02-12T08:26:15Z

Still, this new test suite made me learn that the new internalized service provider is (#13521) greatly incomplete.

@pinzon: yes, that is true. :)
Regarding #13521: I would highly suggest to prepare a realistic scenario for simultaneous logs writing as well, so we are confident that the new provider is able to handle the amount of incoming write requests. I am a bit worried that sqlite may not be the best choice as it only allows one write at a time. I think this was good enough for cloudwatch metrics, but for logs it may slow down the use cases.
Anyhow, thank you so much for tackling this and driving this forward 🙌

pinzon changed the base branch from main to logs/experiment-sqlite February 3, 2026 21:09

pinzon force-pushed the logs/internalize-test-suit branch from 308c858 to 84cc8b9 Compare February 4, 2026 18:56

pinzon requested a review from a team February 6, 2026 21:11

pinzon added aws:logs AWS CloudWatch Logs semver: patch Non-breaking changes which can be included in patch releases docs: skip Pull request does not require documentation changes notes: skip Pull request does not have to be mentioned in the release notes labels Feb 6, 2026

pinzon force-pushed the logs/experiment-sqlite branch from d6d17ca to 2a52f91 Compare February 6, 2026 21:13

pinzon force-pushed the logs/internalize-test-suit branch from bc34e9a to bd83044 Compare February 6, 2026 21:13

pinzon force-pushed the logs/experiment-sqlite branch from 2a52f91 to c597fa9 Compare February 9, 2026 15:26

pinzon force-pushed the logs/internalize-test-suit branch from bd83044 to 1449515 Compare February 9, 2026 15:26

pinzon force-pushed the logs/internalize-test-suit branch from 5fd158c to b403b67 Compare February 10, 2026 17:07

pinzon changed the base branch from logs/experiment-sqlite to main February 10, 2026 17:08

pinzon force-pushed the logs/internalize-test-suit branch from b403b67 to 9915490 Compare February 10, 2026 17:11

pinzon marked this pull request as ready for review February 10, 2026 18:16

pinzon requested a review from steffyP as a code owner February 10, 2026 18:16

steffyP reviewed Feb 11, 2026

View reviewed changes

pinzon requested a review from dominikschubert as a code owner February 11, 2026 20:31

pinzon added 7 commits February 11, 2026 16:27

remove old tests

4890634

add tests for log groups

7f0bcea

add tests for log streams

7786185

add tests for log events

4cb433f

add tests for log deliveries

851e46a

add tests for destinations

3503ef3

add tests for exports

0434b7d

pinzon added 11 commits February 11, 2026 16:27

add tests for log filtering

63d5a08

add tests for metric filters

106f73e

add tests for log insight queries

a82c20c

add tests for logs resource policies

c1783d6

add tests for logs subscription filters

4592c05

plux ini file

ae374be

fix test_get_delivery_source_not_found

66987bb

skip test failing in pro

7fbe51e

plux file

eff2bc9

consolidate fixtures and remove old file

a8519fa

explicit reasons for skips

589e278

pinzon force-pushed the logs/internalize-test-suit branch from 5cfcdc4 to 589e278 Compare February 11, 2026 21:27

steffyP approved these changes Feb 12, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

CW Logs: Test suite for service internalization #13692

CW Logs: Test suite for service internalization #13692

pinzon commented Feb 3, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Feb 3, 2026

Uh oh!

github-actions bot commented Feb 3, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Feb 3, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Feb 9, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Feb 9, 2026 •

edited

Loading

Uh oh!

steffyP left a comment

Uh oh!

steffyP Feb 11, 2026

Uh oh!

steffyP Feb 11, 2026

Uh oh!

pinzon commented Feb 11, 2026

Uh oh!

steffyP left a comment

Uh oh!

steffyP commented Feb 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

CW Logs: Test suite for service internalization #13692

Are you sure you want to change the base?

CW Logs: Test suite for service internalization #13692

Conversation

pinzon commented Feb 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Changes

Migration Mapping by Test File

Related

Notes

Uh oh!

github-actions bot commented Feb 3, 2026

Test Results (MA/MR) - Preflight, Unit

Uh oh!

github-actions bot commented Feb 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results - Preflight, Unit

Uh oh!

github-actions bot commented Feb 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

LocalStack Community integration with Pro

Uh oh!

github-actions bot commented Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results (amd64) - Acceptance

Uh oh!

github-actions bot commented Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results (amd64) - Integration, Bootstrap

Uh oh!

steffyP left a comment

Choose a reason for hiding this comment

Uh oh!

steffyP Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

steffyP Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

pinzon commented Feb 11, 2026

Uh oh!

steffyP left a comment

Choose a reason for hiding this comment

Uh oh!

steffyP commented Feb 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

pinzon commented Feb 3, 2026 •

edited

Loading

github-actions bot commented Feb 3, 2026 •

edited

Loading

github-actions bot commented Feb 3, 2026 •

edited

Loading

github-actions bot commented Feb 9, 2026 •

edited

Loading

github-actions bot commented Feb 9, 2026 •

edited

Loading