Skip to content

Sqs/remove dynamic attrs for avro#13655

Merged
baermat merged 6 commits intomainfrom
sqs/remove-dynamic-attrs-for-avro
Jan 30, 2026
Merged

Sqs/remove dynamic attrs for avro#13655
baermat merged 6 commits intomainfrom
sqs/remove-dynamic-attrs-for-avro

Conversation

@baermat
Copy link
Member

@baermat baermat commented Jan 27, 2026

Motivation

Our new avro framework cannot serialize the lambda functions for our approximate* attributes. The point of those lambdas was that the approximate* attributes are not like the other typical attrs, but rather calculated freshly every time they are accessed. Since we already need some additional code to call the lambda that is returned, and therefore only access the queue attributes via a helper function get_queue_attributes, I went one step further and completely removed them from the stored attributes. The necessary logic was moved completely to get_queue_attributes. /cc @giograno please let me know if this fixes the issue at hand

closes PNX-585

Changes

  • remove ApproximateNumberOfMessages, ApproximateNumberOfMessagesDelayed, ApproximateNumberOfMessagesNotVisible attributes from the model
  • add these attributes on demand only (e.g. `get_queue_attributes(AttributeNames=All)

Tests

Related

@github-actions
Copy link

github-actions bot commented Jan 27, 2026

Test Results - Preflight, Unit

23 084 tests  ±0   21 225 ✅ ±0   6m 33s ⏱️ -6s
     1 suites ±0    1 859 💤 ±0 
     1 files   ±0        0 ❌ ±0 

Results for commit 7467c04. ± Comparison against base commit 44d1022.

This pull request removes 1 and adds 1 tests. Note that renamed tests count towards both.
tests.unit.aws.handlers.test_service.TestServiceExceptionSerializer ‑ test_not_implemented_error_uses_catalog_when_message_is_empty[AVAILABLE_WITH_LICENSE_UPGRADE-is not supported with your LocalStack license]
tests.unit.aws.handlers.test_service.TestServiceExceptionSerializer ‑ test_not_implemented_error_uses_catalog_when_message_is_empty[AVAILABLE_WITH_LICENSE_UPGRADE-is not included within your LocalStack license]

♻️ This comment has been updated with latest results.

@github-actions
Copy link

github-actions bot commented Jan 27, 2026

Test Results (amd64) - Acceptance

7 tests  ±0   5 ✅ ±0   3m 0s ⏱️ -3s
1 suites ±0   2 💤 ±0 
1 files   ±0   0 ❌ ±0 

Results for commit 7467c04. ± Comparison against base commit 44d1022.

♻️ This comment has been updated with latest results.

@baermat baermat added semver: patch Non-breaking changes which can be included in patch releases docs: skip Pull request does not require documentation changes notes: skip Pull request does not have to be mentioned in the release notes labels Jan 27, 2026
@github-actions
Copy link

github-actions bot commented Jan 27, 2026

Test Results (amd64) - Integration, Bootstrap

    5 files  ±    0      5 suites  ±0   1h 35m 8s ⏱️ - 1h 1m 19s
3 564 tests  - 2 033  3 378 ✅  - 1 661  186 💤  - 372  0 ❌ ±0 
3 570 runs   - 2 033  3 378 ✅  - 1 661  192 💤  - 372  0 ❌ ±0 

Results for commit 7467c04. ± Comparison against base commit 44d1022.

This pull request removes 2033 tests.
tests.aws.scenario.bookstore.test_bookstore.TestBookstoreApplication ‑ test_lambda_dynamodb
tests.aws.scenario.bookstore.test_bookstore.TestBookstoreApplication ‑ test_opensearch_crud
tests.aws.scenario.bookstore.test_bookstore.TestBookstoreApplication ‑ test_search_books
tests.aws.scenario.bookstore.test_bookstore.TestBookstoreApplication ‑ test_setup
tests.aws.scenario.kinesis_firehose.test_kinesis_firehose.TestKinesisFirehoseScenario ‑ test_kinesis_firehose_s3
tests.aws.scenario.lambda_destination.test_lambda_destination_scenario.TestLambdaDestinationScenario ‑ test_destination_sns
tests.aws.scenario.lambda_destination.test_lambda_destination_scenario.TestLambdaDestinationScenario ‑ test_infra
tests.aws.scenario.loan_broker.test_loan_broker.TestLoanBrokerScenario ‑ test_prefill_dynamodb_table
tests.aws.scenario.loan_broker.test_loan_broker.TestLoanBrokerScenario ‑ test_stepfunctions_input_recipient_list[step_function_input0-SUCCEEDED]
tests.aws.scenario.loan_broker.test_loan_broker.TestLoanBrokerScenario ‑ test_stepfunctions_input_recipient_list[step_function_input1-SUCCEEDED]
…

♻️ This comment has been updated with latest results.

@github-actions
Copy link

github-actions bot commented Jan 27, 2026

LocalStack Community integration with Pro

    2 files  ±    0      2 suites  ±0   1h 9m 30s ⏱️ - 48m 2s
3 540 tests  - 1 636  3 351 ✅  - 1 437  189 💤  - 199  0 ❌ ±0 
3 542 runs   - 1 636  3 351 ✅  - 1 437  191 💤  - 199  0 ❌ ±0 

Results for commit 7467c04. ± Comparison against base commit 44d1022.

This pull request removes 1636 tests.
tests.aws.scenario.bookstore.test_bookstore.TestBookstoreApplication ‑ test_lambda_dynamodb
tests.aws.scenario.bookstore.test_bookstore.TestBookstoreApplication ‑ test_opensearch_crud
tests.aws.scenario.bookstore.test_bookstore.TestBookstoreApplication ‑ test_search_books
tests.aws.scenario.bookstore.test_bookstore.TestBookstoreApplication ‑ test_setup
tests.aws.scenario.kinesis_firehose.test_kinesis_firehose.TestKinesisFirehoseScenario ‑ test_kinesis_firehose_s3
tests.aws.scenario.lambda_destination.test_lambda_destination_scenario.TestLambdaDestinationScenario ‑ test_destination_sns
tests.aws.scenario.lambda_destination.test_lambda_destination_scenario.TestLambdaDestinationScenario ‑ test_infra
tests.aws.scenario.loan_broker.test_loan_broker.TestLoanBrokerScenario ‑ test_prefill_dynamodb_table
tests.aws.scenario.loan_broker.test_loan_broker.TestLoanBrokerScenario ‑ test_stepfunctions_input_recipient_list[step_function_input0-SUCCEEDED]
tests.aws.scenario.loan_broker.test_loan_broker.TestLoanBrokerScenario ‑ test_stepfunctions_input_recipient_list[step_function_input1-SUCCEEDED]
…

♻️ This comment has been updated with latest results.

@baermat baermat marked this pull request as ready for review January 28, 2026 10:02
@baermat baermat requested a review from giograno January 28, 2026 10:03
Copy link
Member

@thrau thrau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good to me! The callable references were bound to cause issues with persistence at some point, and this solution seems good to me.

Can be merged as is, but have a nit if you want to do an iteration.

Comment on lines 729 to 732
if attr in APPROXIMATE_DYNAMIC_ATTRIBUTES:
# The approximate_* attributes are calculated on the spot when accessed.
# We have a @property for each of those which calculates the value.
value = str(getattr(self, camel_to_snake_case(attr)))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think i would prefer a less magical approach and just have a match/case block. this way we know where the properties are being access:

something like

        result: dict[QueueAttributeName, str] = {}

        for attr in attribute_names:
            try:
                getattr(QueueAttributeName, attr)
            except AttributeError:
                raise InvalidAttributeName(f"Unknown Attribute {attr}.")
            
            value = None
            match attr:
                case QueueAttributeName.ApproximateNumberOfMessages:
                    value = self.approx_number_of_messages
                case QueueAttributeName.ApproximateNumberOfMessagesDelayed:
                    value = ...
                case _:
                    value = self.attributes.get(attr)

            if value == "False" or value == "True":
                result[attr] = value.lower()
            elif value is not None:
                result[attr] = value
        return result

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do, and yes, it is definitely simpler this way. I guess I couldn't resist the magical approach 😃


# the default maximum message size in SQS
DEFAULT_MAXIMUM_MESSAGE_SIZE = 1048576
APPROXIMATE_DYNAMIC_ATTRIBUTES = [
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe just call it DYNAMIC_ATTRIBUTES?

@baermat baermat merged commit 582c9e8 into main Jan 30, 2026
45 checks passed
@baermat baermat deleted the sqs/remove-dynamic-attrs-for-avro branch January 30, 2026 08:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs: skip Pull request does not require documentation changes notes: skip Pull request does not have to be mentioned in the release notes semver: patch Non-breaking changes which can be included in patch releases

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants