Skip to content

Fix RPC v2 CBOR timestamp parsing for float#13541

Merged
bentsku merged 3 commits intomainfrom
fix-smithy-timestamp
Dec 17, 2025
Merged

Fix RPC v2 CBOR timestamp parsing for float#13541
bentsku merged 3 commits intomainfrom
fix-smithy-timestamp

Conversation

@bentsku
Copy link
Contributor

@bentsku bentsku commented Dec 17, 2025

Motivation

As reported with #13538, we have an issue parsing CBOR datetime values that are encoded as float. In CBOR, datetime values can encoded as different major types: integer and float (0, 1, 7, 25, 26, 27). See:
https://smithy.io/2.0/additional-specs/protocols/smithy-rpc-v2.html#timestamp-type-serialization and https://datatracker.ietf.org/doc/html/rfc8949.html#section-3.4

We missed it in our testing that the Java SDK v2 would encode outgoing values as Double with Length 8 (27 major type).

This PR also removes the spec patches that are not relevant anymore since the last ASF update, as the new CloudWatch spec contains the switch and the needed changes to support json and smithy-rpc-v2-cbor protocols for the service.

Changes

The fix is to not delegate the usual datetime parsing to the base parser, as the Smithy specs are pretty adamant that CBOR has its own handling. Instead, we now make sure we cap the maximum precision to millisecond, and parse the incoming timestamp normally.

I've done additional testing with the Java SDK v2 to make sure it is now able to send requests and receive our responses without issues.
I've adapted the tests to make sure we return the right format as well, same as AWS (datetime: type 6 with tag 1, encoded as 27 type float).

Tests

I've done extensive testing, and also added Java SDK v2 tests, can be found attached. Not sure how reproducible they are as I'm running them in the context of the LocalStack Utils Java library, but I could confirm I was having the same issue as reported before, and now it works.

Details

package cloud.localstack.awssdkv2;
import org.junit.Assert;

import org.junit.Test;
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.cloudwatch.model.*;
import software.amazon.awssdk.services.cloudwatch.CloudWatchClient;

import java.time.Instant;
import java.time.ZoneOffset;
import java.time.ZonedDateTime;
import java.time.format.DateTimeFormatter;
import java.time.temporal.ChronoUnit;
import java.util.ArrayList;
import java.net.URI;
import java.util.List;

public class BasicFeaturesSDKV2Test {
    @Test
    public void testCWPutMetrics() throws Exception {
        final CloudWatchClient cwClient = CloudWatchClient.builder()
                .credentialsProvider(StaticCredentialsProvider.create(
                        AwsBasicCredentials.create("test", "test")))
                .region(Region.of("us-east-1"))
                .endpointOverride(URI.create("http://localhost.localstack.cloud:4566"))
                .build();

        Dimension dimension = Dimension.builder()
                .name("UNIQUE_PAGES")
                .value("URLS")
                .build();

        // Set an Instant object
        String time = ZonedDateTime.now( ZoneOffset.UTC ).format( DateTimeFormatter.ISO_INSTANT );
        Instant instant = Instant.parse(time);

        double dataPoint = 1.23423;

        MetricDatum datum = MetricDatum.builder()
                .metricName("PAGES_VISITED")
                .unit(StandardUnit.NONE)
                .value(dataPoint)
                .timestamp(instant)
                // .dimensions(dimension)
                .build();

        PutMetricDataRequest request = PutMetricDataRequest.builder()
                .namespace("SITE/TRAFFIC")
                .metricData(datum).build();

        PutMetricDataResponse response =  cwClient.putMetricData(request);
        Assert.assertNotNull(response);

        // GetMetricDataRequest request2 = GetMetricDataRequest.builder()
        // Set the current time and date range for metric query.
        Instant nowDate = Instant.now();
        long hours = 1;
        long minutes = 30;
        Instant startTime = nowDate.minus(5, ChronoUnit.MINUTES);
        Instant endTime = nowDate.plus(hours, ChronoUnit.HOURS).plus(minutes, ChronoUnit.MINUTES);

        Metric met = Metric.builder()
                .metricName("PAGES_VISITED")
                .namespace("SITE/TRAFFIC")
                .build();

        MetricStat metStat = MetricStat.builder()
                .stat("Sum")
                .period(60)  // Assuming period in seconds
                .metric(met)
                .build();

        MetricDataQuery dataQuery = MetricDataQuery.builder()
                .metricStat(metStat)
                .id("foo2")
                .returnData(true)
                .build();

        List<MetricDataQuery> dq = new ArrayList<>();
        dq.add(dataQuery);

        GetMetricDataRequest getMetricDataRequest = GetMetricDataRequest.builder()
                // .maxDatapoints(10)
                // .scanBy(ScanBy.TIMESTAMP_DESCENDING)
                .startTime(startTime)
                .endTime(endTime)
                .metricDataQueries(dq)
                .build();

        // Call the async method for CloudWatch data retrieval.
        GetMetricDataResponse responseGetData = cwClient.getMetricData(getMetricDataRequest);
        Assert.assertNotNull(responseGetData);
        System.out.println(responseGetData.metricDataResults());
    }
}

Related

@bentsku bentsku added this to the 4.13 milestone Dec 17, 2025
@bentsku bentsku self-assigned this Dec 17, 2025
@bentsku bentsku added aws:cloudwatch Amazon CloudWatch area: asf semver: patch Non-breaking changes which can be included in patch releases docs: skip Pull request does not require documentation changes notes: skip Pull request does not have to be mentioned in the release notes labels Dec 17, 2025
@bentsku bentsku linked an issue Dec 17, 2025 that may be closed by this pull request
1 task
Copy link
Member

@alexrashed alexrashed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Great catch with the timestamp precision, and really nice to see this being explicitly covered in an AWS validated snapshot test! 💯 🦸🏽 🐛 🔨

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

praise: Nice to see the patches introduced with #13103 are removed here since the official specs in botocore already contain these changes now! 💯 🧹

from cbor2._decoder import loads as cbor2_loads

# import the unpatched cbor2 on purpose to avoid being polluted by Kinesis-only patches
from cbor2 import loads as cbor2_loads
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

praise: Whoof, great catch!

@github-actions
Copy link

github-actions bot commented Dec 17, 2025

S3 Image Test Results (AMD64 / ARM64)

    2 files      2 suites   8m 12s ⏱️
  552 tests   500 ✅  52 💤 0 ❌
1 104 runs  1 000 ✅ 104 💤 0 ❌

Results for commit ffd14ad.

♻️ This comment has been updated with latest results.

@github-actions
Copy link

github-actions bot commented Dec 17, 2025

Test Results - Preflight, Unit

22 982 tests  +1   21 140 ✅ +1   6m 4s ⏱️ -5s
     1 suites ±0    1 842 💤 ±0 
     1 files   ±0        0 ❌ ±0 

Results for commit ffd14ad. ± Comparison against base commit 14d1ccb.

♻️ This comment has been updated with latest results.

@github-actions
Copy link

Test Results (amd64) - Acceptance

7 tests  ±0   5 ✅ ±0   2m 59s ⏱️ -3s
1 suites ±0   2 💤 ±0 
1 files   ±0   0 ❌ ±0 

Results for commit ffd14ad. ± Comparison against base commit 14d1ccb.

@github-actions
Copy link

Test Results - Alternative Providers

180 tests   - 1 288    39 ✅  - 848   2m 29s ⏱️ - 30m 40s
  1 suites  -     4   141 💤  - 440 
  1 files    -     4     0 ❌ ±  0 

Results for commit ffd14ad. ± Comparison against base commit 14d1ccb.

This pull request removes 1288 tests.
tests.aws.services.cloudformation.api.test_changesets ‑ test_autoexpand_capability_requirement
tests.aws.services.cloudformation.api.test_changesets ‑ test_create_and_then_remove_non_supported_resource_change_set
tests.aws.services.cloudformation.api.test_changesets ‑ test_create_and_then_remove_supported_resource_change_set
tests.aws.services.cloudformation.api.test_changesets ‑ test_create_and_then_update_refreshes_template_metadata
tests.aws.services.cloudformation.api.test_changesets ‑ test_create_change_set_create_existing
tests.aws.services.cloudformation.api.test_changesets ‑ test_create_change_set_invalid_params
tests.aws.services.cloudformation.api.test_changesets ‑ test_create_change_set_missing_stackname
tests.aws.services.cloudformation.api.test_changesets ‑ test_create_change_set_no_changes
tests.aws.services.cloudformation.api.test_changesets ‑ test_create_change_set_update_nonexisting
tests.aws.services.cloudformation.api.test_changesets ‑ test_create_change_set_update_without_parameters
…

@github-actions
Copy link

Test Results (amd64) - Integration, Bootstrap

    5 files  ±0      5 suites  ±0   2h 34m 54s ⏱️ - 1m 12s
5 554 tests ±0  4 997 ✅ ±0  557 💤 ±0  0 ❌ ±0 
5 560 runs  ±0  4 997 ✅ ±0  563 💤 ±0  0 ❌ ±0 

Results for commit ffd14ad. ± Comparison against base commit 14d1ccb.

@bentsku bentsku marked this pull request as ready for review December 17, 2025 16:32
@github-actions
Copy link

LocalStack Community integration with Pro

    2 files  ±0      2 suites  ±0   1h 55m 8s ⏱️ +31s
5 152 tests ±0  4 756 ✅ ±0  396 💤 ±0  0 ❌ ±0 
5 154 runs  ±0  4 756 ✅ ±0  398 💤 ±0  0 ❌ ±0 

Results for commit ffd14ad. ± Comparison against base commit 14d1ccb.

@bentsku bentsku merged commit 033f449 into main Dec 17, 2025
50 checks passed
@bentsku bentsku deleted the fix-smithy-timestamp branch December 17, 2025 16:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: asf aws:cloudwatch Amazon CloudWatch docs: skip Pull request does not require documentation changes notes: skip Pull request does not have to be mentioned in the release notes semver: patch Non-breaking changes which can be included in patch releases

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: Cloutwatch PutMetricData fails with AWS Java SDK 2.40.10

2 participants