使用Spring Boot将Rest Client异步到DynamoDB

1. Overview

从Spring框架5.0和Spring Boot 2.0开始，该框架提供对异步编程的支持，从2.0版本开始的AWS开发工具包也是如此。

在本文中，我将通过构建简单的反应式REST应用程序来探索使用异步DynamoDB API和Spring Webflux。 假设我们需要处理HTTP请求以检索或存储一些Event（id：string，body：string）。 事件将存储在DynamoDB中。

It might be easier to simply look at the code on Github and follow it there.

2. Dependencies

让我们从WebFlux和DynamoDB SDK的Maven依赖关系开始

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-webflux</artifactId>
        <version>2.2.4.RELEASE</version>
    </dependency>
    <dependency>
        <groupId>software.amazon.awssdk</groupId>
        <artifactId>dynamodb</artifactId>
        <version>2.10.65</version>
    </dependency>
</dependencies>

3. DynamoDB

3.1 Spring Configuration

一个简单的配置就是我们建立了与DynamoDB的连接。 为了测试目的，我们需要指定dynamoEndpoint在Forreal应用程序中，我们需要指定aws区域。

@Configuration
public class AppConfig {
    @Value("${aws.accessKey}")
    String accessKey;

    @Value("${aws.secretKey}")
    String secretKey;

    @Value("${dynamodb.endpoint:}")
    String dynamoEndpoint;

    @Bean
    AwsBasicCredentials awsBasicCredentials(){
        return AwsBasicCredentials.create(accessKey, secretKey);
    }

    @Bean
    DynamoDbAsyncClient dynamoDbAsyncClient(AwsBasicCredentials awsBasicCredentials){
        DynamoDbAsyncClientBuilder clientBuilder = DynamoDbAsyncClient.builder();
        clientBuilder
                .credentialsProvider(StaticCredentialsProvider.create(awsBasicCredentials));
                if(!dynamoEndpoint.isEmpty()){
                    clientBuilder.endpointOverride(URI.create(dynamoEndpoint));
                }
        return clientBuilder.build();
    }
}

application.yaml带有连接细节

aws:
  accessKey: any
  secretKey: any
dynamodb:
  endpoint: http://localhost:8000/

3.2 Reactive DynamoDB Service

Unfortunately, second version of AWS SDK doesn’t have support for DynamoDBMapper yet(you can track mapper’s readiness here), so table creation, sending requests and parsing responses need to be done by “low level” API.

在DynamoDbService我们会：

• 如果不存在则创建表Implement methods for saving and retrieving event

@Service
public class DynamoDbService {

    public static final String TABLE_NAME = "events";
    public static final String ID_COLUMN = "id";
    public static final String BODY_COLUMN = "body";

    final DynamoDbAsyncClient client;

    @Autowired
    public DynamoDbService(DynamoDbAsyncClient client) {
        this.client = client;
    }

    //Creating table on startup if not exists
    @PostConstruct
    public void createTableIfNeeded() throws ExecutionException, InterruptedException {
        ListTablesRequest request = ListTablesRequest.builder().exclusiveStartTableName(TABLE_NAME).build();
        CompletableFuture<ListTablesResponse> listTableResponse = client.listTables(request);

        CompletableFuture<CreateTableResponse> createTableRequest = listTableResponse
                .thenCompose(response -> {
                    boolean tableExist = response.tableNames().contains(TABLE_NAME);
                    if (!tableExist) {
                        return createTable();
                    } else {
                        return CompletableFuture.completedFuture(null);
                    }
                });

        //Wait in synchronous manner for table creation
        createTableRequest.get();
    }

    public CompletableFuture<PutItemResponse> saveEvent(Event event) {
        Map<String, AttributeValue> item = new HashMap<>();
        item.put(ID_COLUMN, AttributeValue.builder().s(event.getUuid()).build());
        item.put(BODY_COLUMN, AttributeValue.builder().s(event.getBody()).build());

        PutItemRequest putItemRequest = PutItemRequest.builder()
                .tableName(TABLE_NAME)
                .item(item)
                .build();

        return client.putItem(putItemRequest);
    }

    public CompletableFuture<Event> getEvent(String id) {
        Map<String, AttributeValue> key = new HashMap<>();
        key.put(ID_COLUMN, AttributeValue.builder().s(id).build());

        GetItemRequest getRequest = GetItemRequest.builder()
                .tableName(TABLE_NAME)
                .key(key)
                .attributesToGet(BODY_COLUMN)
                .build();

        return client.getItem(getRequest).thenApply(item -> {
            if (!item.hasItem()) {
                return null;
            } else {
                Map<String, AttributeValue> itemAttr = item.item();
                String body = itemAttr.get(BODY_COLUMN).s();
                return new Event(id, body);
            }
        });
    }

    private CompletableFuture<CreateTableResponse> createTable() {

        CreateTableRequest request = CreateTableRequest.builder()
                .tableName(TABLE_NAME)

                .keySchema(KeySchemaElement.builder().attributeName(ID_COLUMN).keyType(KeyType.HASH).build())
                .attributeDefinitions(AttributeDefinition.builder().attributeName(ID_COLUMN).attributeType(ScalarAttributeType.S).build())
                .billingMode(BillingMode.PAY_PER_REQUEST)
                .build();

        return client.createTable(request);
    }
}

4. Reactive REST Controller

一个简单的控制器，具有用于通过id检索事件的GET方法和用于在DynamoDB中保存事件的POST方法。 我们可以通过两种方式来做到这一点-用批注实现或摆脱批注并以功能性方式实现。这不会对性能产生影响，在大多数情况下，这完全取决于个人喜好。

4.1 Annotated Controllers

@RestController
@RequestMapping("/event")
public class AnnotatedController {

    final DynamoDbService dynamoDbService;

    public AnnotatedController(DynamoDbService dynamoDbService) {
        this.dynamoDbService = dynamoDbService;
    }

    @GetMapping(value = "/{eventId}", produces = MediaType.APPLICATION_JSON_VALUE)
    public Mono<Event> getEvent(@PathVariable String eventId) {
        CompletableFuture<Event> eventFuture = dynamoDbService.getEvent(eventId);
        return Mono.fromCompletionStage(eventFuture);
    }

    @PostMapping(consumes = MediaType.APPLICATION_JSON_VALUE)
    public void saveEvent(@RequestBody Event event) {
        dynamoDbService.saveEvent(event);
    }
}

4.2 Functional Endpoints

这是一个轻量级的函数编程模型，其中的函数用于路由和处理请求。

@Configuration
public class HttpRouter {

    @Bean
    public RouterFunction<ServerResponse> eventRouter(DynamoDbService dynamoDbService) {
        EventHandler eventHandler = new EventHandler(dynamoDbService);
        return RouterFunctions
                .route(GET("/eventfn/{id}")
                        .and(accept(APPLICATION_JSON)), eventHandler::getEvent)
                .andRoute(POST("/eventfn")
                        .and(accept(APPLICATION_JSON))
                        .and(contentType(APPLICATION_JSON)), eventHandler::saveEvent);
    }

    static class EventHandler {
        private final DynamoDbService dynamoDbService;

        public EventHandler(DynamoDbService dynamoDbService) {
            this.dynamoDbService = dynamoDbService;
        }

        Mono<ServerResponse> getEvent(ServerRequest request) {
            String eventId = request.pathVariable("id");
            CompletableFuture<Event> eventGetFuture = dynamoDbService.getEvent(eventId);
            Mono<Event> eventMono = Mono.fromFuture(eventGetFuture);
            return ServerResponse.ok().body(eventMono, Event.class);
        }

        Mono<ServerResponse> saveEvent(ServerRequest request) {
            Mono<Event> eventMono = request.bodyToMono(Event.class);
            eventMono.map(dynamoDbService::saveEvent);
            return ServerResponse.ok().build();
        }
    }
}

5. Spring DynamoDB Integration Test

5.1 Maven dependencies

为了与DynamoDB运行集成测试，我们需要DynamoDBLocal，它不是真正的DynamoDB，而是在其之上具有实现的DynamoDB接口的SQLite。

<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>DynamoDBLocal</artifactId>
    <version>1.12.0</version>
    <scope>test</scope>
</dependency>

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-dependency-plugin</artifactId>
            <version>2.10</version>
            <executions>
                <execution>
                    <id>copy</id>
                    <phase>test-compile</phase>
                    <goals>
                        <goal>copy-dependencies</goal>
                    </goals>
                    <configuration>
                        <includeScope>test</includeScope>
                        <includeTypes>so,dll,dylib</includeTypes>
                        <!--Keen an eye on output directory - it will be used for starting dynamodb-->
                        <outputDirectory>${project.basedir}/target/native-libs</outputDirectory>
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

<repositories>
    <repository>
        <id>dynamodb-local-oregon</id>
        <name>DynamoDB Local Release Repository</name>
        <url>https://s3-us-west-2.amazonaws.com/dynamodb-local/release</url>
    </repository>
</repositories>

5.2 DynamoDB server

现在我们需要在测试运行之前启动DynamoDB。 我更喜欢将其作为JUnit类规则来执行，但我们也可以将其作为spring bean来执行。

public class LocalDynamoDbRule extends ExternalResource {

    protected DynamoDBProxyServer server;

    public LocalDynamoDbRule() {
        //here we set the path from "outputDirectory" of maven-dependency-plugin
        System.setProperty("sqlite4java.library.path", "target/native-libs");
    }

    @Override
    protected void before() throws Exception {
        this.server = ServerRunner
            .createServerFromCommandLineArgs(new String[]{"-inMemory", "-port", "8000"});
        server.start();
    }

    @Override
    protected void after() {
        this.stopUnchecked(server);
    }

    protected void stopUnchecked(DynamoDBProxyServer dynamoDbServer) {
        try {
            dynamoDbServer.stop();
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }
}

5.3 Running test

现在，我们可以创建一个集成测试，并通过id和save事件测试get事件。

@RunWith(SpringRunner.class)
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class IntegrationTest {

    @ClassRule
    public static LocalDynamoDbRule dynamoDbRule = new LocalDynamoDbRule();

    @Autowired
    private WebTestClient webTestClient;

    @Test
    public void getEvent() {
        // Create a GET request to test an endpoint
        webTestClient
                .get().uri("/event/1")
                .accept(MediaType.APPLICATION_JSON)
                .exchange()
                // and use the dedicated DSL to test assertions against the response
                .expectStatus().isOk()
                .expectBody(String.class).isEqualTo(null);
    }

    @Test
    public void saveEvent() throws InterruptedException {
        Event event = new Event("10", "event");
        webTestClient
                .post().uri("/event/")
                .body(BodyInserters.fromValue(event))
                .exchange()
                .expectStatus().isOk();
        Thread.sleep(1500);
        webTestClient
                .get().uri("/event/10")
                .accept(MediaType.APPLICATION_JSON)
                .exchange()
                .expectStatus().isOk()
                .expectBody(Event.class).isEqualTo(event);
    }
}

6. Docker

在这里，我们将准备在docker中运行的应用程序，以便可以将其部署到AWS。

小提示：从Java 10开始，您可以指定JVM将使用多少内存，具体取决于容器内存。 -XX：MaxRAMPercentage = 75.0意味着JVM不会使用超过75％的容器内存。

Dockerfile

# Use our standard java12 baseimage
FROM openjdk:12-alpine

# Copy the artifact into the container
COPY target/dynamodb-spring-*-exec.jar /srv/service.jar

# Run the artifact and expose the default port
WORKDIR /srv

ENTRYPOINT [ "java", \
    "-XX:+UnlockExperimentalVMOptions", \
    "-XX:+ExitOnOutOfMemoryError", \
    "-XX:MaxRAMPercentage=75.0", \
    "-Djava.security.egd=file:/dev/./urandom", \
    "-jar", "service.jar", \
    "--spring.profiles.active=prod" ]

EXPOSE 8080

构建Docker容器本身dockerbuild-tspring-dynamo。 Alsolet'sseewhatwasgeneratedbysudodockerimagels

REPOSITORY          TAG                 IMAGE ID            CREATED              SIZE
spring-dynamo       latest              a974d880400e        About a minute ago   364MB
openjdk             12-alpine           0c68e7c5b7a0        12 months ago        339MB

---

终于，我们的poc准备好了！

快乐的编码:)

from: https://dev.to//byegor/spring-webflux-with-async-dynamodb-4ig0