Jackson+Feign反序列化问题排查

概述

本文记录在使用Spring Cloud微服务开发时遇到的一个反序列化问题，RPC/HTTP框架使用的是Feign，JSON序列化反序列化工具是Jackson。

问题

测试环境的ELK告警日志如下：

- [43f42bf7] 500 Server Error for HTTP POST "/api/open/dialog/nextQuestion"
feign.codec.DecodeException: Error while extracting response for type [AbaResponse<UserAccountVO>] 
and content type [application/json;charset=UTF-8]; 
nested exception is org.springframework.http.converter.HttpMessageNotReadableException:
JSON parse error: Expected array or string.; 
nested exception is com.fasterxml.jackson.databind.exc.MismatchedInputException: Expected array or string.
at [Source: (ByteArrayInputStream); line: 1, column: 295] (through reference chain: com.aba.common.utils.context.AbaResponse["data"]->com.aba.enduser.common.vo.UserAccountVO["privacySettings"]->java.util.LinkedHashMap["MINIMUM_LEGAL_AGE"]->com.aba.enduser.common.dto.account.PrivacySettings["timestamp"])
at feign.SynchronousMethodHandler.decode(SynchronousMethodHandler.java:180)
at feign.SynchronousMethodHandler.executeAndDecode(SynchronousMethodHandler.java:140)
at feign.SynchronousMethodHandler.invoke(SynchronousMethodHandler.java:78)
at feign.ReflectiveFeign$FeignInvocationHandler.invoke(ReflectiveFeign.java:103)

报错产生自gateway-open服务，gateway-open服务把接口请求/api/open/dialog/nextQuestion转发到dialog服务，dialog服务在Feign调用另外一个enduser服务时发生。很熟悉的报错，Feign反序列化问题。

排查

no Creators, like default construct, exist: cannot deserialize from Object value no delegate- or property-based Creator

为了排查问题，首先想到本地复现问题。本地启动dialog和enduser服务，postman请求dialog服务的接口/dialog/nextQuestion。却出现另一个问题，且这个报错发生在解析requestBody时。在Controller层方法里第一行加断点，程序都没在断点处停止，直接报错：

Caught unhandled generic exception in com.aba.dialog.controller.DialogController
org.springframework.http.converter.HttpMessageConversionException: Type definition error: [simple type, class com.aba.dialog.service.domain.assessment.dialog.answer.DialogAnswerItem]; nested exception is com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of `com.aba.dialog.service.domain.assessment.dialog.answer.DialogAnswerItem` (no Creators, like default construct, exist): cannot deserialize from Object value (no delegate- or property-based Creator)
 at [Source: (PushbackInputStream); line: 1, column: 2]
at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.readJavaType(AbstractJackson2HttpMessageConverter.java:242)
at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.read(AbstractJackson2HttpMessageConverter.java:227)
at org.springframework.web.servlet.mvc.method.annotation.AbstractMessageConverterMethodArgumentResolver.readWithMessageConverters(AbstractMessageConverterMethodArgumentResolver.java:204)

dialog服务最近没有任何改动啊。enduser服务有改动，也和dialog服务无关；毕竟dialog服务断点没进去。

报错代码：

@PostMapping(value = "/nextQuestion")
public DialogDTO handleDialog(@RequestBody DialogAnswerItem item) {
	// 断点行
    String platform = httpServletRequest.getHeader("dialogPlatform");
}

@RequestBody注解的POJO类：

data class DialogAnswerItem(val stateId: StateId,
                            var answer: GivenAnswer,
                            val progress: Double = 0.0,
                            val entryPoint: String? = null)

不甚熟悉的kotlin语言。

看起来一时半会搞不定。

Expected array or string

既然上面的问题没搞定，先解决测试环境的问题。本地启动第三个应用gateway服务，postman模拟调用gateway服务，由gateway负责转发。问题重现：

诸多分析，Google搜到一个靠谱的stackoverflow答案：feign-client-decodeexception-error-while-extracting-response。

修改enduser服务代码：

@Data
@AllArgsConstructor
@NoArgsConstructor
public class PrivacySettings implements Serializable {

    private Boolean value;

    @JsonSerialize(using = LocalDateTimeSerializer.class)
    @JsonDeserialize(using = LocalDateTimeDeserializer.class)
    private LocalDateTime timestamp;
}

本地调试，问题解决。

wait but why。

上面也提到【enduser服务有改动，也和dialog服务无关】，现在为了解决Feign + Jackson远程调用反序列化失败问题，去修改enduser代码，增加2个Jackson提供的注解@JsonSerialize和@JsonDeserialize。

问题虽然解决，总感觉哪里不对劲。但是测试环境里，前端等着使用相关接口，没成多想，发布测试环境。

Feign

结果发布到测试环境后，测试环境里ELK也记录到我一开始在本地调试重现问题时遇到的另外一个问题：
no Creators, like default construct, exist: cannot deserialize from Object value no delegate- or property-based Creator。

看来这个问题是绕不过去的坎。诸般Google/百度搜索与尝试，始终没解决问题。

最后还是仔仔细细看Google给出的第一篇stackoverflow文章no-creators-like-default-construct-exist-cannot-deserialize-from-object-valu，看到：

register jackson module kotlin to ObjectMapper.

才突然意识到，最近对一个common-web组件库做了mvn clean deploy操作。deploy包括install，所以本地环境和测试环境都有相同问题。

再检查common-web下面的配置类：

@Component
public class JsonConfig {
    /**
     * 解决JSON parse error: Unrecognized field "xxx"异常问题
     */
    @Bean
    public MappingJackson2HttpMessageConverter mappingJackson2HttpMessageConverter() {
        MappingJackson2HttpMessageConverter converter = new MappingJackson2HttpMessageConverter();
        ObjectMapper objectMapper = new ObjectMapper();
        objectMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
        converter.setObjectMapper(objectMapper);
        return converter;
    }
}

如上述代码里注释所述，增加此配置是为了解决JSON: Unrecognized field, not marked as ignorable问题，参考stackoverflow的问答jackson-with-json-unrecognized-field-not-marked-as-ignorable。

之前在另外2个服务都出现过此问题，出现此问题的场景都是A服务调用B服务，B服务在业务开发时增加字段（杜绝修改字段和删除字段的开发bad practice）。A服务在微服务体系里还是在使用旧版本的B-api.jar，也就是说A服务的镜像里的jar里还是使用旧的版本，但是在Feign调用B服务时，B服务返回一个新版本的B-api.jar，多了一个字段。于是报错？？

A服务重新编译新版本，则会把新版本的B-api.jar纳入到镜像里，也就是说发布新版本即可解决问题。

想要一劳永逸解决此类问题，在A服务里新增上述配置类就可以了吗？待验证。

考虑到Spring Cloud微服务体系，加字段是很常见的事情，那是不是可以把配置类放在common-web组件库，让所有服务都有此配置类。待验证。

正是因为上述猜想待验证，代码一直在本地。common-web组件库里其他类加以调整时，把JsonConfig配置类编译到dialog服务。

最后，两个问题的解决方法都是移除JsonConfig配置类，并且enduser服务的两个Jackson注解都可以revert。

问题是得以"解决"，但是为啥呢？

后面仔细看dialog服务代码，好几个Jackson配置：

@Configuration
@EnableAsync
open class ApplicationConfig {
    private val log = LoggerFactory.getLogger(this.javaClass)

    @Bean
    open fun restTemplateCommon(): RestTemplate {
        val restTemplate = RestTemplate()
        addOwnMappingJackson2HttpMessageConverter(restTemplate)
        val interceptors = listOf(
            ClientHttpRequestInterceptor { request, body, execution ->
                val headers = request.headers
                headers.add("Accept", MediaType.APPLICATION_JSON_VALUE)
                headers.add("Content-Type", MediaType.APPLICATION_JSON_VALUE)
                execution.execute(request, body)
            }
        )
        restTemplate.interceptors = interceptors
        return restTemplate
    }

    private fun addOwnMappingJackson2HttpMessageConverter(restTemplate: RestTemplate) {
        val converter = MappingJackson2HttpMessageConverter()
        val objectMapper = ObjectMapper()
            .findAndRegisterModules()
            // needed that the LocalDate is not serialized to [2000,1,1] but to "2000-01-01"
            .configure(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS, false)
            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
        converter.objectMapper = objectMapper

        val jacksonMappers = restTemplate.messageConverters
            .filter { httpMessageConverter -> httpMessageConverter is MappingJackson2HttpMessageConverter }

        if (jacksonMappers.isNotEmpty()) {
            restTemplate.messageConverters.remove(jacksonMappers.first())
        }
        restTemplate.messageConverters.add(1, converter)
    }

}

上面这个是kotlin语言。以及

@Configuration
public class HttpConverterConfig implements WebMvcConfigurer {

    @Bean
    public MappingJackson2HttpMessageConverter mappingJackson2HttpMessageConverter() {
        AdaJackson2ObjectMapperBuilder adaJackson2ObjectMapperBuilder = new AdaJackson2ObjectMapperBuilder();
        return new MappingJackson2HttpMessageConverter(adaJackson2ObjectMapperBuilder.build()) {

            @Override
            protected void writeInternal(@NotNull Object object, Type type, @NotNull HttpOutputMessage outputMessage) throws IOException, HttpMessageNotWritableException {
                if (object instanceof String) {
                    Charset charset = this.getDefaultCharset();
                    StreamUtils.copy((String) object, charset, outputMessage.getBody());
                } else {
                    super.writeInternal(object, type, outputMessage);
                }
            }
        };
    }
}

以及：

@Component
public class AdaJackson2ObjectMapperBuilder extends Jackson2ObjectMapperBuilder {

    public AdaJackson2ObjectMapperBuilder() {
        serializationInclusion(JsonInclude.Include.NON_NULL);
        serializationInclusion(JsonInclude.Include.NON_ABSENT);

        featuresToDisable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS,
                SerializationFeature.WRITE_DATE_TIMESTAMPS_AS_NANOSECONDS);
        modules(new AdaModule(), new GuavaModule(), new JavaTimeModule(), new Jdk8Module(), new ParameterNamesModule());
    }

    @Override
    public void configure(@NotNull ObjectMapper objectMapper) {
        super.configure(objectMapper);
        // disable constructor, getter and setter detection
        objectMapper.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.NONE);
        objectMapper.setVisibility(PropertyAccessor.FIELD, JsonAutoDetect.Visibility.ANY);
        objectMapper.registerModule(new KotlinModule());
    }

	private static class AdaModule extends SimpleModule {
	    public AdaModule() {
	        addSerializer(JSONError.class, new JSONErrorSerializer());
	    }
	}
}

以及：

public class JSONErrorSerializer extends JsonSerializer<JSONError> {

    private static final String KEY_STATUS_CODE = "statusCode";
    private static final String KEY_ERROR = "error";
    private static final String KEY_MESSAGE = "message";

    @Override
    public void serialize(JSONError jsonError, JsonGenerator jsonGenerator, SerializerProvider serializerProvider) throws IOException {
        jsonGenerator.writeStartObject();
        jsonGenerator.writeStringField(KEY_STATUS_CODE, String.valueOf(jsonError.getStatusCode()));
        jsonGenerator.writeStringField(KEY_ERROR, jsonError.getError());
        if (jsonError.getMessage() != null && !jsonError.getMessage().isEmpty()) {
            jsonGenerator.writeStringField(KEY_MESSAGE, jsonError.getMessage());
        }
        jsonGenerator.writeEndObject();
    }
}

参考