Jason4Zhu: HttpClient

Saturday, April 28, 2018

POST请求超时问题和消息体过大问题

超时问题

1. 现象

HttpClient请求后，会在sender端报如下错误：


2018-04-28T10:08:46,144 ERROR [d78a9e9c-eac5-44ad-bf1b-4c0e8d387540 HiveServer2-Handler-Pool: Thread-90] common.HookHelper: failureHookRequest fail, url:http://url/func

com.foo.analytics.auth.org.apache.http.ConnectionClosedException: Premature end of chunk coded message body: closing chunk expected

    at com.foo.analytics.auth.org.apache.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:266) ~[foo-analytics-auth-1.0.0.jar:?]

    at com.foo.analytics.auth.org.apache.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:225) ~[foo-analytics-auth-1.0.0.jar:?]

    at com.foo.analytics.auth.org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:184) ~[foo-analytics-auth-1.0.0.jar:?]

    at com.foo.analytics.auth.org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:135) ~[foo-analytics-auth-1.0.0.jar:?]

    at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284) ~[?:1.8.0_144]

    at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326) ~[?:1.8.0_144]

    at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) ~[?:1.8.0_144]

    at java.io.InputStreamReader.read(InputStreamReader.java:184) ~[?:1.8.0_144]

    at java.io.Reader.read(Reader.java:140) ~[?:1.8.0_144]

    at com.foo.analytics.auth.org.apache.http.util.EntityUtils.toString(EntityUtils.java:227) ~[foo-analytics-auth-1.0.0.jar:?]

    at com.foo.analytics.auth.org.apache.http.util.EntityUtils.toString(EntityUtils.java:308) ~[foo-analytics-auth-1.0.0.jar:?]

    at com.foo.analytics.auth.hive.common.HookHelper.smartEngineRequest(HookHelper.java:219) [foo-analytics-auth-1.0.0.jar:?]

    at com.foo.analytics.auth.hive.Hook.HivePreSemanticAnalyzerHook.func(HivePreSemanticAnalyzerHook.java:44) [foo-analytics-auth-1.0.0.jar:?]

2. 排查思路

首先确定了在webserver端的日志里根本找不到此请求对应的unique id, 也就是说，根本没有进入controller
经google，发现HttpClient和nginx都有对请求的timeout设置，分别做如下改动：

HttpClient端把timeout从60s改为10min：


PoolingHttpClientConnectionManager connManager = new PoolingHttpClientConnectionManager();

// Create socket configuration

SocketConfig socketConfig = SocketConfig.custom().setSoKeepAlive(true)

        .setSoTimeout(180000).setSoReuseAddress(true).setTcpNoDelay(true).build();




// Configure the connection manager to use socket configuration either

// by default or for a specific host.

connManager.setDefaultSocketConfig(socketConfig);

// Validate connections after 1 minute of inactivity

connManager.setValidateAfterInactivity(180000);

connManager.setMaxTotal(100);

connManager.setDefaultMaxPerRoute(20);




// Create global request configuration

RequestConfig defaultRequestConfig = RequestConfig.custom()

        .setCookieSpec(CookieSpecs.DEFAULT).setExpectContinueEnabled(true)

        .setConnectTimeout(180000).setSocketTimeout(180000)

        .setConnectionRequestTimeout(180000).build();




CloseableHttpClient hc = HttpClients.custom().setConnectionManager(connManager)

        .setDefaultRequestConfig(defaultRequestConfig).setDefaultSocketConfig(socketConfig)

        .build();







String encodedMsg = new String(Files.readAllBytes(Paths.get(args[0])), UTF_8);

UrlEncodedFormEntity entity = new UrlEncodedFormEntity(Lists.newArrayList(new BasicNameValuePair("message", encodedMsg)), UTF_8);

HttpUriRequest httpPost = HttpUtils.post("http://url/func", entity);




try (CloseableHttpResponse response = hc.execute(httpPost)) {

    // ...

}

在nginx对应的upstream设置：


proxy_connect_timeout 300s;

proxy_read_timeout 300s;

proxy_send_timeout 300s;

client_body_timeout 300s;

send_timeout 300s;

消息体过大问题

1. 现象

通过HttpClient发送POST请求到SpringBoot（Embedded Tomcat）时，如果send的data较小，则一切正常；如果data较大（~3MB），则controller端接收到的String为空。

2. 排查思路

因为请求是先经过nginx，再redirect到两台webserver。所以先看了下nginx的access log，发现请求会经过nginx并不报错。排除。
直接把sending的message保存到文件，模拟一个test case，直接将请求发送到一台具体的webserver。在controller接收端打印接收到的message和对应的size，为空。
google了下，发现nginx、SpringBoot和tomcat都有对request size的限制：

在application.properties里添加如下参数：


spring.servlet.multipart.max-file-size=30MB

spring.servlet.multipart.max-request-size=30MB

server.tomcat.max-http-post-size=-1

在nginx对应的upstream里添加：

client_max_body_size 10M;

REFERENCE:

1. https://stackoverflow.com/questions/50074707/received-field-is-empty-when-the-size-is-big-in-spring-boot-post-method/50074885#50074885

2. https://blog.csdn.net/LZJLZJLZJLZJLZJLZJ/article/details/79179279

3. https://docs.spring.io/spring-boot/docs/current/reference/html/common-application-properties.html

Thursday, July 2, 2015

Encoding Issues Related With Spring MVC Web-Service Project Based On Tomcat

It won't be more agonised and perplexed when we facing with encoding problems in our project. Here's a brief summary on most of (if possible) scenarios encoding problems may occur. Of all the scenarios, I'll take UTF-8 as an instance.

Project Package Encoding

When building our project via maven, we need to specify charset encoding in the following way:

<build>
  <plugins>
    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>2.0.2</version>
        <configuration>
            <source>1.7</source>
            <target>1.7</target>
            <encoding>UTF-8</encoding>
        </configuration>
    </plugin>
  </plugins>
</build>

Web Request Encoding

As for web request charset encoding, firstly we need to clarify that there are two kinds of request encoding, namely, 'URI encoding' and 'body encoding'. When we do GET operation upon an URL, all parameters in the URL will be applied 'URI encoding',whose default value is 'ISO-8859-1', whereas POST operation will go through 'body encoding'.

URI Encoding

If we don't intend to change the default charset encoding for URI encoding, we could retrieve the correct String parameter using the following code in SpringMVC controller.

new String(request.getParameter("cdc").getBytes("ISO-8859-1"), "utf-8");

Conversely, the way to change the default charset encoding is to edit '$TOMCAT_HOME/conf/server.xml' file, all <connector> tags in which need to be set `URIEncoding="UTF-8"` as an attribute.

<Connector port="8080" protocol="HTTP/1.1"
           connectionTimeout="20000"
           redirectPort="8443"
           URIEncoding="UTF-8" />
...
<Connector port="8009" protocol="AJP/1.3"
           redirectPort="8443"
           URIEncoding="UTF-8" />
...

Body Encoding

This is set by adding filter in web.xml of your project:

<filter>
    <filter-name>utf8-encoding</filter-name>
    <filter-class>org.springframework.web.filter.CharacterEncodingFilter</filter-class>
    <init-param>
        <param-name>encoding</param-name>
        <param-value>utf-8</param-value>
    </init-param>
    <init-param>
        <param-name>forceEncoding</param-name>
        <param-value>true</param-value>
    </init-param>
</filter>
<filter-mapping>
    <filter-name>utf8-encoding</filter-name>
    <servlet-name>your_servlet_name</servlet-name>
</filter-mapping>

HttpClient Request Encoding

When applying HttpClient for POST request, there are times that we need to instantiate a StringEntity object. Be sure that you always add encoding information explicitly.

StringEntity se = new StringEntity(xmlRequestData, CharEncoding.UTF_8);

Java String/Byte Conversion Encoding

Similarly to the above StringEntity scenarios in HttpClient Request Encoding, when it comes to talking about String and byte in Java, we always need to specify encoding voluntarily. String has the concept of charset encoding whereas byte does not.

byte[] bytes = "some_unicode_words".getBytes(CharEncoding.UTF_8);
new String(bytes, CharEncoding.UTF_8);

Tomcat catalog.out Encoding

Tomcat's inner logging charset encoding could be set in '$TOMCAT_HOME/bin/catalina.sh'. Find the location of 'JAVA_OPTS' keyword, then append the following setting:

JAVA_OPTS="$JAVA_OPTS -Dfile.encoding=utf-8"

log4j Log File Encoding

Edit 'log4j.properties' file, add the following property under the corresponding appender.

log4j.appender.appender_name.encoding = UTF-8

OS Encoding

Operation System's charset encoding could be checked by `locale` command on Linux. Append the following content to '/etc/profile' will set encoding to UTF-8.

# -- locale --
export LANG=en_US.UTF-8
export LC_CTYPE=en_US.UTF-8
export LC_NUMERIC=en_US.UTF-8
export LC_TIME=en_US.UTF-8
export LC_COLLATE=en_US.UTF-8
export LC_MONETARY=en_US.UTF-8
export LC_MESSAGES=en_US.UTF-8
export LC_PAPER=en_US.UTF-8
export LC_NAME=en_US.UTF-8
export LC_ADDRESS=en_US.UTF-8
export LC_TELEPHONE=en_US.UTF-8
export LC_MEASUREMENT=en_US.UTF-8
export LC_IDENTIFICATION=en_US.UTF-8
export LC_ALL=en_US.UTF-8

REFERENCE:
1. Get Parameter Encoding
2. Spring MVC UTF-8 Encoding