问题描述
一次上线之后
从JVM上观察
- YGC变频繁,从1次/5min -> 3次/1min
- 出现FGC
3. 从性能来看 左侧是问题,右侧是正常,各项指标异常,MAX能到2s,
- 可用率不足100%
背景描述:
两个需求同时上线,涉及两个接口,部署在同一应用
A. (有问题代码)
B. (正常)
排查步骤:
如何确定呢?
通过nginx切流量,来移除变量观察监控定位到是店内活动页造成的问题
通过location来匹配匹配url
通过upstream来负载转移匹配的url请求到另一台机器上
upstream tomcat_pdjact { server 127.0.0.1:1601 weight=10 max_fails=2 fail_timeout=30s ;
} upstream tomcat_pdjact1 { server *.*.*.*:1601 weight=10 max_fails=2 fail_timeout=30s ;
}
log_format realaddr_21407 '$remote_addr - $remote_user [$time_local] "$http_x_forwarded_for" "$http_j_forwarded_for" ' '"$request" $status $bytes_sent ' '"$http_referer" "$http_user_agent" ' '"$gzip_ratio" ' '$request_time $upstream_response_time';
server { listen 80; server_name 域名 域名 域名 access_log /export/servers/nginx/logs/域名/域名_access.log realaddr_21407; error_log /export/servers/nginx/logs/域名/域名_error.log warn;
root /export/Packages/应用名/latest/;
location / { proxy_next_upstream http_500 http_502 http_503 http_504 error timeout invalid_header; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_pass http://tomcat_*; expires 0; }
location 路径(Mapping){ proxy_next_upstream http_500 http_502 http_503 http_504 error timeout invalid_header; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_pass http://tomcat_pdjact1; expires 0; }
location /logs/ { autoindex off; deny all; } } |
2,3步骤分析之后没看出来大对象是啥
New Long[] 中括号代表类型和可以指定数组大小
new一个数组里面放多个变量 new Long[]{1L,2L,3L}
new一个数组里面只放1个变量 new Long[]{activityPageRequest.getStoreId()}
- 确定是哪个接口造成的影响?
- jstack查看线程
- jmap查看内存
- 分析代码查看改动
- 分析代码改动,发现new了一个大空数组,空间8位数
一行错误代码带来的影响:new Long[activityPageRequest.getStoreId()]