1 背景
在k8s中基于alpine做底层系统的容器进行git lfs push操作时,发现报错无法上传成功
Fatal error: Server error: http://git.ops.xxxxx.com/xxxx/yyyy.git/gitlab-lfs/objects/b6f9dd313cde39ae1b87e63b9b457029bcea6e9520b5db5de20d3284e4c0259e/3
error: failed to push some refs to 'http://git.ops.xxxx.com/xxxx/yyyy.git'
而git clone、commit等操作均正常,并且git lfs push的操作在另一个机器A上能正常执行。
2 探索思路
2.1 从报错入手
先根据报错信息搜索,基本回答是此报错是git server(gitlab)侧发生错误,不过可以使用如下命令来打印trace跟踪信息(via: https://github.com/git-lfs/git-lfs/issues/3613#issuecomment-483852691)
GIT_TRACE=1 GIT_TRANSFER_TRACE=1 GIT_CURL_VERBOSE=1 git push origin xxx/xxx
确实打印了更多信息,包括与Server的请求过程,如下:
07:18:13.442791 trace git-lfs: HTTP: POST http://xxxx:xxxx@git.ops.yunnex.com/suncard/packages.git/info/lfs/objects/batch
> POST /suncard/packages.git/info/lfs/objects/batch HTTP/1.1
> Host: git.ops.yunnex.com
> Accept: application/vnd.git-lfs+json; charset=utf-8
> Content-Length: 158
> Content-Type: application/vnd.git-lfs+json; charset=utf-8
> User-Agent: git-lfs/2.5.1 (GitHub; linux amd64; go 1.11.4; git 7c940069fa)
>
{"operation":"upload","objects":[{"oid":"b6f9dd313cde39ae1b87e63b9b457029bcea6e9520b5db5de20d3284e4c0259e","size":3}],"ref":{"name":"refs/heads/test-branch"}}07:18:13.618655 trace git-lfs: HTTP: 200
< HTTP/1.1 200 OK
< Transfer-Encoding: chunked
< Cache-Control: max-age=0, private, must-revalidate
< Connection: keep-alive
< Content-Type: application/json; charset=utf-8
< Date: Thu, 24 Sep 2020 07:18:13 GMT
< Etag: W/"0fb39c6aeeda2850fdcf244083961713"
< Server: nginx
< Strict-Transport-Security: max-age=31536000
< X-Content-Type-Options: nosniff
< X-Frame-Options: DENY
< X-Request-Id: 570154fd-b15f-4e8a-a735-b034e98dfbfd
< X-Runtime: 0.165423
< X-Ua-Compatible: IE=edge
< X-Xss-Protection: 1; mode=block
<
07:18:13.619071 trace git-lfs: HTTP: {"objects":[{"oid":"b6f9dd313cde39ae1b87e63b9b457029bcea6e9520b5db5de20d3284e4c0259e","size":3,"actions":{"upload":{"href":"http://git.ops.yunnex.com/suncard/packages.git/gitlab-lfs/objects/b6f9dd313cde39ae1b87e63b9b457029bcea6e9520b5db5de20d3284e4c0259e/3","header":{"Authorization":"Basic cHJvZHVjdGlvbjpVc2VfMTk5MA=="}}}}]}
{"objects":[{"oid":"b6f9dd313cde39ae1b87e63b9b457029bcea6e9520b5db5de20d3284e4c0259e","size":3,"actions":{"upload":{"href":"http://git.ops.yunnex.com/suncard/packages.git/gitlab-lfs/objects/b6f9dd313cde39ae1b87e63b9b457029bcea6e9520b5db5de20d3284e4c0259e/3","header":{"Authorization":"Basic cHJvZHVjdGlvbjpVc2VfMTk5MA=="}}}}]}07:18:13.619307 trace git-lfs: tq: starting transfer adapter "basic"
07:18:13.619345 trace git-lfs: xfer: adapter "basic" Begin() with 8 workers
07:18:13.619383 trace git-lfs: xfer: adapter "basic" started
07:18:13.619413 trace git-lfs: xfer: adapter "basic" worker 1 starting
07:18:13.619537 trace git-lfs: xfer: adapter "basic" worker 1 waiting for Auth
07:18:13.619549 trace git-lfs: xfer: adapter "basic" worker 5 starting
07:18:13.619585 trace git-lfs: xfer: adapter "basic" worker 5 waiting for Auth
07:18:13.619642 trace git-lfs: xfer: adapter "basic" worker 0 starting
07:18:13.619673 trace git-lfs: xfer: adapter "basic" worker 6 starting
07:18:13.619681 trace git-lfs: xfer: adapter "basic" worker 0 processing job for "b6f9dd313cde39ae1b87e63b9b457029bcea6e9520b5db5de20d3284e4c0259e"
07:18:13.619702 trace git-lfs: xfer: adapter "basic" worker 7 starting
07:18:13.619722 trace git-lfs: xfer: adapter "basic" worker 7 waiting for Auth
07:18:13.619732 trace git-lfs: xfer: adapter "basic" worker 2 starting
07:18:13.619775 trace git-lfs: xfer: adapter "basic" worker 2 waiting for Auth
07:18:13.619723 trace git-lfs: xfer: adapter "basic" worker 4 starting
07:18:13.619775 trace git-lfs: xfer: adapter "basic" worker 3 starting
07:18:13.619805 trace git-lfs: xfer: adapter "basic" worker 3 waiting for Auth
07:18:13.619788 trace git-lfs: xfer: adapter "basic" worker 4 waiting for Auth
07:18:13.619697 trace git-lfs: xfer: adapter "basic" worker 6 waiting for Auth
07:18:13.621028 trace git-lfs: HTTP: PUT http://git.ops.yunnex.com/suncard/packages.git/gitlab-lfs/objects/b6f9dd313cde39ae1b87e6