【Gitea】Git LFS push中断的问题（issue摘录）

由于Git服务器发送的token会在一定时间内过期，但并没有向Git LFS说明这一点：

根据：git lfs authentication fails with error 401 after 30 minutes of inactivity when uploading very big file #3012

”……GitLab is sending you an authentication token over SSH (via git-lfs-authenticate) but the token expires after 30 minutes.

The Git LFS specification includes support for two features that help with this: expires_at and expires_in. Both are extra properties that can be attached to a response from your provider’s implementation of git-lfs-authenticate and they specify an absolute time or relative duration (relatively) after which your token is no longer valid.

Unfortunately, your provider’s implementation isn’t sending either of these properties back, so Git LFS assumes that the authentication token will be valid forever.

I would recommend sending a link to this issue to your provider’s support@ email and seeing if they might consider adding support to their git-lfs-authenticate program to the above effect. In the meantime, you can run git lfs push again (or, repeatedly, as it were) until all of the objects are uploaded. The server will “skip” objects it already has, so each invocation should take less time than the previous.“

大致：Git服务器通过SSH提供的authentication token会在一定时间过期，但并没有向Git LFS说明这点（或多长时间后token会过期），所以Git LFS假定token一直有效。所以一定时间后token失效，push失败。以及可以继续使用git lfs push操作，因为服务器会跳过已经push上的对象。

因此可以更改gitea配置（app.ini）中的LFS_HTTP_AUTH_EXPIRY选项，设置auth的过期时间

; LFS authentication validity period (in time.Duration), pushes taking longer than this may fail.
LFS_HTTP_AUTH_EXPIRY = 

除此之外，Git LFS会产生大量的请求连接，也与数据库的连接数有关：关于”batch response: Repository or object not found“

根据：LFS: Cloning objects / batch not found #8273

”I suppose that Gitea is exceeding the number of local socket connections permitted by the OS.

Failure: cannot assign requested address

See also explanation and possible solution here:
golang/go#16012 (comment)“

“The problem seems to be the huge amount of connections for the Get request (more than 10k connections for one single client!). See also here:

https://medium.com/@valyala/net-http-client-has-the-following-additional-limitations-318ac870ce9d.
https://medium.com/@nate510/don-t-use-go-s-default-http-client-4804cb19f779”

“……we had more than 10k connections in a TIME_WAIT state. Now there are still approximately 3.5k connections in the TIME_WAIT state. I assume if multiple clients will access the LFS server the same problem could still occur.”

“……During LFS checkout about 3.5k connections and then some minutes later again 2 connections.

This article could be interesting:
http://www.serverframework.com/asynchronousevents/2011/01/time-wait-and-its-design-implications-for-protocols-and-scalable-servers.html”

“From What are CLOSE_WAIT and TIME_WAIT states?

TIME_WAIT indicates that local endpoint (this side) has closed the connection.

I think you may have a network configuration problem. TIME_WAIT lingering too much is a common problem for web servers; usually because the default timeout is too long. Search around because there are many documents dealing with this. Just a “first to show up in a search” pick:

• Huge amount of TIME_WAIT connections
• Avoid TIME_WAIT connections
• Troubleshoot port exhaustion issues”

“I think the problem is more the following:

“Your problem is that you are not reusing your MySQL connections within your app but instead you are creating a new connection every time you want to run an SQL query. This involves not only setting up a TCP connection, but then also passing authentication credentials across it. And this is happening for every query (or at least every front-end web request) and it’s wasteful and time consuming.”

I think this would also speed up Gitea’s LFS server a lot.

source: https://serverfault.com/questions/478691/avoid-time-wait-connections“

”OK We do recycle connections. We use the underlying go sql connection pool.

For MySQL there are the following in the [database] part of the app.ini:

• MAX_IDLE_CONNS 0: Max idle database connections on connnection pool, default is 0
• CONN_MAX_LIFETIME 3s: Database connection max lifetime

https://docs.gitea.io/en-us/config-cheat-sheet/#database-database

I think MAX_IDLE_CONNECTIONS was set to 0 because MySQL doesn’t like long lasting connections.“

”

“Probably the best option, if it’s doable: refactor your protocol so that connections that are finished aren’t closed, but go into an “idle” state so they can be re-used later, instead of opening up a new connection (like HTTP keep-alive).”

“Setting SO_REUSEADDR on the client side doesn’t help the server side unless it also sets SO_REUSEADDR”

I don’t think SO_REUSEADDR applies here. If you’re down into this level of optimization, I’d suggest tuning the tcp_fin_timeout parameter in the kernel. Too short a value will have ill side effects, though; I’d wouldn’t set it below 30 seconds.

But TIME_WAIT is actually the symptom, not the problem.“

Giea中有关数据库的配置：

[database]
MAX_IDLE_CONNS = 
CONN_MAX_LIFETIME = 30s
MAX_OPEN_CONNS =