python database is locked_#3503 (OperationalError: database is locked) – The Trac Project...

抄送:

exarkun@… added

Hiya Mark!

hiya Andy!

Christian Boos, 14年 ago

属主:

从 Jonas Borgström 改变为 Christian Boos

里程碑:

→ 0.10

We should probably transform this into a TracError …

Since two weeks (still with 0.9.6) we are experiencing "database locked" errors quite often, maybe 3 or 4 times per week.

What can I do to help debugging this ?

Until now I only have the error message which appears in Trac.

Somebody does something, then something goes wrong, and maybe the first user after that event gets a "database locked".

How can I find out more ?

Alex

跟随：

17

Christian Boos, 14年 ago

Unfortunately, "database locked" errors are somewhat inherent the to default architecture used by Trac (see also #3446 for enhancement proposals).

This ticket is about the suggestion of transforming the "database locked" error into a more "friendly" error message, if that's possible.

The only situations where the "database locked" errors are indeed problematic are the ones in which the database remains locked for all subsequent requests. Fortunately, we haven't seen that since a while, now.

Just as a note: we are running trac in fastcgi-mode. If the lock happens, it doesn't go away. We have to login to the server and kill the trac.fcgi processes with killall.

At least one of them can't be killed friendly, but SIGKILL has to be used. Then we have to restart apache.

We are running Trac 0.9.4 and sqlite etc. from march or april.

Alex

Similarly, I'm using WSGI and if the lock happens, I also have to SIGKILL the server process to get it to exit.

SQLite, subversion, and Python bindings for both are the versions packaged in Ubuntu Breezy.

At this point we're not in need of more feedback from 0.9.x versions, since there have been improvement in the trunk since then. So, if you're interested in providing more testing please upgrade to 0.10b1 or the trunk.

cboos: do you have a plan for addressing this in 0.10, or can we push it to 0.10.1 since there doesn't seem to be sufficient information to debug it?

I should have mentioned the version of trac I am using is a fairly recent trunk, r3585 (0.9.x had no WSGI support, did it?)

in reply to: 10

Christian Boos, 14年 ago

里程碑:

0.10 → 0.10.1

Replying to mgood:

cboos: do you have a plan for addressing this in 0.10, or can we push it to 0.10.1 since there doesn't seem to be sufficient information to debug it?

For 0.10, I only planned to implement the suggestion that we should convert the database is locked error to a more "friendly" TracError, as the lock error seems to be inevitable at this point, but this shouldn't block 0.10 and can well be post-poned to 0.10.1.

However, the fact that there are apparently still serious persitent locks around would probably justify that we reopen #1661, which was closed due to lack of feedback and because I believed that people stopped being affected by this, which is apparently wrong…

Unfortunately I have been hitting "Database is locked" from switching to current trunk. Seems something that changed in last ~20 commits that caused database to be locked rather often.

Christian Boos, 14年 ago

#3756 marked as duplicate.

in reply to: 13

;

跟随：

16

匿名用户, 14年 ago

Replying to kre:

Unfortunately I have been hitting "Database is locked" from switching to current trunk. Seems something that changed in last ~20 commits that caused database to be locked rather often.

Does this mean it happens more often with 0.10.0 than with previous versions ?

If that's the case, we can't upgrade to 0.10.0, because this would block us even more.

(I understand that reports about 0.9.x don't help you in any way anymore)

Alex

in reply to: 15

Christian Boos, 14年 ago

Replying to anonymous:

Does this mean it happens more often with 0.10.0 than with previous versions ?

I don't think so. In addition, with r3830, the locks seem to be much harder to reproduce. At least, I couldn't get one anymore, using tracd.

I think I'll backport to 0.10.1, but more user feedback is needed first, so I'll be glad to hear from you if 0.10 + r3830 helps.

If that's the case, we can't upgrade to 0.10.0, because this would block us even more.

(I understand that reports about 0.9.x don't help you in any way anymore)

No, 0.9.x in general will not get any more improvements, but only security bug fixes. Your best move would be to have a test drive of 0.10 like suggested above: this will help us to make a more robust 0.10.1 that you'll also benefit from.

in reply to: 7

;

跟随：

18

ThurnerRupert, 14年 ago

Replying to cboos:

Unfortunately, "database locked" errors are somewhat inherent the to default architecture used by Trac (see also #3446 for enhancement proposals).

This ticket is about the suggestion of transforming the "database locked" error into a more "friendly" error message, if that's possible.

The only situations where the "database locked" errors are indeed problematic are the ones in which the database remains locked for all subsequent requests. Fortunately, we haven't seen that since a while, now.

unfortunately, we had it today. v 0.10.

in reply to: 17

ThurnerRupert, 14年 ago

Replying to cboos:

Unfortunately, "database locked" errors are somewhat inherent the to default architecture used by Trac (see also #3446 for enhancement proposals).

This ticket is about the suggestion of transforming the "database locked" error into a more "friendly" error message, if that's possible.

The only situations where the "database locked" errors are indeed problematic are the ones in which the database remains locked for all subsequent requests. Fortunately, we haven't seen that since a while, now.

unfortunately, we had it today. v 0.10.

read #1661 … we did not wait a couple of minutes, but 3-4 times as long as with the normal (frequent) locking errors.

处理结果:

→ duplicate

状态:

new → closed

seems to be a duplicate for #3446, there is a solution. we try it and report back.

Christian Boos, 14年 ago

优先级:

high → highest

关键词:

session pysqlite database lock added

处理结果:

duplicate

状态:

closed → reopened

No, it's not the same. The error reported in this ticket is specific to the req.session.save() done after the response was sent. The problem here is that we attempt to do a req.send_error() which will necessarily fail (as a response was already sent).

Also, it doesn't seem to be a good idea to persist the session changes if the request actually failed.

It would be possible to catch the session save failure and present it to the user if that is done before sending the response. However, this might increase the visible rate of database locks, which I think would be nevertheless OK given the recent progresses on this front.

跟随：

22

Christian Boos, 14年 ago

Above changes implemented in r4048.

I should note that I still have a hard way to trigger database locks, even with that change, probably thanks to r3830 (see #3446).

However, I was able to see the problem with the TracTimeline just after a commit (i.e. #2902).

Will backport this later.

in reply to: 21

Christian Boos, 14年 ago

Replying to cboos:

Above changes implemented in r4048.

Oops, not really… I actually misplaced the req.session.save, and of course there's no locks as nothing gets changed :\

Christian Boos, 14年 ago

…and with r4050, this should now be done correctly.

I still don't get database locks anymore… :)

The only database error that's still there (rarely) is: ERROR: columns sid, authenticated are not unique, which is a well known issue and not that critical (see #3563).

Christian Boos, 14年 ago

(and now I'll take a break, some sleep etc., as I realized that r4048 was OK; r4050 only moves the session.save after the post-processing, which is of course a bit better but wasn't that critical ;) )

跟随：

27

ThurnerRupert , 14年 ago

处理结果:

→ fixed

状态:

reopened → closed

we upgraded recently and we do not experience lock errors any more. before there was one lock error every 5-10 minutes caused by the company search engine spidering the trac instance.

Christian Boos, 14年 ago

处理结果:

fixed

状态:

closed → reopened

Thanks very much for the feedback, but I've left the issue opened as a reminder for me to backport it to 0.10-stable, as currently this is only fixed in trunk.

in reply to: 25

Christian Boos, 14年 ago

处理结果:

→ fixed

状态:

reopened → closed

r4048 was adapted for 0.10-stable in r4171.

we upgraded recently and we do not experience lock errors any more. before there was one lock error every 5-10 minutes caused by the company search engine spidering the trac instance.

and this is probably due to r3830, ported in r3910.

We just experienced the problem with version 0.11.1 - it went away after a few minutes.

处理结果:

fixed

状态:

closed → reopened

We have started experiencing this in 0.10.4 as of a few days ago. What gives?

in reply to: 29

Emmanuel Blot, 12年 ago

处理结果:

→ fixed

状态:

reopened → closed

Replying to anonymous:

We have started experiencing this in 0.10.4 as of a few days ago. What gives?

Options are:

move to 0.11.2,

move to fastcgi, instead of mod_python,

move to PostgreSQL,

We just experienced the problem with version 0.11.5.

Can anyone give a way to relock the database?

If the lock is not release,will trac work well without any problems?

跟随：

33

Christian Boos, 11年 ago

The database is locked issue is nearly always a transient issue.

If so, the problem is believed to be fixed now (#3446), so you could upgrade to 0.11.6dev (see TracDownload#Tracstable).

But are you saying that you have a persistent lock, i.e. that the database stays locked, even after a restart of the server?

in reply to: 32

匿名用户, 11年 ago

Replying to cboos:

The database is locked issue is nearly always a transient issue.

If so, the problem is believed to be fixed now (#3446), so you could upgrade to 0.11.6dev (see TracDownload#Tracstable).

But are you saying that you have a persistent lock, i.e. that the database stays locked, even after a restart of the server?

Thank you for replying.

It was a transient issue and now it work well again.but I will upgrade the trac

when possible.

I have got this error running Trac 0.12multirepos-r8178 on a virtual machine (Virtualbox 3.1.4) where the trac-db was mounted (cifs) from a machine running Vista.

Solved the problem by copying the db into the virtual-machine's filesystem.

Today we had this issue on TRAC 0.11.7. It wasn't permanent though.

4 years and counting. This bug is so annoying.

Why not just work around it, by quietly re-trying the request??

跟随：

39

Christian Boos, 10年 ago

Well, this exception has become very … exceptional, these days. If you're seeing that frequently, then I'm pretty confident you're not running a recent version of Trac, PySqlite or SQLite.

in reply to: 34

匿名用户, 10年 ago

Replying to patrick232@…:

I have got this error running Trac 0.12multirepos-r8178 on a virtual machine (Virtualbox 3.1.4) where the trac-db was mounted (cifs) from a machine running Vista.

Solved the problem by copying the db into the virtual-machine's filesystem.

it appears using nobrl option in mount.cifs solves the problem.

in reply to: 37

alexander.papaspyrou@…, 10年 ago

Replying to cboos:

Well, this exception has become very … exceptional, these days. If you're seeing that frequently, then I'm pretty confident you're not running a recent version of Trac, PySqlite or SQLite.

Hm, not sure. I am using quite recent versions (over apache2 and mod_python):

[ ~ ]

alexp@forge:0$ dpkg -l | grep trac

ii trac 0.11.7-1 Enhanced wiki and issue tracking system for

[ ~ ]

alexp@forge:0$ cat /etc/lsb-release

DISTRIB_ID=Ubuntu

DISTRIB_RELEASE=10.04

DISTRIB_CODENAME=lucid

DISTRIB_DESCRIPTION="Ubuntu 10.04.1 LTS"

and

Python 2.6.5 (r265:79063, Apr 16 2010, 13:09:56)

[GCC 4.4.3] on linux2

Type "help", "copyright", "credits" or "license" for more information.

>>> import trac.db.sqlite_backend as test

>>> test._ver

(3, 6, 22)

>>> test.have_pysqlite

2

>>> test.sqlite.version

'2.5.5'

>>>

Accessing the login page constantly (even over apache2 restarts)

Traceback (most recent call last):

File "/usr/lib/python2.6/dist-packages/trac/web/api.py", line 376, in send_error

'text/html')

File "/usr/lib/python2.6/dist-packages/trac/web/chrome.py", line 733, in render_template

message = req.session.pop('chrome.%s.%d' % (type_, i))

File "/usr/lib/python2.6/dist-packages/trac/web/api.py", line 195, in __getattr__

value = self.callbacks[name](self)

File "/usr/lib/python2.6/dist-packages/trac/web/main.py", line 265, in _get_session

return Session(self.env, req)

File "/usr/lib/python2.6/dist-packages/trac/web/session.py", line 161, in __init__

self.promote_session(sid)

File "/usr/lib/python2.6/dist-packages/trac/web/session.py", line 248, in promote_session

db.commit()

OperationalError: database is locked

Same in the logs. The only thing that comes up to my mind is that the trac environments live on an NFS share…

Thoughts?

-Alexander

It still occur every day! even after upraded to latest version(Trac 0.13dev-r10406) :(

2011-01-06 17:35:09,572 Trac[main] ERROR: Internal Server Error:

Traceback (most recent call last):

File "build\bdist.win32\egg\trac\web\main.py", line 447, in _dispatch_request

dispatcher.dispatch(req)

File "build\bdist.win32\egg\trac\web\main.py", line 201, in dispatch

req.session.save()

File "build\bdist.win32\egg\trac\web\session.py", line 140, in save

""", (mintime,))

File "build\bdist.win32\egg\trac\db\api.py", line 140, in __exit__

self.db.commit()

OperationalError: database is locked

why not keep session in memory? main trac use enviroment is small team, I think.

处理结果:

fixed

状态:

closed → reopened

里程碑:

0.10.1 → 0.13

It still occur every day! even after upraded to latest version(Trac 0.13dev-r10406) :(

is it a mission impossible ?

in reply to: 41

Christian Boos, 10年 ago

Replying to anonymous:

It still occur every day! even after upraded to latest version(Trac 0.13dev-r10406) :(

is it a mission impossible ?

It all depends on you… and whether you're responsive to our requests for feedback or not. Let's see.

Please give us at least the following information:

version information for Python, the PySqlite bindings and SQLite itself

which platform, which web front-end (+ version information when relevant)

on what kind of filesystem is the /db/trac.db file located? Same question for /tmp (if you're on Unix)?

are you using any Trac plugins? does the problem also happen when you try without any plugin enabled?

And also:

how many users? what is the estimate of the number of concurrent requests when a lock happens? (you can add timing information in the Trac log, if you can't gather this information from the web server logs, see TracLogging)

post a typical backtrace

does the problem happens for any kind of request, or just some actions?

do you have a reproduction scenario?

Christian Boos, 10年 ago

处理结果:

→ fixed

状态:

reopened → closed

… right.

We started seeing this problem recently, once we moved TRAC from a Netapp to a Hitachi SAN. That was literally the only difference.

Hitachi deals with file locks in a very different way then Netapp, which the Trac db doesn't seem to like.

If you can properly lock the file on writes, or leave it on local disk, you should not run into these problems any further.

in reply to: 41

Christian Boos, 8年 ago

里程碑:

1.0 → 0.10.1

restoring original milestone