Copy link
Collaborator
Yep, I can also reproduce on Ubuntu with master.
Copy link
Collaborator
I'll try regenerating the metadata first as a quick fix.
Copy link
Collaborator
Right, that worked, I'll have a look at the old files and try to work out what went wrong. I have a suspicion it's my fault and the staging LVFS accidentally uploaded metadata with the test key.
Copy link
Collaborator
Nope, it appears the right key was used:
$ gpg2 --verify firmware.xml.gz.asc firmware.xml.gz
gpg: Signature made Wed 07 Feb 2018 12:47:04 GMT
gpg: using RSA key 48A6D80E4538BAC2
gpg: BAD signature from "Linux Vendor Firmware Service " [unknown]
(I have the public key imported system wide on this machine)
Copy link
Contributor
Author
Thanks for the fix, that was impressively fast! I’ll leave it up to you to decide when to close the issue.
Copy link
Collaborator
In the event log:
2018-02-07 12:47:03 | ???.???.???.??? | dellqa | Uploaded file 05e0d0906098a57c4d0aa71d5f5fc4e620e7ee03-Signed_1152921504627420111.cab to stable /lvfs/upload
...which is what triggered the metadata rebuild (correctly).
Copy link
Collaborator
And nothing untoward in the uwsgi logs:
[pid: 28217|app: 0|req: 1175/1175] ???.???.???.??? () {52 vars in 1302 bytes} [Wed Feb 7 12:47:00 2018] POST /lvfs/upload => generated 317 bytes in 10536 msecs (HTTP/1.1 302)
4 headers in 526 bytes (2 switches on core 0)
uploading /var/www/lvfs/downloads/firmware-testing.xml.gz as downloads/firmware-testing.xml.gz
uploading /var/www/lvfs/downloads/firmware-testing.xml.gz.asc as downloads/firmware-testing.xml.gz.asc
uploading /var/www/lvfs/downloads/firmware.xml.gz as downloads/firmware.xml.gz
uploading /var/www/lvfs/downloads/firmware.xml.gz.asc as downloads/firmware.xml.gz.asc
Copy link
Collaborator
$ sha1sum firmware.xml.gz*
6443eac9f838c6b12edaaf2fcd56cf140b1e5309 firmware.xml.gz
3268b224447fa12b5dcd95dc8f9c1be59ecf5fe9 firmware.xml.gz.1
e817446b3a96aadd1ad9c5a4de72d20a1f2db1d7 firmware.xml.gz.asc
..where the .1 is after the rebuild.
Copy link
Collaborator
Ohh, that's expected, other firmwares got pushed to stable in the meantime (new are appearing... but that should have synced the cdn too -- but there is nothing in the logs.
Copy link
Collaborator
I can't work this one out -- I'll close and we can reopen if this every happens again.
Please reopen, it's happening again. Since yesterday I get "Failed to update metadata for lvfs: '48A6D80E4538BAC2' is not a valid signature" on two machines running Fedora 27.
👍
3
It's gone again. If nobody changed anything, it seems to be something intermittent...
reopening due to:
fwupdmgr get-updates -v 16:19:47 ✘ 1
Firmware metadata has not been updated for 30 days and may not be up to date.
Update now? (Requires internet connection) [y|N]: y
(fwupdmgr:15127): Fu-DEBUG: creating path /root/.cache/fwupdmgr
(fwupdmgr:15127): Fwupd-DEBUG: looking for os-release at /etc/os-release
(fwupdmgr:15127): Fu-DEBUG: downloading https://cdn.fwupd.org/downloads/firmware.xml.gz to /root/.cache/fwupdmgr/lvfs-metadata.xml.gz
Fetching metadata https://cdn.fwupd.org/downloads/firmware.xml.gz
(fwupdmgr:15127): Fu-DEBUG: progress: 26%
Downloading… [********** ](fwupdmgr:15127): Fu-DEBUG: progress: 54%
Downloading… [********************* ](fwupdmgr:15127): Fu-DEBUG: progress: 81%
Downloading… [******************************* ](fwupdmgr:15127): Fu-DEBUG: progress: 100%
Downloading… [***************************************]
(fwupdmgr:15127): Fu-DEBUG: downloading https://cdn.fwupd.org/downloads/firmware.xml.gz.asc to /root/.cache/fwupdmgr/lvfs-metadata.xml.gz.asc
Fetching signature https://cdn.fwupd.org/downloads/firmware.xml.gz.asc
(fwupdmgr:15127): Fu-DEBUG: progress: 100%
Failed to update metadata for lvfs: '48A6D80E4538BAC2' is not a valid signature
Copy link
Collaborator
@brumle80 can you upload /root/.cache/fwupdmgr/firmware.xml.gz and /root/.cache/fwupdmgr/firmware.xml.gz to this ticket please.
Copy link
Collaborator
I managed to find someone else having the same problem. This is what I see:
$ shasum /home/hughsie/.cache/fwupdmgr/lvfs-metadata.xml.gz*
f14067a80825e321ab36c9c0063e1766d4f78e30 /home/hughsie/.cache/fwupdmgr/lvfs-metadata.xml.gz
e1a2f0b2402c6e9dc9b3b217366471a362460c4b /home/hughsie/.cache/fwupdmgr/lvfs-metadata.xml.gz.asc
$ gpg /home/hughsie/.cache/fwupdmgr/lvfs-metadata.xml.gz.asc
gpg: assuming signed data in `/home/hughsie/.cache/fwupdmgr/lvfs-metadata.xml.gz'
gpg: Signature made Tue 06 Mar 2018 09:06:39 GMT using RSA key ID 4538BAC2
gpg: Good signature from "Linux Vendor Firmware Service "
Now, the other person provided me with files that were downloaded from the CDN just a few minutes ago:
shasum /home/hughsie/Downloads/metadata.xml.gz*
f14067a80825e321ab36c9c0063e1766d4f78e30 /home/hughsie/Downloads/metadata.xml.gz
e0d1a5cbb1fa4dd0a7f9e79505b1b9bb57d50d28 /home/hughsie/Downloads/metadata.xml.gz.asc
Hmm, so the detached signature is different.
$ gpg /home/hughsie/Downloads/metadata.xml.gz.asc
gpg: assuming signed data in `/home/hughsie/Downloads/metadata.xml.gz'
gpg: Signature made Wed 28 Feb 2018 20:07:37 GMT using RSA key ID 4538BAC2
gpg: BAD signature from "Linux Vendor Firmware Service "
A file generated on the 28th Feb?!
Copy link
Collaborator
Some followup; it seems the CDN is shipping old files for some zones. I've purged all zones manually, and it should "fix" it, although we need some kind of root cause analysis and a plan to detect this again automatically in the future.
👍
2
the purge fixed it as it's working fine now, thanks for the quick resolve.
Copy link
Still/again Getting the same
(fwupdmgr:2051): Fu-DEBUG: 16:46:58.763: creating path /root/.cache/fwupdmgr
(fwupdmgr:2051): Fwupd-DEBUG: 16:46:58.765: looking for os-release at /etc/os-release
(fwupdmgr:2051): Fu-DEBUG: 16:46:58.765: downloading https://cdn.fwupd.org/downloads/firmware.xml.gz to /root/.cache/fwupdmgr/lvfs-metadata.xml.gz
Fetching metadata https://cdn.fwupd.org/downloads/firmware.xml.gz
(fwupdmgr:2051): Fu-DEBUG: 16:46:59.386: progress: 24%
Downloading… [********* ](fwupdmgr:2051): Fu-DEBUG: 16:46:59.386: progress: 50%
Downloading… [******************* ](fwupdmgr:2051): Fu-DEBUG: 16:46:59.444: progress: 76%
Downloading… [***************************** ](fwupdmgr:2051): Fu-DEBUG: 16:46:59.444: progress: 100%
Downloading… [***************************************]
(fwupdmgr:2051): Fu-DEBUG: 16:46:59.449: downloading https://cdn.fwupd.org/downloads/firmware.xml.gz.asc to /root/.cache/fwupdmgr/lvfs-metadata.xml.gz.asc
Fetching signature https://cdn.fwupd.org/downloads/firmware.xml.gz.asc
(fwupdmgr:2051): Fu-DEBUG: 16:46:59.513: progress: 100%
Failed to update metadata for lvfs: '48A6D80E4538BAC2' is not a valid signature
Copy link
Same here
Copy link
Collaborator
I've purged manually; which seems to fix things for me too, but we really need a way to purge triggered from the LVFS.
Copy link
Again now:
> fwupdmgr refresh
Fetching metadata https://cdn.fwupd.org/downloads/firmware.xml.gz
Downloading… [***************************************]
Fetching signature https://cdn.fwupd.org/downloads/firmware.xml.gz.asc
Failed to update metadata for lvfs: '48A6D80E4538BAC2' is not a valid signature
> shasum ~/.cache/fwupdmgr/lvfs-metadata.xml.gz*
d4a73a0924d9bed830ddff85f748dbe6bb3a3473 fwupdmgr/lvfs-metadata.xml.gz
be5d50b6fd62ed59edd0e10396a4f7499b56dfa8 fwupdmgr/lvfs-metadata.xml.gz.asc
It should refresh CDN automatically
👍
2
Copy link
Collaborator
Different error this time; a permissions error on the server. I think I've fixed up everything now, but I'll do a root cause after breakfast.
👍
1
I've run into the same problem today:
fwupdmgr refresh
Fetching metadata https://cdn.fwupd.org/downloads/firmware.xml.gz
Downloading… [***************************************]
Fetching signature https://cdn.fwupd.org/downloads/firmware.xml.gz.asc
Failed to update metadata for lvfs: '48A6D80E4538BAC2' is not a valid signature
Copy link
Collaborator
Works now, right? I broke the metadata sync when converting everything to Python 3.
Copy link
@hughsie ran it a few more times and still the same thing, I'm guessing it's the cdn cache.
48 hidden items
Load more…
hughsie
added a commit
that referenced
this issue
Apr 3, 2020
Switch to downloading the signature first, which we can then load to get the
suffixed build-specific URL of the actual metadata file. You need to have
libjcat 0.1.1 installed and fwupd built against the new version for this to
work.
Fixes #391
hughsie
added a commit
to fwupd/lvfs-website-archived
that referenced
this issue
Apr 3, 2020
When a fwupd client wants to refresh the metadata it firsts download the tiny
signature (typically a JCat file) and then if that has changed compared to the
existing signature on disk the much larged gzipped XML metadata catalog is then
downloaded. This saves a lot of mirror bandwidth as some remotes do not change
very much, and even stable only regenerates 4 times a day.
The drawback of this is that we have two things that need to be used to verify
a signature, and one of four scenarios could happen:
1. The same version of sig and catalog are delivered by the CDN
2. Between getting the sig the remote is resigned and the CDN is invalidated
3. The CDN sends an older cached version of the sig and the current catalog
4. The CDN sends the current sig and a cached older version of the catalog
Over the last few years the first scenario happens 99.999999% of the time, but
0.000001 * millions of daily downloads means that the bug tracking the CDN issue
gets "me too"'d almost every week.
To fix the 3 failure scenarios, do three things:
* Write each metadata catalog as a new file with a unique URL for the CDN
* Write the name of the unique file in the JCat file so the client knows what
to download with an alias for the old name for our future selves.
* Delete old metadata catalogs, leaving the last 5 builds.
In scenarios 2 and 3 we just download the n-1 metadata catalog, which is fine,
and scenario 4 is now impossible as the URLs are now different.
Fixes the server side of fwupd/fwupd#391
hughsie
added a commit
to fwupd/lvfs-website-archived
that referenced
this issue
Apr 3, 2020
When a fwupd client wants to refresh the metadata it firsts download the tiny
signature (typically a JCat file) and then if that has changed compared to the
existing signature on disk the much larged gzipped XML metadata catalog is then
downloaded. This saves a lot of mirror bandwidth as some remotes do not change
very much, and even stable only regenerates 4 times a day.
The drawback of this is that we have two things that need to be used to verify
a signature, and one of four scenarios could happen:
1. The same version of sig and catalog are delivered by the CDN
2. Between getting the sig the remote is resigned and the CDN is invalidated
3. The CDN sends an older cached version of the sig and the current catalog
4. The CDN sends the current sig and a cached older version of the catalog
Over the last few years the first scenario happens 99.999999% of the time, but
0.000001 * millions of daily downloads means that the bug tracking the CDN issue
gets "me too"'d almost every week.
To fix the 3 failure scenarios, do three things:
* Write each metadata catalog as a new file with a unique URL for the CDN
* Write the name of the unique file in the JCat file so the client knows what
to download with an alias for the old name for our future selves.
* Delete old metadata catalogs, leaving the last 5 builds.
In scenarios 2 and 3 we just download the n-1 metadata catalog, which is fine,
and scenario 4 is now impossible as the URLs are now different.
Fixes the server side of fwupd/fwupd#391
hughsie
added a commit
that referenced
this issue
Apr 3, 2020
Switch to downloading the signature first, which we can then load to get the
suffixed build-specific URL of the actual metadata file. You need to have
libjcat 0.1.1 installed and fwupd built against the new version for this to
work.
Fixes #391
hughsie
added a commit
to fwupd/lvfs-website-archived
that referenced
this issue
Apr 3, 2020
When a fwupd client wants to refresh the metadata it firsts download the tiny
signature (typically a JCat file) and then if that has changed compared to the
existing signature on disk the much larged gzipped XML metadata catalog is then
downloaded. This saves a lot of mirror bandwidth as some remotes do not change
very much, and even stable only regenerates 6 times a day.
The drawback of this is that we have two things that need to be used to verify
a signature, and one of four scenarios could happen:
1. The same version of sig and catalog are delivered by the CDN
2. Between getting the sig the remote is resigned and the CDN is invalidated
3. The CDN sends an older cached version of the sig and the current catalog
4. The CDN sends the current sig and a cached older version of the catalog
Over the last few years the first scenario happens 99.999999% of the time, but
0.000001 * millions of daily downloads means that the bug tracking the CDN issue
gets "me too"'d almost every week.
To fix the 3 failure scenarios, do three things:
* Write each metadata catalog as a new file with a unique URL for the CDN
* Write the name of the unique file in the JCat file so the client knows what
to download with an alias for the old name for our future selves.
* Delete old metadata catalogs, leaving the last 6 builds.
In scenarios 2 and 3 we just download the n-1 metadata catalog, which is fine,
and scenario 4 is now impossible as the URLs are now different.
Fixes the server side of fwupd/fwupd#391
hughsie
added a commit
that referenced
this issue
Apr 3, 2020
Switch to downloading the signature first, which we can then load to get the
suffixed build-specific URL of the actual metadata file. You need to have
libjcat 0.1.1 installed and fwupd built against the new version for this to
work.
Fixes #391
hughsie
added a commit
to fwupd/lvfs-website-archived
that referenced
this issue
Apr 3, 2020
When a fwupd client wants to refresh the metadata it firsts download the tiny
signature (typically a JCat file) and then if that has changed compared to the
existing signature on disk the much larged gzipped XML metadata catalog is then
downloaded. This saves a lot of mirror bandwidth as some remotes do not change
very much, and even stable only regenerates 6 times a day.
The drawback of this is that we have two things that need to be used to verify
a signature, and one of four scenarios could happen:
1. The same version of sig and catalog are delivered by the CDN
2. Between getting the sig the remote is resigned and the CDN is invalidated
3. The CDN sends an older cached version of the sig and the current catalog
4. The CDN sends the current sig and a cached older version of the catalog
Over the last few years the first scenario happens 99.999999% of the time, but
0.000001 * millions of daily downloads means that the bug tracking the CDN issue
gets "me too"'d almost every week.
To fix the 3 failure scenarios, do three things:
* Write each metadata catalog as a new file with a unique URL for the CDN
* Write the name of the unique file in the JCat file so the client knows what
to download with an alias for the old name for our future selves.
* Delete old metadata catalogs, leaving the last 6 builds.
In scenarios 2 and 3 we just download the n-1 metadata catalog, which is fine,
and scenario 4 is now impossible as the URLs are now different.
Fixes the server side of fwupd/fwupd#391
hughsie
added a commit
to fwupd/lvfs-website-archived
that referenced
this issue
Apr 3, 2020
When a fwupd client wants to refresh the metadata it firsts download the tiny
signature (typically a JCat file) and then if that has changed compared to the
existing signature on disk the much larged gzipped XML metadata catalog is then
downloaded. This saves a lot of mirror bandwidth as some remotes do not change
very much, and even stable only regenerates 6 times a day.
The drawback of this is that we have two things that need to be used to verify
a signature, and one of four scenarios could happen:
1. The same version of sig and catalog are delivered by the CDN
2. Between getting the sig the remote is resigned and the CDN is invalidated
3. The CDN sends an older cached version of the sig and the current catalog
4. The CDN sends the current sig and a cached older version of the catalog
Over the last few years the first scenario happens 99.999999% of the time, but
0.000001 * millions of daily downloads means that the bug tracking the CDN issue
gets "me too"'d almost every week.
To fix the 3 failure scenarios, do three things:
* Write each metadata catalog as a new file with a unique URL for the CDN
* Write the name of the unique file in the JCat file so the client knows what
to download with an alias for the old name for our future selves.
* Delete old metadata catalogs, leaving the last 6 builds.
In scenarios 2 and 3 we just download the n-1 metadata catalog, which is fine,
and scenario 4 is now impossible as the URLs are now different.
Fixes the server side of fwupd/fwupd#391
Copy link
Collaborator
If you want to test all this out, you currently need to install both libjcat and fwupd from git master, although new tarballs won't be much further away.
I am running into this on ubuntu 20.04
Copy link
Collaborator
The fix is in the fwupd 1.4.x branch, which is not present in Ubuntu 20.04
Is the 1.4 branch in the Flatpak version? It seems not to be.
Copy link
Collaborator
Is the 1.4 branch in the Flatpak version
I'm just building 1.4.4 in flathub now.
Copy link
Ubuntu 20.04. Same issue.
Copy link
Failed to update metadata for lvfs: '48A6D80E4538BAC2' is not a valid signature
Ubuntu Server 20.04. Same issue.
Copy link
Jumping on the bandwagon
jojojo@JWHISKEYE:~$ sudo fwupdmgr refresh -v
(fwupdmgr:4344): FuCommon-DEBUG: 23:38:41.144: creating path /home/jojojo/.cache/fwupd
(fwupdmgr:4344): Fwupd-DEBUG: 23:38:41.147: looking for os-release at /etc/os-release
(fwupdmgr:4344): FuMain-DEBUG: 23:38:41.147: downloading https://cdn.fwupd.org/downloads/firmware.xml.gz to /home/jojojo/.cache/fwupd/lvfs-metadata.xml.gz
Fetching metadata https://cdn.fwupd.org/downloads/firmware.xml.gz
(fwupdmgr:4344): FuMain-DEBUG: 23:38:41.333: progress: 0%
Downloading… [- ](fwupdmgr:4344): FuMain-DEBUG: 23:38:41.333: progress: 0%
Downloading… [ - ](fwupdmgr:4344): FuMain-DEBUG: 23:38:41.333: progress: 0%
(fwupdmgr:4344): FuMain-DEBUG: 23:38:41.335: progress: 1%
Downloading… [ ](fwupdmgr:4344): FuMain-DEBUG: 23:38:41.336: progress: 1%
[...]
(fwupdmgr:4344): FuMain-DEBUG: 23:38:41.413: downloading https://cdn.fwupd.org/downloads/firmware.xml.gz.asc to /home/jojojo/.cache/fwupd/lvfs-metadata.xml.gz.asc
Fetching signature https://cdn.fwupd.org/downloads/firmware.xml.gz.asc
(fwupdmgr:4344): FuMain-DEBUG: 23:38:41.429: progress: 100%
Failed to update metadata for lvfs: '48A6D80E4538BAC2' is not a valid signature
When trying get-updates:
jojojo@JWHISKEYE:~$ fwupdmgr get-updates -v
No releases found for device: no HWIDs matched b31aa89d-cfb8-5c67-8502-ce76a9e7b580
Additionally: not sure if this is the right place for this; and I'm kind of a noob with linux in general still so maybe I'm just dumb, but I may have some kind of malware on my machine at the moment, which I have no idea if it has anything to do with this, but I suspect it poisons the DNS cache somehow (have had problems with all kinds of websites in a browser, some sketchy certificate issues)... for what it's worth. Highly doubt fwupd is a likely cause of that though, if anything merely a symptom - I'm not completely sure if I have anything malicious at all at this point (been trying to diagnose for months) but if I do, it's very deep in the UEFI or something, and started in Windows soo. But in case anyone else sees this who has been wondering the same thing on their machines.
Copy link
Getting the same error
Fetching metadata https://cdn.fwupd.org/downloads/firmware.xml.gz
Downloading… [***************************************]
Fetching signature https://cdn.fwupd.org/downloads/firmware.xml.gz.asc
Failed to update metadata for lvfs: '48A6D80E4538BAC2' is not a valid signature
What was the fix for this?
Thank you
On Centos 8
Copy link
Same error on Elitebook 845 G7.
Fetching metadata https://cdn.fwupd.org/downloads/firmware-testing.xml.gz
Downloading… [***************************************]
Fetching signature https://cdn.fwupd.org/downloads/firmware-testing.xml.gz.asc
Failed to update metadata for lvfs-testing: '48A6D80E4538BAC2' is not a valid signature
Same error on recent ubuntu 20.04.1 server install:
/usr/bin/fwupdmgr refresh --no-metadata-check
Fetching metadata https://cdn.fwupd.org/downloads/firmware.xml.gz
Downloading… [***************************************]
Fetching signature https://cdn.fwupd.org/downloads/firmware.xml.gz.asc
Failed to update metadata for lvfs: '48A6D80E4538BAC2' is not a valid signature
It is happening again currently:
sudo fwupdmgr refresh
[sudo] password for mapa:
Fetching metadata https://cdn.fwupd.org/downloads/firmware.xml.gz
Downloading… [***************************************]
Fetching signature https://cdn.fwupd.org/downloads/firmware.xml.gz.asc
Failed to update metadata for lvfs: '48A6D80E4538BAC2' is not a valid signature
As of now works again, that was fast! Thanks!
I am currently getting the same error reported here, as of the time of this comment:
$ sudo fwupdmgr refresh
Fetching metadata https://cdn.fwupd.org/downloads/firmware.xml.gz
Downloading… [***************************************]
Fetching signature https://cdn.fwupd.org/downloads/firmware.xml.gz.asc
Failed to update metadata for lvfs: '48A6D80E4538BAC2' is not a valid signature
I got this regularly across servers for weeks. apt purge fwupd solved it - I don't know about the wisdom of including this by default in Ubuntu but if your servers don't support/need automatic firmware updates then just disable it permanently.
Copy link
Collaborator
The fix to this was introduced in 1.4.x with switching to jcat based metadata.
Unfortunately that means that 1.3.x and earlier will continue to encounter it from time to time due to cdn mirroring delays.
If you continue to encounter it on 1.3.x and earlier please try again in a few minutes.
fwupd
locked and limited conversation to collaborators
Jan 8, 2021
扫一扫
489
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
