How to set up systemd-resolved + dnssec (by quqi99)

作者：张华 发表于：2020-02-25
版权声明：可以任意转载，转载时请务必以超链接形式标明文章原始出处和作者信息及本版权声明

Set up test env

#first let us know 8.8.8.8 supports dnssec, while 114.114.114.114 doesn't support
dig paypal.com +dnssec @8.8.8.8 |grep RRSIG
dig paypal.com +dnssec @114.114.114.114 |grep RRSIG

#set up dnsmasq as upstream dns server
IFACE=dummy0
SUBNET=10.0.0
sudo ip link add $IFACE type dummy
sudo ifconfig $IFACE ${SUBNET}.1/24
#https://www.cnblogs.com/taoyuxuan/p/11205491.html
#sudo dnsmasq -h -R -d -C /dev/null -z -i $IFACE -I lo --host-record=test.test,${SUBNET}.1 -2 $IFACE &
#sudo dnsmasq -h -R -d -C /dev/null -z -i $IFACE -I lo -S /test/ --host-record=test.test,${SUBNET}.1 -2 $IFACE &

#sudo dnsmasq --no-hosts --no-resolv --no-daemon --no-dhcp-interface==$IFACE --bind-interfaces --interface=$IFACE \
    --except-interface=lo --server=/test/ --host-record=test.test,${SUBNET}.1 --conf-file=/dev/null --log-queries &
sudo dnsmasq --no-hosts --no-resolv --no-daemon --no-dhcp-interface==$IFACE --bind-interfaces --interface=$IFACE \
    --except-interface=lo --server=/test/10.0.0.1 --host-record=test.test,${SUBNET}.1 --log-queries \
    --dnssec --conf-file=/usr/share/dnsmasq-base/trust-anchors.conf --dnssec-check-unsigned --dnssec-debug --server=8.8.8.8 &
dig paypal.com +dnssec @10.0.0.1 |grep RRSIG

#test dns server with dig
ubuntu@bastion:~$ dig -t a test.test @10.0.0.1 | grep EDNS
; EDNS: version: 0, flags:; udp: 4096
ubuntu@bastion:~$ dig -t aaaa test.test @10.0.0.1 | grep EDNS
# again, should return "; EDNS ..." but doesn't

#configure systemd-resolved to look at only 10.0.0.1
ubuntu@bastion:~$ grep -Ev '#|^$' /etc/systemd/resolved.conf
[Resolve]
DNS=10.0.0.1
DNSSEC=yes
Cache=no
ubuntu@bastion:~$ grep -Ev '#|^$'  /etc/resolv.conf 
nameserver 127.0.0.53
options edns0

#debug with systemd-resolved
sudo systemctl disable systemd-resolved   #seems systemd-resolved always restart if not disable it as well
sudo systemctl stop systemd-resolved
sudo SYSTEMD_LOG_LEVEL=debug script -c /lib/systemd/systemd-resolved /tmp/debug.txt
systemd-resolve --status --no-pager |egrep 'DNSSEC|DNS Servers'
#In systemd 239 systemd-resolve has been renamed to resolvectl
resolvectl status
#repeat the following 3 lines to test every time - ./src/resolve/resolved-dns-server.c
sudo systemd-resolve --flush-caches; sudo systemd-resolve --reset-server-features; systemd-resolve test.test -t A
sudo systemd-resolve --flush-caches; sudo systemd-resolve --reset-server-features; systemd-resolve paypal.com -t A
systemd-resolve test.test -t A   #don't reset, try directly
systemd-resolve paypal.com -t A
systemd-resolve example.com ubuntu.com us.archive.ubuntu.com packages.icinga.com mirror.steadfast.net archive.ubuntu.com ppa.launchpad.net private-ppa.launchpad.net sigok.verteiltesysteme.net sigfail.verteiltesysteme.net verisign.com www.verisign.com

一些术语

• LLMNR: 当dns服务不可用时, 支持使用组播dns协议(mDNS)继续在子网内局部查询dns直至网络连接还原为止.
• Knot-DNS: 类似于dnsmasq, bind9的dns server
• EDNS:允许将DNS消息大小从标准的512字节扩展（当UDP用作传输协议时），而无需切换到TCP. 它能提升访问CDN的性能，因为：标准DNS协议只能传递域名参数查IP,　而不能传递域名+用户参数来源查IP．查询下游的CDN, DNS服务器应该是根据用户的来源查询,而不是将上游DNS服务做为来源查询.由于国内的运营商之间互联的带宽是很低的,　所以造成访问速度超慢.
• EDNS0:可以将UDP包从512字节扩展到4096字节从而可以容纳DNSEC包, systemd-resolved通过/etc/resovled.conf里的’options edns0’通知dns server可以edns0, server再回的时候会将其+1, 所以: 雖然是EDNS0 封包，可是不代表這是 DNSSEC 封包，在EDNS0 中有一個 bit: DO 被設定為 1 的時候，才代表這是 DNSSEC 封包 (通过: dig +dnssec example.com ns |grep EDNS 命令可以查看)

Some bugs

1, https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1857639
upstream dns能力被检测正确让dnssec工作(现在一旦突然检查错了, resovle就停止work了), 或者允许disable能力检查
https://github.com/systemd/systemd/issues/6490
https://github.com/systemd/systemd/issues/14435
https://github.com/systemd/systemd/pull/8849
a, 第一种解决方案
resolve不能看到timeout or error了就降级(tcp->dns over tls -> edns0)声称不支持某个特性, 它的特性应该独立于实际查询
https://github.com/systemd/systemd/issues/9384
b, 允许disable dnsec能力检查
https://github.com/systemd/systemd/issues/14435
c, 从resolved中移除dnssec

如有些咖啡厅等公共网络不支持ends0查询，会响应domain not found, 即收到＇NXDOMAIN when queried with EDNS0＇消息时(the query is with EDNS0 enabled and D0 bit set to zero)需要继续只根据EDNS0继续重试
https://github.com/systemd/systemd/pull/8608
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1727237
https://github.com/systemd/systemd/commit/7abcaa93ab13b1a7b11d50fcd43938de44687970 - 代码
journalctl -b -u systemd-resolve | grep DVE-2018 - 确诊
Workaround:
sudo systemctl disable systemd-resolved.service
sudo service systemd-resolved stop
sudo rm /etc/resolv.conf
sudo nano /etc/NetworkManager/NetworkManager.conf
then add “dns=default” under [main]
sudo service network-manager restart

上面的降级（resolved-Mitigate-DVE-2018-0001-by-retrying-NXDOMAIN-with.patch）应该只在DNSSEC=allow-downgrade or DNSSEC=no时运行．在真实世界中，NetworkManager’s在无法降级时会告诉用户关掉dnssec
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1796501

2, https://github.com/systemd/systemd/pull/9264/commits/844c58fb27b4420f5f8b07070fe2319af6537ae3
systemd-resolved只是一个stub resovler, 它只是转发dns请求到upstream dns server并等待请求(等待时间为RTT)，它没有考虑到上游递归dns服务器(interative resolver, eg: dnsmasq)的query timeout(authoritative server是用内存来存储dns记录所以它快，但也不是所有的都快），这样systemd-resolved会经常错误的降级（Therefore it often mistakenly degrades the feature set of its　upstream resolvers if it takes them longer than usual to answer a query），特别当DNSSEC=yes不充许降级时就更严重，所以systemd-resolved应该使用RTT+上游的查询时间，而不是只用RTT (DNS_TIMEOUT_MAX_USEC=DNS_SERVER_FEATURE_RETRY_ATTEMPTS * DNS_TIMEOUT_MAX_USEC = 15s），当然，这也不是解决根本问题，但可以缓解．

3, https://github.com/systemd/systemd/commit/7abcaa93ab13b1a7b11d50fcd43938de44687970
https://github.com/systemd/systemd/issues/11171
https://github.com/systemd/systemd/issues/4315 - https://github.com/poettering/systemd/commit/fc50d3f20d56862520635c3703fd90725b288afb
https://github.com/systemd/systemd/pull/8608
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1727237
这个commit (7abcaa93ab - debian/patches/resolved-Mitigate-DVE-2018-0001-by-retrying-NXDOMAIN-with.patch)可以解决上面关于降级的问题，但是它又引入了回归bug, 没DNSSEC=yes时没有做安全检查 (为DNSSEC=yes时不从UDP+ends0(DNS_SERVER_FEATURE_LEVEL_EDNS0)降级互UDP(DNS_SERVER_FEATURE_LEVEL_UDP)但upstream还未接受patch，故先在debian/patches/resolved_disable-connection-downgrade-when-DNSSEC-yes.patch

4, https://github.com/dns-violations/dns-violations/issues/62
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1785383
dnsmasq 2.79当返回空时省略了EDNS0 OPT记录 (例如访问无AAAA记录时 - patch - 1682d15a744880b0398af75eadf68fe66128af78)) , systemd-resolved能hang (但只是使用了dnsmasq的router会如此)

20230428 - using dnsmasq instead of systemd-resolve

hua@x1:~$ cat /etc/systemd/resolved.conf
[Resolve]
DNS=127.0.0.1
Cache=no
DNSStubListener=no

hua@x1:~$ cat /etc/resolv.conf 
nameserver 127.0.0.1

hua@x1:~$ cat /etc/dnsmasq.conf 
listen-address=127.0.0.1
conf-file=/etc/dnsmasq.conf
conf-dir=/etc/dnsmasq.d
port=53
#cache-size=0
cache-size=20000
server=223.5.5.5
addn-hosts=/etc/hosts

#for white list in /etc/dnsmasq.d
server=/api.charmhub.io/127.0.0.1#53
ipset=/api.charmhub.io/whitelist
#for black list in /etc/dnsmasq.d
server=/xxx.com/127.0.0.1#5335
ipset=/xxx.com/gfwlist

# then updatedns.sh is not required now
dev tun
proto tcp
tun-mtu 1492
verb 3
client
remote-cert-tls server
remote xxx 1194
cipher AES-256-CBC
max-routes 999
comp-lzo
script-security 2
#up /home/ubuntu/updatedns.sh
#down /home/ubuntu/restoredns.sh
<ca>
xxx
</ca>
<cert>
</cert>
<key>
</key>