OpenHarmony移植OpenSSH-9.6p1

本文主要介绍如何在OpenHarmony系统下通过ndk工具移植OpenSSH-9.6p1。

安装NDK工具

未编译过ohos-sdk的话，先执行下面的命令编译sdk：

./build.sh --product-name ohos-sdk --ccache

编译好ohos-sdk之后，可以直接从编译路径下拷贝到指定路径，当然也可以直接将out目录下的ohos sdk的native路径作为NDK的路径。如果下载的public-sdk，也可以从压缩包中解压native的压缩包（例如：native-linux-x64-4.0.10.13-Release.zip）到指令路径。这里以我们自行编译的full-sdk下的native拷贝为例（不同的版本生成路径可能会有差异）：

mkdir -p /opt/sdk/ohos/native/
cp -r ${OHOS_SRC}/out/sdk/packages/ohos-sdk/linux/10/native/ /opt/sdk/ohos/native/4.0.10.13

之后使用ndk编译时，该路径负责提供交叉编译工具和sysroot等依赖。

交叉编译zlib

zlib其实可以默认使用ndk中自带的库，这里也给出zlib的交叉编译方式。
下载源码和解压：

wget https://www.zlib.net/zlib-1.3.tar.gz
tar -xf zlib-1.3.tar.gz
cd zlib-1.3

编写编译脚本build.sh：

#!/bin/bash

set -e

if [ "$1" == "clean" ] || [ "$1" == "distclean" ];then
    make $1
    exit 0
fi

export OHOS_SDK_HONE=/opt/sdk/ohos
export OHOS_NATIVE_HOME=/opt/sdk/ohos/native/4.0.10.13

export PATH=$OHOS_NATIVE_HOME/llvm/bin:$PATH

export AR="$OHOS_NATIVE_HOME/llvm/bin/llvm-ar"
export AS="$OHOS_NATIVE_HOME/llvm/bin/llvm-as"
export LD="$OHOS_NATIVE_HOME/llvm/bin/ld.lld"
export RANLIB="$OHOS_NATIVE_HOME/llvm/bin/llvm-ranlib"
export STRIP="$OHOS_NATIVE_HOME/llvm/bin/llvm-strip"
export CC="$OHOS_NATIVE_HOME/llvm/bin/clang"
export NM="$OHOS_NATIVE_HOME/llvm/bin/llvm-nm"
export OBJDUMP="$OHOS_NATIVE_HOME/llvm/bin/llvm-objdump"

export CFLAGS="--target=arm-linux-ohos --sysroot=$OHOS_NATIVE_HOME/sysroot -g -fvisibility=hidden -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -fno-addrsig -Wformat -Werror=format-security -fno-limit-debug-info  -fPIC -march=armv7-a -mthumb"
export CPPFLAGS="--target=arm-linux-ohos --sysroot=$OHOS_NATIVE_HOME/sysroot"
export LDFLAGS="--target=arm-linux-ohos --rtlib=compiler-rt -fuse-ld=lld -lunwind"

./configure --prefix=${PWD}/_install --static

make && make install

执行编译命令：

bash build.sh distclean
bash build.sh

交叉编译openssl

这里移植移植最新的v3.2.0版本的OpenSSL，测试过openssl-3.0.12也可以正常编译。
下载源码和解压：

wget https://www.openssl.org/source/openssl-3.2.0.tar.gz
tar -xf openssl-3.2.0.tar.gz
cd openssl-3.2.0

编写编译脚本build.sh：

#!/bin/bash

set -e

if [ "$1" == "clean" ] || [ "$1" == "distclean" ];then
    make $1
    exit 0
fi

export OHOS_SDK_HONE=/opt/sdk/ohos
export OHOS_NATIVE_HOME=/opt/sdk/ohos/native/4.0.10.13

export PATH=$OHOS_NATIVE_HOME/llvm/bin:$PATH

export AR="$OHOS_NATIVE_HOME/llvm/bin/llvm-ar"
export AS="$OHOS_NATIVE_HOME/llvm/bin/llvm-as"
export LD="$OHOS_NATIVE_HOME/llvm/bin/ld.lld"
export RANLIB="$OHOS_NATIVE_HOME/llvm/bin/llvm-ranlib"
export STRIP="$OHOS_NATIVE_HOME/llvm/bin/llvm-strip"
export CC="$OHOS_NATIVE_HOME/llvm/bin/clang"
export NM="$OHOS_NATIVE_HOME/llvm/bin/llvm-nm"
export OBJDUMP="$OHOS_NATIVE_HOME/llvm/bin/llvm-objdump"

export CFLAGS="--target=arm-linux-ohos --sysroot=$OHOS_NATIVE_HOME/sysroot -g -fvisibility=hidden -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -fno-addrsig -Wformat -Werror=format-security -fno-limit-debug-info  -fPIC -march=armv7-a -mthumb"
export CPPFLAGS="--target=arm-linux-ohos --sysroot=$OHOS_NATIVE_HOME/sysroot"
export LDFLAGS="--target=arm-linux-ohos --rtlib=compiler-rt -fuse-ld=lld -lunwind -L${PWD}/_install/lib"

#./Configure linux-armv4 --prefix=${PWD}/_install zlib no-asm no-shared no-unit-test no-tests
./Configure linux-armv4 --prefix=${PWD}/_install zlib no-asm shared no-unit-test no-tests

# remove library atomic
sed -i 's/-latomic//g' ./Makefile

# use static library crypto and ssl
sed -i 's/-lcrypto/libcrypto.a/g' ./Makefile
sed -i 's/-lssl/libssl.a/g' ./Makefile

make -j$(($(nproc)*2)) && make install

由于ohos ndk中不带libatomic，编译时会报错，这里直接从自动生成的Makefile中删除atomic的依赖即可。另外我们希望openssh编译时以静态库的方式来连接crypto和ssl库，所以这里可以考虑把这两个库直接改成静态库的方式连接，也可以不改，不改的话默认链动态库。

执行编译命令：

bash build.sh distclean
bash build.sh

交叉编译openssh

zlib和openssl编译成功之后，就可以开始编译openssh了。
下载源码和解压：

wget https://ftp.openbsd.org/pub/OpenBSD/OpenSSH/portable/openssh-9.6p1.tar.gz
tar -xf openssh-9.6p1.tar.gz
cd openssh-9.6p1

编写编译脚本build.sh：

#!/bin/bash

set -e

if [ "$1" == "clean" ] || [ "$1" == "distclean" ];then
    make $1
    exit 0
fi

export OHOS_SDK_HONE=/opt/sdk/ohos
export OHOS_NATIVE_HOME=/opt/sdk/ohos/native/4.0.10.13

export PATH=$OHOS_NATIVE_HOME/llvm/bin:$PATH

export AR="$OHOS_NATIVE_HOME/llvm/bin/llvm-ar"
export AS="$OHOS_NATIVE_HOME/llvm/bin/llvm-as"
export LD="$OHOS_NATIVE_HOME/llvm/bin/ld.lld"
export RANLIB="$OHOS_NATIVE_HOME/llvm/bin/llvm-ranlib"
export STRIP="$OHOS_NATIVE_HOME/llvm/bin/llvm-strip"
export CC="$OHOS_NATIVE_HOME/llvm/bin/clang"
export NM="$OHOS_NATIVE_HOME/llvm/bin/llvm-nm"
export OBJDUMP="$OHOS_NATIVE_HOME/llvm/bin/llvm-objdump"

export CFLAGS="--target=arm-linux-ohos --sysroot=$OHOS_NATIVE_HOME/sysroot -g -fvisibility=hidden -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -fno-addrsig -Wformat -Werror=format-security -fno-limit-debug-info  -fPIC -march=armv7-a -mthumb"
export CPPFLAGS="--target=arm-linux-ohos --sysroot=$OHOS_NATIVE_HOME/sysroot"
export LDFLAGS="--target=arm-linux-ohos --rtlib=compiler-rt -fuse-ld=lld -lunwind"

./configure --prefix=${PWD}/_install \
            --target=arm-linux-ohos \
            --host=arm-linux \
            --sysconfdir=/etc/ssh \
            --with-libs --with-zlib=${PWD}/../zlib-1.3/_install \
            --with-ssl-dir=${PWD}/../openssl-3.2.0/_install \
            --disable-etc-default-login \
            ${EXTOPTS}

make -j$(($(nproc)*2)) && make install

执行编译命令：

bash build.sh distclean
bash build.sh

报错1：
configure报错找不到libcrypto库，发现默认会链动态库，动态库不生成时静态库会优先从sysroot中查找，此时会提示缺少很多符号，解决方案是在LDFLAGS中追加libcrypto.a的静态库方式连接，以及指定zlib库（libcrypto依赖zlib）：

${PWD}/../openssl-3.0.12/_install/lib/libcrypto.a -lz

报错2：

/opt/sdk/ohos/native/4.0.10.13/llvm/bin/clang --target=arm-linux-ohos --sysroot=/opt/sdk/ohos/native/4.0.10.13/sysroot -g -fvisibility=hidden -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -fno-addrsig -Wformat -Werror=format-security -fno-limit-debug-info  -fPIC -march=armv7-a -mthumb -pipe -Wunknown-warning-option -Qunused-arguments -Wall -Wextra -Wpointer-arith -Wuninitialized -Wsign-compare -Wformat-security -Wsizeof-pointer-memaccess -Wno-pointer-sign -Wno-unused-parameter -Wno-unused-result -Wmisleading-indentation -Wbitwise-instead-of-logical -fno-strict-aliasing -mretpoline -D_FORTIFY_SOURCE=2 -ftrapv -fno-builtin-memset -fstack-protector-strong -fPIE   -I. -I. -I/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/../openssl-3.0.12/_install/include -I/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/../zlib-1.3/_install/include --target=arm-linux-ohos --sysroot=/opt/sdk/ohos/native/4.0.10.13/sysroot -D_XOPEN_SOURCE=600 -D_BSD_SOURCE -D_DEFAULT_SOURCE -D_GNU_SOURCE -DSSHDIR=\"/etc/ssh\" -D_PATH_SSH_PROGRAM=\"/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/_install/bin/ssh\" -D_PATH_SSH_ASKPASS_DEFAULT=\"/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/_install/libexec/ssh-askpass\" -D_PATH_SFTP_SERVER=\"/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/_install/libexec/sftp-server\" -D_PATH_SSH_KEY_SIGN=\"/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/_install/libexec/ssh-keysign\" -D_PATH_SSH_PKCS11_HELPER=\"/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/_install/libexec/ssh-pkcs11-helper\" -D_PATH_SSH_SK_HELPER=\"/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/_install/libexec/ssh-sk-helper\" -D_PATH_SSH_PIDDIR=\"/var/run\" -D_PATH_PRIVSEP_CHROOT_DIR=\"/var/empty\" -DHAVE_CONFIG_H -c rijndael.c -o rijndael.o
/opt/sdk/ohos/native/4.0.10.13/llvm/bin/clang --target=arm-linux-ohos --sysroot=/opt/sdk/ohos/native/4.0.10.13/sysroot -g -fvisibility=hidden -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -fno-addrsig -Wformat -Werror=format-security -fno-limit-debug-info  -fPIC -march=armv7-a -mthumb -pipe -Wunknown-warning-option -Qunused-arguments -Wall -Wextra -Wpointer-arith -Wuninitialized -Wsign-compare -Wformat-security -Wsizeof-pointer-memaccess -Wno-pointer-sign -Wno-unused-parameter -Wno-unused-result -Wmisleading-indentation -Wbitwise-instead-of-logical -fno-strict-aliasing -mretpoline -D_FORTIFY_SOURCE=2 -ftrapv -fno-builtin-memset -fstack-protector-strong   -fPIC -I. -I.. -I. -I./.. -I/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/../openssl-3.0.12/_install/include -I/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/../zlib-1.3/_install/include --target=arm-linux-ohos --sysroot=/opt/sdk/ohos/native/4.0.10.13/sysroot -D_XOPEN_SOURCE=600 -D_BSD_SOURCE -D_DEFAULT_SOURCE -D_GNU_SOURCE -DHAVE_CONFIG_H -c getgrouplist.c
xcrypt.c:134:21: warning: call to undeclared function 'getspnam'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
        struct spwd *spw = getspnam(pw->pw_name);
                           ^
xcrypt.c:134:15: error: incompatible integer to pointer conversion initializing 'struct spwd *' with an expression of type 'int' [-Wint-conversion]
        struct spwd *spw = getspnam(pw->pw_name);
                     ^     ~~~~~~~~~~~~~~~~~~~~~
2 warnings and 1 error generated.
make[1]: *** [Makefile:106: xcrypt.o] Error 1
make[1]: *** Waiting for unfinished jobs....

这是因为ndk目录下的musl库和头文件shadow.h没有支持这个函数，查看源码路径third_party/musl以及out目录下生成的libc.so，发现getspnam是支持的，只是sdk编译时没有带而已。所以直接拷贝替换即可：

cp /opt/sdk/ohos/native/4.0.10.13/sysroot/usr/include/shadow.h /opt/sdk/ohos/native/4.0.10.13/sysroot/usr/include/shadow.h.bak
cp ${OHOS_SRC}/out/${PRODUCT}/obj/third_party/musl/usr/include/arm-linux-ohos/shadow.h /opt/sdk/ohos/native/4.0.10.13/sysroot/usr/include/shadow.h

cp /opt/sdk/ohos/native/4.0.10.13/sysroot/usr/lib/arm-linux-ohos/libc.so /opt/sdk/ohos/native/4.0.10.13/sysroot/usr/lib/arm-linux-ohos/libc.so.bak
cp ${OHOS_SRC}/out/${PRODUCT}/obj/third_party/musl/usr/lib/arm-linux-ohos/libc.so /opt/sdk/ohos/native/4.0.10.13/sysroot/usr/lib/arm-linux-ohos/libc.so

报错3：

/opt/sdk/ohos/native/4.0.10.13/llvm/bin/clang --target=arm-linux-ohos --sysroot=/opt/sdk/ohos/native/4.0.10.13/sysroot -g -fvisibility=hidden -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -fno-addrsig -Wformat -Werror=format-security -fno-limit-debug-info  -fPIC -march=armv7-a -mthumb -pipe -Wunknown-warning-option -Qunused-arguments -Wall -Wextra -Wpointer-arith -Wuninitialized -Wsign-compare -Wformat-security -Wsizeof-pointer-memaccess -Wno-pointer-sign -Wno-unused-parameter -Wno-unused-result -Wmisleading-indentation -Wbitwise-instead-of-logical -fno-strict-aliasing -mretpoline -D_FORTIFY_SOURCE=2 -ftrapv -fno-builtin-memset -fstack-protector-strong -fPIE   -I. -I. -I/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/../openssl-3.0.12/_install/include -I/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/../zlib-1.3/_install/include --target=arm-linux-ohos --sysroot=/opt/sdk/ohos/native/4.0.10.13/sysroot -D_XOPEN_SOURCE=600 -D_BSD_SOURCE -D_DEFAULT_SOURCE -D_GNU_SOURCE -DSSHDIR=\"/etc/ssh\" -D_PATH_SSH_PROGRAM=\"/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/_install/bin/ssh\" -D_PATH_SSH_ASKPASS_DEFAULT=\"/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/_install/libexec/ssh-askpass\" -D_PATH_SFTP_SERVER=\"/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/_install/libexec/sftp-server\" -D_PATH_SSH_KEY_SIGN=\"/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/_install/libexec/ssh-keysign\" -D_PATH_SSH_PKCS11_HELPER=\"/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/_install/libexec/ssh-pkcs11-helper\" -D_PATH_SSH_SK_HELPER=\"/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/_install/libexec/ssh-sk-helper\" -D_PATH_SSH_PIDDIR=\"/var/run\" -D_PATH_PRIVSEP_CHROOT_DIR=\"/var/empty\" -DHAVE_CONFIG_H -c ssh-ecdsa.c -o ssh-ecdsa.o
xcrypt.c:88:4: warning: call to undeclared function 'explicit_bzero'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
                        explicit_bzero(passwd, strlen(passwd));
                        ^
xcrypt.c:117:12: warning: call to undeclared function 'crypt'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
        crypted = crypt(password, salt);
                  ^
xcrypt.c:117:10: error: incompatible integer to pointer conversion assigning to 'char *' from 'int' [-Wint-conversion]
        crypted = crypt(password, salt);
                ^ ~~~~~~~~~~~~~~~~~~~~~
2 warnings and 1 error generated.
make[1]: *** [Makefile:106: xcrypt.o] Error 1
make[1]: *** Waiting for unfinished jobs....
sshbuf.c:189:3: warning: call to undeclared function 'explicit_bzero'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
                explicit_bzero(buf->d, buf->alloc);
                ^
sshbuf.c:215:2: warning: call to undeclared function 'explicit_bzero'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
        explicit_bzero(buf->d, buf->alloc);
        ^
/opt/sdk/ohos/native/4.0.10.13/llvm/bin/clang --target=arm-linux-ohos --sysroot=/opt/sdk/ohos/native/4.0.10.13/sysroot -g -fvisibility=hidden -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -fno-addrsig -Wformat -Werror=format-security -fno-limit-debug-info  -fPIC -march=armv7-a -mthumb -pipe -Wunknown-warning-option -Qunused-arguments -Wall -Wextra -Wpointer-arith -Wuninitialized -Wsign-compare -Wformat-security -Wsizeof-pointer-memaccess -Wno-pointer-sign -Wno-unused-parameter -Wno-unused-result -Wmisleading-indentation -Wbitwise-instead-of-logical -fno-strict-aliasing -mretpoline -D_FORTIFY_SOURCE=2 -ftrapv -fno-builtin-memset -fstack-protector-strong -fPIE   -I. -I. -I/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/../openssl-3.0.12/_install/include -I/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/../zlib-1.3/_install/include --target=arm-linux-ohos --sysroot=/opt/sdk/ohos/native/4.0.10.13/sysroot -D_XOPEN_SOURCE=600 -D_BSD_SOURCE -D_DEFAULT_SOURCE -D_GNU_SOURCE -DSSHDIR=\"/etc/ssh\" -D_PATH_SSH_PROGRAM=\"/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/_install/bin/ssh\" -D_PATH_SSH_ASKPASS_DEFAULT=\"/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/_install/libexec/ssh-askpass\" -D_PATH_SFTP_SERVER=\"/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/_install/libexec/sftp-server\" -D_PATH_SSH_KEY_SIGN=\"/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/_install/libexec/ssh-keysign\" -D_PATH_SSH_PKCS11_HELPER=\"/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/_install/libexec/ssh-pkcs11-helper\" -D_PATH_SSH_SK_HELPER=\"/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/_install/libexec/ssh-sk-helper\" -D_PATH_SSH_PIDDIR=\"/var/run\" -D_PATH_PRIVSEP_CHROOT_DIR=\"/var/empty\" -DHAVE_CONFIG_H -c ssh-ecdsa-sk.c -o ssh-ecdsa-sk.o
arc4random.c:123:2: warning: call to undeclared function 'explicit_bzero'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
        explicit_bzero(rnd, sizeof(rnd));       /* discard source seed */
        ^
xmalloc.c:67:12: warning: call to undeclared function 'reallocarray'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
        new_ptr = reallocarray(ptr, nmemb, size);
                  ^
xmalloc.c:67:10: error: incompatible integer to pointer conversion assigning to 'void *' from 'int' [-Wint-conversion]
        new_ptr = reallocarray(ptr, nmemb, size);
                ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning and 1 error generated.
readpass.c:103:3: warning: call to undeclared function 'explicit_bzero'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
                explicit_bzero(buf, sizeof(buf));
                ^
readpass.c:109:2: warning: call to undeclared function 'explicit_bzero'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
        explicit_bzero(buf, sizeof(buf));
        ^
readpass.c:194:2: warning: call to undeclared function 'explicit_bzero'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
        explicit_bzero(buf, sizeof(buf));
        ^
bcrypt_pbkdf.c:108:2: warning: call to undeclared function 'explicit_bzero'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
        explicit_bzero(ciphertext, sizeof(ciphertext));
        ^
make: *** [Makefile:195: xmalloc.o] Error 1
make: *** Waiting for unfinished jobs....

explicit_bzero 隐式申明的函数，由musl实现，需要替换libc.so库和头文件string.h：

cp /opt/sdk/ohos/native/4.0.10.13/sysroot/usr/include/string.h /opt/sdk/ohos/native/4.0.10.13/sysroot/usr/include/string.h.bak
cp ${OHOS_SRC}/out/${PRODUCT}/obj/third_party/musl/usr/include/arm-linux-ohos/string.h /opt/sdk/ohos/native/4.0.10.13/sysroot/usr/include/string.h

crypt需要替换libc.so库和头文件crypt.h：

cp /opt/sdk/ohos/native/4.0.10.13/sysroot/usr/include/crypt.h /opt/sdk/ohos/native/4.0.10.13/sysroot/usr/include/crypt.h.bak
cp ${OHOS_SRC}/out/${PRODUCT}/obj/third_party/musl/usr/include/arm-linux-ohos/crypt.h /opt/sdk/ohos/native/4.0.10.13/sysroot/usr/include/crypt.h

reallocarray需要替换libc.so库和头文件stdlib.h：

cp /opt/sdk/ohos/native/4.0.10.13/sysroot/usr/include/stdlib.h /opt/sdk/ohos/native/4.0.10.13/sysroot/usr/include/stdlib.h.bak
cp ${OHOS_SRC}/out/${PRODUCT}/obj/third_party/musl/usr/include/arm-linux-ohos/stdlib.h /opt/sdk/ohos/native/4.0.10.13/sysroot/usr/include/stdlib.h

报错4：
提示头文件linux/socket.h和sys/socket.h中的结构体sockaddr_storage重复定义：

In file included from port-net.c:46:
In file included from /opt/sdk/ohos/native/4.0.10.13/sysroot/usr/include/linux/if.h:23:
/opt/sdk/ohos/native/4.0.10.13/sysroot/usr/include/linux/socket.h:23:8: error: redefinition of 'sockaddr_storage'
struct sockaddr_storage {
       ^
/opt/sdk/ohos/native/4.0.10.13/sysroot/usr/include/sys/socket.h:377:8: note: previous definition is here
struct sockaddr_storage {

查看这两个头文件：
/opt/sdk/ohos/native/4.0.10.13/sysroot/usr/include/sys/socket.h

#ifndef __MUSL__
struct sockaddr_storage {
    sa_family_t ss_family;
    char __ss_padding[128-sizeof(long)-sizeof(sa_family_t)];
    unsigned long __ss_align;
};
#endif

/opt/sdk/ohos/native/4.0.10.13/sysroot/usr/include/linux/socket.h

#ifndef _UAPI_LINUX_SOCKET_H
#define _UAPI_LINUX_SOCKET_H
#define _K_SS_MAXSIZE 128
typedef unsigned short __kernel_sa_family_t;
struct sockaddr_storage {
  union {
    struct {
      __kernel_sa_family_t ss_family;
      char __data[_K_SS_MAXSIZE - sizeof(unsigned short)];
    };
    void * __align;
  };
};
#endif

可见没有定义__MUSL__宏时，sys/socket.h中的struct sockaddr_storage定义和linux/socket.h冲突了。定义__MUSL__宏即可。

报错5：

/opt/sdk/ohos/native/4.0.10.13/llvm/bin/clang -o ssh ssh.o readconf.o clientloop.o sshtty.o sshconnect.o sshconnect2.o mux.o ssh-sk-client.o -L. -Lopenbsd-compat/ -L/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/../openssl-3.0.12/_install/lib -L/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/../zlib-1.3/_install/lib --target=arm-linux-ohos --rtlib=compiler-rt -fuse-ld=lld -lunwind /work/platform/other/test/ohos/thirdparty/openssh-9.6p1/../openssl-3.0.12/_install/lib/libcrypto.a -lz -Wl,-z,retpolineplt -Wl,-z,relro -Wl,-z,now -Wl,-z,noexecstack -fstack-protector-strong -pie  -lssh -lopenbsd-compat   -lcrypto  -lz
2 warnings generated.
ld.lld: error: undefined symbol: __res_state
>>> referenced by getrrsetbyname.c:195 (/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/openbsd-compat/getrrsetbyname.c:195)
>>>               getrrsetbyname.o:(getrrsetbyname) in archive openbsd-compat/libopenbsd-compat.a
clang-15: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Makefile:234: ssh-keyscan] Error 1
make: *** Waiting for unfinished jobs....
ld.lld: error: undefined symbol: __res_state
>>> referenced by getrrsetbyname.c:195 (/work/platform/other/test/ohos/thirdparty/openssh-9.6p1/openbsd-compat/getrrsetbyname.c:195)
>>>               getrrsetbyname.o:(getrrsetbyname) in archive openbsd-compat/libopenbsd-compat.a
clang-15: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Makefile:207: ssh] Error 1

musl c库中没有定义__res_state，所以我们这里实际上不应该使用它。

diff --git a/openbsd-compat/Makefile.in b/openbsd-compat/Makefile.in
index 1d54995..4b19428 100644
--- a/openbsd-compat/Makefile.in
+++ b/openbsd-compat/Makefile.in
@@ -17,6 +17,9 @@ INSTALL=@INSTALL@
 LDFLAGS=-L. @LDFLAGS@
 LDFLAGS_NOPIE=-L. -Lopenbsd-compat/ @LDFLAGS_NOPIE@

+#remove for ohos
+       # getrrsetbyname.o \
+
 OPENBSD=arc4random.o \
        arc4random_uniform.o \
        base64.o \
@@ -33,7 +36,6 @@ OPENBSD=arc4random.o \
        getcwd.o \
        getgrouplist.o \
        getopt_long.o \
-       getrrsetbyname.o \
        glob.o \
        inet_aton.o \
        inet_ntoa.o \
@@ -66,6 +68,9 @@ OPENBSD=arc4random.o \
        timingsafe_bcmp.o \
        vis.o

+#remove for ohos
+#      getrrsetbyname-ldns.o \
+
 COMPAT= bsd-asprintf.o \
        bsd-closefrom.o \
        bsd-cygwin_util.o \
@@ -88,7 +93,6 @@ COMPAT= bsd-asprintf.o \
        bsd-timegm.o \
        bsd-waitpid.o \
        fake-rfc2553.o \
-       getrrsetbyname-ldns.o \
        kludge-fd_set.o \
        openssl-compat.o \
        libressl-api-compat.o \
diff --git a/dns.c b/dns.c
index 9392414..0125780 100644
--- a/dns.c
+++ b/dns.c
@@ -220,12 +220,17 @@ verify_host_key_dns(const char *hostname, struct sockaddr *address,
                return -1;
        }

+#if !defined(__OHOS__)
        result = getrrsetbyname(hostname, DNS_RDATACLASS_IN,
            DNS_RDATATYPE_SSHFP, 0, &fingerprints);
        if (result) {
                verbose("DNS lookup error: %s", dns_result_totext(result));
                return -1;
        }
+#else
+       /* unsupported in OpenHarmony */
+       result = -1;
+#endif

        if (fingerprints->rri_flags & RRSET_VALIDATED) {
                *flags |= DNS_VERIFY_SECURE;
@@ -259,7 +264,9 @@ verify_host_key_dns(const char *hostname, struct sockaddr *address,
                    &hostkey_digest, &hostkey_digest_len, hostkey)) {
                        error("Error calculating key fingerprint.");
                        free(dnskey_digest);
+#if !defined(__OHOS__)
                        freerrset(fingerprints);
+#endif
                        return -1;
                }

@@ -281,7 +288,9 @@ verify_host_key_dns(const char *hostname, struct sockaddr *address,
                free(hostkey_digest); /* from sshkey_fingerprint_raw() */
        }

+#if !defined(__OHOS__)
        freerrset(fingerprints);
+#endif

        /* If any fingerprint failed to validate, return failure. */
        if (*flags & DNS_VERIFY_FAILED)

报错6：
安装报错，无法创建/var/empty：

/usr/bin/mkdir -p /work/platform/other/test/ohos/thirdparty/openssh-9.6p1/_install/bin
/usr/bin/mkdir -p /work/platform/other/test/ohos/thirdparty/openssh-9.6p1/_install/sbin
/usr/bin/mkdir -p /work/platform/other/test/ohos/thirdparty/openssh-9.6p1/_install/share/man/man1
/usr/bin/mkdir -p /work/platform/other/test/ohos/thirdparty/openssh-9.6p1/_install/share/man/man5
/usr/bin/mkdir -p /work/platform/other/test/ohos/thirdparty/openssh-9.6p1/_install/share/man/man8
/usr/bin/mkdir -p /work/platform/other/test/ohos/thirdparty/openssh-9.6p1/_install/libexec
/usr/bin/mkdir -p -m 0755 /var/empty
/usr/bin/mkdir: cannot create directory ‘/var/empty’: Permission denied

本地编译才需要创建，仔细查看Makefile逻辑，发现make install可以通过追加DESTDIR来决定目标路径：

make install DESTDIR=${PWD}/_install

最终编译openssh的脚本build.sh：

#!/bin/bash

set -e

if [ "$1" == "clean" ] || [ "$1" == "distclean" ];then
    make $1
    exit 0
fi

export OHOS_SDK_HONE=/opt/sdk/ohos
export OHOS_NATIVE_HOME=/opt/sdk/ohos/native/4.0.10.13

export PATH=$OHOS_NATIVE_HOME/llvm/bin:$PATH

export AR="$OHOS_NATIVE_HOME/llvm/bin/llvm-ar"
export AS="$OHOS_NATIVE_HOME/llvm/bin/llvm-as"
export LD="$OHOS_NATIVE_HOME/llvm/bin/ld.lld"
export RANLIB="$OHOS_NATIVE_HOME/llvm/bin/llvm-ranlib"
export STRIP="$OHOS_NATIVE_HOME/llvm/bin/llvm-strip"
export CC="$OHOS_NATIVE_HOME/llvm/bin/clang"
export NM="$OHOS_NATIVE_HOME/llvm/bin/llvm-nm"
export OBJDUMP="$OHOS_NATIVE_HOME/llvm/bin/llvm-objdump"

export CFLAGS="--target=arm-linux-ohos --sysroot=$OHOS_NATIVE_HOME/sysroot -g -fvisibility=hidden -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -fno-addrsig -Wformat -Werror=format-security -fno-limit-debug-info  -fPIC -march=armv7-a -mthumb -D__MUSL__ -D__OHOS__"
export CPPFLAGS="--target=arm-linux-ohos --sysroot=$OHOS_NATIVE_HOME/sysroot -D__MUSL__ -D__OHOS__"
export LDFLAGS="--target=arm-linux-ohos --rtlib=compiler-rt -fuse-ld=lld -lunwind -lz"

export LDFLAGS="--target=arm-linux-ohos --rtlib=compiler-rt -fuse-ld=lld -lunwind"

ZLIB_VER="1.3"
SSL_VER="3.2.0"
LIBCRYPO_STATIC="${PWD}/../openssl-${SSL_VER}/_install/lib/libcrypto.a"

export LIBS="${LIBCRYPO_STATIC} -lz"

./configure --prefix=/system \
            --target=arm-linux-ohos \
            --host=arm-linux \
            --bindir=/system/bin \
            --sbindir=/system/sbin \
            --libexecdir=/system/libexe \
            --sysconfdir=/system/etc/ssh \
            --localstatedir=/data/ssh/var \
            --libdir=/system/lib \
            --with-libs --with-zlib=${PWD}/../zlib-${ZLIB_VER}/_install \
            --with-ssl-dir=${PWD}/../openssl-${SSL_VER}/_install \
            --disable-etc-default-login \
            ${EXTOPTS}

#make -j$(($(nproc)*2)) CHANNELLIBS=${LIBCRYPO_STATIC} SSHDLIBS=${LIBCRYPO_STATIC} piddir="/data/ssh/var/run"
make -j$(($(nproc)*2)) CHANNELLIBS="" SSHDLIBS="" piddir="/data/ssh/var/run"
make install-nokeys DESTDIR=${PWD}/_install STRIP_OPT="-s --strip-program=${STRIP}"

功能调试和代码修改

编译通过之后，接下来就是魔改openssh的源码来调试sshd/ssh的远程登录、scp和sftp的安全拷贝功能了。遇到最主要的问题：因为openharmony下的passwd文件没有定义用户的home目录，导致无法在登录之后切换到home路径；passwd中定义的shell为/bin/false导致用户无法登录；没有shadow文件，导致不能密码登录（当然密钥登录的方式还是相对安全性更可靠一点）。代码修改的细节且按下不表，待续…

添加到系统编译流程

这里不讲如何将上述的Makefile项目转换成BUILD.gn添加到OHOS源码目录下的third_party/目录中进行编译，不难，但是要花点时间转换。所以，我们这里直接将编译好的可执行程序等以prebuilt文件的形式直接拷贝到编译。
以放到device/board目录为例，我们已经在该目录有现成的BUILD.gn文件了，且这个BUILD.gn中编译目录已经被依赖：
device/board/${VENDOR}/${PRODUCT}/files/BUILD.gn

declare_args() {
  use_openssh = true
  use_dropbear = false
}

group("prebuilt_files") {
  deps = [
    ":busybox",
    ":sysparam.sh",
  ]
  if (use_openssh) {
    deps += [ "openssh:openssh" ]
  }
  if (use_dropbear) {
    deps += [ "//third_party/dropbear:dropbear_group" ]
  }
}

增加gn参数use_openssh和use_dropbear用于决定集成openssh还是dropbear，如果没有移植过dropbear，这里可以把dropbear相关的直接删除掉。

然后将前面openssh编译安装目录下的文件拷贝到device/board/${VENDOR}/${PRODUCT}/files/openssh中：

openssh/
|-- bin
|   |-- scp
|   |-- sftp
|   |-- ssh
|   |-- ssh-add
|   |-- ssh-agent
|   |-- ssh-keygen
|   `-- ssh-keyscan
|-- BUILD.gn
|-- etc
|   |-- ssh
|   |   |-- authorized_keys
|   |   |-- moduli
|   |   |-- ssh_config
|   |   |-- sshd_config
|   |   |-- ssh_host_dsa_key
|   |   |-- ssh_host_dsa_key.pub
|   |   |-- ssh_host_rsa_key
|   |   `-- ssh_host_rsa_key.pub
|   `-- sshd.cfg
|-- libexe
|   |-- sftp-server
|   |-- ssh-keysign
|   |-- ssh-pkcs11-helper
|   `-- ssh-sk-helper
`-- sbin
    `-- sshd

device/board/${VENDOR}/${PRODUCT}/files/openssh/BUILD.gn

# Copyright (C) 2024 
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import("//build/ohos.gni")

bin_source_list = [
  "bin/scp",
  "bin/sftp",
  "bin/ssh",
  "sbin/sshd",
  "libexe/sftp-server",
]

foreach(_bin_src, bin_source_list) {
  _target_name = string_replace(_bin_src, "/", "_")
  ohos_prebuilt_executable("${_target_name}_bin") {
    source = "${_bin_src}"
    install_enable = true
    install_images = [ system_base_dir ]
    module_install_dir = "bin"
    part_name = "rockchip_products"
  }
}

etc_source_list = [
  "etc/ssh/authorized_keys",
  "etc/ssh/moduli",
  "etc/ssh/ssh_config",
  "etc/ssh/sshd_config",
  "etc/ssh/ssh_host_dsa_key",
  "etc/ssh/ssh_host_dsa_key.pub",
  "etc/ssh/ssh_host_rsa_key",
  "etc/ssh/ssh_host_rsa_key.pub",
]

foreach(_etc_src, etc_source_list) {
  _target_name = string_replace(_etc_src, "/", "_")
  ohos_prebuilt_etc("${_target_name}_etc") {
    source = "${_etc_src}"
    install_enable = true
    install_images = [ system_base_dir ]
    module_install_dir = "etc/ssh"
    part_name = "rockchip_products"
  }
}

ohos_prebuilt_etc("sshd.init") {
  source = "etc/sshd.cfg"
  relative_install_dir = "init"
  part_name = "rockchip_products"
}

group("openssh") {
  deps = [ ":sshd.init" ]
  foreach(_bin_src, bin_source_list) {
    _depname = string_replace(_bin_src, "/", "_")
    deps += [ ":${_depname}_bin" ]
  }
  foreach(_etc_src, etc_source_list) {
    _depname = string_replace(_etc_src, "/", "_")
    deps += [ ":${_depname}_etc" ]
  }
}

gn语法中可以通过string_replace将源路径字符串中的"/“替换为”_“，因为依赖目标名称中不能出现”/"，否则gn会报错。gn更多语法可以阅读源码third_party/gn/docs/reference.md

device/board/${VENDOR}/${PRODUCT}/files/openssh/etc/sshd.cfg

{
    "jobs" : [{
            "name" : "post-fs",
            "cmds" : [
                "mkdir /data/ssh",
                "mkdir /data/ssh/var",
                "mkdir /data/ssh/var/log",
                "mkdir /data/ssh/var/run",
                "mkdir /data/ssh/var/empty"
            ]
        }, {
            "name" : "boot && param:persist.sys.sshd.enable=*",
            "condition" : "boot && persist.sys.sshd.enable=*",
            "cmds" : [
                "setparam sys.sshd.enable ${persist.sys.sshd.enable}"
            ]
        }, {
            "name" : "param:persist.sys.sshd.enable=*",
            "condition" : "persist.sys.sshd.enable=*",
            "cmds" : [
                "setparam sys.sshd.enable ${persist.sys.sshd.enable}"
            ]
        }, {
            "name" : "param:sys.sshd.enable=true",
            "condition" : "sys.sshd.enable=true",
            "cmds" : [
                "mkdir /data/ssh",
                "start sshd"
            ]
        }, {
            "name" : "param:sys.sshd.enable=false",
            "condition" : "sys.sshd.enable=false",
            "cmds" : [
                "stop ssh"
            ]
        }
    ],
    "services" : [{
            "name" : "sshd",
            "path" : ["/system/bin/sshd", "-f", "/etc/ssh/sshd_config", "-D"],
            "uid" : "root",
            "gid" : [ "root", "shell", "log", "readproc" ],
            "sandbox" : 0,
            "start-mode" : "condition",
            "secon" : "u:r:sh:s0",
            "disabled" : 1
        }
    ]
}

这里需要将系统参数persist.sys.sshd.enable设置为true才能让sshd服务默认启动。

device/board/${VENDOR}/${PRODUCT}/files/openssh/etc/ssh/sshd_config

Protocol 2
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_dsa_key
AuthorizedKeysFile  /etc/ssh/authorized_keys
PasswordAuthentication yes
PidFile /data/ssh/var/run/sshd.pid
Subsystem   sftp    internal-sftp

密钥对文件ssh_host_dsa_key和ssh_host_dsa_key.pub，ssh_host_rsa_key和ssh_host_rsa_key.pub，以及已认证公钥列表文件authorized_keys请自行生成。
另外DAC配置文件中需要为ssh的密钥文件配置权限，其他用户不能对私钥文件有任何可访问权限，否则启动sshd服务时会因为私钥文件权限太开放而退出：
build/ohos/images/mkimage/dac.txt

diff --git a/ohos/images/mkimage/dac.txt b/ohos/images/mkimage/dac.txt
index 163b069..d2646f8 100644
--- a/ohos/images/mkimage/dac.txt
+++ b/ohos/images/mkimage/dac.txt
@@ -52,3 +62,9 @@ updater/lib/ld-musl-aaarch64-asan.so.1,00755,      0,          2000,       0
 updater/lib/ld-musl-x86_64-asan.so.1,00755,      0,          2000,       0
 vendor/etc/thermal_config/hdf/thermal_hdi_config.xml,00644,    3025,   3025,   0
 system/etc/ledconfig/led_config.json,00644,    3025,   3025,           0
+system/etc/ssh/authorized_keys, 00600,      0,          0,          0
+system/etc/ssh/ssh_host_rsa_key,00600,      0,          0,          0
+system/etc/ssh/ssh_host_dsa_key,00600,      0,          0,          0
+system/etc/ssh/*.pub,           00644,      0,          0,          0
+system/etc/ssh/moduli,          00644,      0,          0,          0
+system/etc/ssh/ssh*_config,     00644,      0,          0,          0

对应system的image config文件中需要通过–dac_config选项来指定该文件作为dac配置。

新增服务sshd需要添加到特权进程列表中：
vendor/${VENDOR}/${PRODUCT}/security_config/high_privilege_process_list.json

        {
            "name": "sshd",
            "uid": "root"
        }

startup_guard的白名单中列表也要增加sshd服务：
developtools/integration_verification/tools/startup_guard/rules/NO-Config-Cmds-In-Init/whitelist.json

[
    {
            "start-modes": [
                {
                    "start-mode":"condition",
                    "service":[
                        "sshd",
                        "mmi_uinput_service"
                    ]
                }
            ],
            "start-cmd":[
                "sshd",
                "concurrent_task_service"
            ]
    }
]