1. jar包解压
jar cf test.jar
2. bat & vb混写
chcgp 65001用于指定代码页为UTF8
::' VBS/Batch Hybrid
rem^ &chcgp 65001
rem^ &echo off &PUSHD %~DP0 &TITLE opsTools
rem^ &mode con cols=80 lines=35
:'menu
rem^ &cls
rem^ &set /p user_input=选择并进入命令:
rem^ &if %user_input%==1 goto 'a
rem^ &if not %user_input%=="" goto 'z
rem call vbs file
:'a
rem^ &cscript //nologo //e:vbscript "%~f0" %user_input%
:'z
rem^ &echo.
rem^ &echo 你的选择无效,请按任意键返回菜单!
rem^ &echo. & pause
rem^ &goto 'menu
'----- VBS portion ------------
Set WshShell = WScript.CreateObject("WScript.Shell")
WshShell.run"cmd"
'wscript.quit(0)
-
generator = pipeline(‘text-generation’, model=‘EleutherAI/gpt-neo-2.7B’)
pipeline 模型下载路径~/.cache/
如果遇到网络中断, 模型下载失败,需要清理缓存目录。 -
gpt2-neo hello world
from transformers import pipeline;
print("hello world")
print(pipeline.__dir__)
generator = pipeline('text-generation', model='EleutherAI/gpt-neo-2.7B')
prompt="how to train gpt?";
res = generator(prompt, max_length=100,do_sample=True, temperature=0.9,);
print(res[0]['generated_text'])
- minio performance tuning
关于性能调优
可以通过/proc/sys/xx/xxxx查看对应的值, 通过/etc/sysctl.conf 进行配置
https://min.io/resources/docs/MinIO-Throughput-Benchmarks-on-HDD-24-Node.pdf
https://github.com/zkryakgul/minio-kernel-tuning
以下设定来自 hhd-24-node.pdf , 对应的主机配置较高, 请合理取用
# maximum number of open files/file descriptors
#文件最大打开数
fs.file-max = 4194303
# use as little swap space as possible
# 当内存使用达到 (100 - vm.swappiness )%时, 才开始使用swap。
vm.swappiness = 1
# prioritize application RAM against disk/swap cache
#告诉内核,当清理inoe/dentry缓存时应该用什么样的优先级。via: https://blog.csdn.net/qq_34246965/article/details/109258656
# 尽可能保留缓存
vm.vfs_cache_pressure = 10
# minimum free memory
#最小内存 参考 https://blog.csdn.net/joyeu/article/details/20063429
vm.min_free_kbytes = 1000000
# maximum receive socket buffer (bytes)
#指定了接收套接字缓冲区大小的最大值(以字节为单位) 最大值 256m
net.core.rmem_max = 268435456
# maximum send buffer socket buffer (bytes) 缓冲区256m
net.core.wmem_max = 268435456
# default receive buffer socket size (bytes) 默认64m
net.core.rmem_default = 67108864
# default send buffer socket size (bytes) 默认64m
net.core.wmem_default = 67108864
# maximum number of packets in one poll cycle
#对于我们的Linux的处理软中断的 ksoftirqd 来说,它也和番茄工作法思路类似。一旦它被硬中断触发开始了工作,它会集中精力处理一波儿网络包(绝不只是1个),然后再去做别的事情。 via https://blog.csdn.net/weixin_42241611/article/details/126875402
net.core.netdev_budget = 1200
# maximum ancillary buffer size per socket
#每个套接字的缓冲区最大大小。 128m
net.core.optmem_max = 134217728
# maximum number of incoming connections
#设置系统中每个套接字的最大排队连接数
net.core.somaxconn = 65535
# maximum number of packets queued
#当每个网络接口 接受数据包的速率比内核处理这些包的速率快时,允许发送到队列的数据包的最大数目
net.core.netdev_max_backlog = 250000
# maximum read buffer space
# https://www.cnblogs.com/51core/articles/13683820.html
net.ipv4.tcp_rmem = 67108864 134217728 268435456
# maximum write buffer space
net.ipv4.tcp_wmem = 67108864 134217728 268435456
# enable low latency mode
# uname -r 查看内核版本。
# /proc/version 查看内核版本
#tcp_low_latency设为0时,以整个操作系统的效率优先,此时TCP会通过使用prequeue队列,使网络软中断的执行时间缩短,回ACK的时机延后,进程读取TCP套接字时略延后。
# 设为1时,以TCP读取消息时,使应用进程减少延迟为优先。二者相反,默认是不打开这个开关的。
#via: http://bbs.chinaunix.net/thread-2071263-1-1.html
net.ipv4.tcp_low_latency = 1
# socket buffer portion used for TCP window
#/proc/sys/net/ipv4/tcp_adv_win_scale
#该文件表示计算缓冲开销bytes/2^tcp_adv_win_scale(如果tcp_adv_win_scale >; 0)或者bytes-bytes/2^(-tcp_adv_win_scale)(如果tcp_adv_win_scale <= 0)。
#cat net.ipv4.tcp_adv_win_scale = 1 这里的tcp_adv_win_scale意味着,将要拿出1/2缓存出来做应用缓存
net.ipv4.tcp_adv_win_scale = 1
# queue length of completely established sockets waiting for accept
# https://blog.51cto.com/wujianwei/2104779
net.ipv4.tcp_max_syn_backlog = 30000
# maximum number of sockets in TIME_WAIT state
net.ipv4.tcp_max_tw_buckets = 2000000
# reuse sockets in TIME_WAIT state when safe
#开启net.ipv4.tcp_tw_reuse可以复用TIME_WAIT的连接,但是要求服务器与客户端都必须开启net.ipv4.tcp_timestamps。
#这也是为了保证安全性,假如说没有net.ipv4.tcp_timestamps,当连接被复用后,如果这时候有延迟或者重发的数据包到达,新的连接就无法判断数据包属于复用前的连接还是复用后的连接了。所以需要客户端与服务器双方都开启net.ipv4.tcp_timestamps
#原文链接:https://blog.csdn.net/qq_42168438/article/details/121943726
net.ipv4.tcp_tw_reuse = 1
# time to wait (seconds) for FIN packet
#提高Linux应对短连接的负载能力, netstat -ant| grep -i time_wait |wc -l 可以看到time wait数量
net.ipv4.tcp_fin_timeout = 5
# disable icmp send redirects
net.ipv4.conf.all.send_redirects = 0
# disable icmp accept redirect
net.ipv4.conf.all.accept_redirects = 0
# drop packets with LSR or SSR
net.ipv4.conf.all.accept_source_route = 0
# MTU discovery, only enable when ICMP blackhole detected
net.ipv4.tcp_mtu_probing = 1
- linux performance tuning
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/pdf/performance_tuning_guide/red_hat_enterprise_linux-7-performance_tuning_guide-en-us.pdf
xargs 的一个选项 -I,使用 -I 指定一个替换字符串 {},这个字符串在 xargs 扩展时会被替换掉,当 -I 与 xargs 结合使用,每一个参数命令都会被执行一次:
cat arg.txt | xargs -I {} ./sk.sh -p {} -l
如何在wireshare中判别重传标记是否正确:
参考文档: https://zhuanlan.zhihu.com/p/656294847
HTTP 1.1中新增keepalive功能
https://www.jianshu.com/p/fd762b41486e