Configuring and tuning Servers for low-latency applications

Configuring and tuning Servers for low-latency applications

是以hp服务器为模板，其实很多都是可以借鉴得通用得方法  主要是从以下几个方法进行调优

Hardware—System design, processor type and speed; memory latency, speed, and capacity; network components; and storage subsystem, including SSDs
• OS selection—Operating system kernels specifically designed and tuned for minimum latency and, in some cases, real-time preemption
• BIOS configuration—BIOS support configured for minimum latency and maximum performance
• Networking fabric—Network technology (1/10/40 Gigabit Ethernet, InfiniBand, Fibre Channel)
• Middleware—Messaging and database services on the network designed for minimum latency and maximum throughput with reliability
• End-user applications—Designed to perform multicast messaging accelerated via kernel bypass and RDMA techniques
• Physical distances—Physical separation between the information sources and clients affects overall system performance.

我们看看OS得调优部分，硬件每家都是不一样得

1.For Red Hat Enterprise Linux Server 7.0 or greater:
Edit /etc/default/grub file and add "nosoftlockup intel_idle.max_cstate=0 mce=ignore_ce" to the “GRUB_CMDLINE_LINUX“value.
Run command:
# grub2-mkconfig -o /boot/grub2/grub.cfg (non-UEFI configurations) or
# grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg (UEFI configurations)
nosoftlockup prevents the kernel from logging an event when a high-priority thread executes continuously on a core for longer than the soft lockup threshold.
intel_idle.max_cstate=0 prevents the kernel from overriding the BIOS C-State setting.
mce=ignore_ce prevents Linux from initiating a poll every five minutes of the Machine Check Banks for correctable errors, which can cause latency spikes. For more information, see the Linux Kernel Archives website

(http://www.kernel.org/doc/Documentation/x86/x86_64/boot-options.txt).

 

2. Set tuned profile (RHEL only). Tuned is a utility introduced in RHEL 6 that allows the user to implement a set of OS optimizations as part of a profile. Red Hat provides a set of pre-defined profiles that can be used. For RHEL 6, it is recommended that the “latency-performance” profile is used for latency-sensitive applications. For RHEL 7, “network-latency” is recommended for low latency environments. For details on the performance options being set, please see the /usr/lib/tuned/[performance profile]/tuned.conf file for the desired performance profile. To set the desired profile, run the command:
# tuned-adm profile latency-performance (RHEL 6) or
# tuned-adm profile network-latency (RHEL 7)

 

3. Reboot the server.
4. After reboot, run the stop-services.sh script to stop extraneous services. The following example stops the services shown and prevents them from starting on subsequent boots:

5. Use the irqbalancer to preclude some cores from servicing software IRQs:
a) Enter the following command:
# service irqbalance stop
b) Do a one-time run of the irq balancer:
# IRQBALANCE_ONESHOT=1 IRQBALANCE_BANNED_CPUS=${CoreMask} irqbalance
c) Wait until the command service irqbalance status returns "irqbalance is stopped."
d) On SLES, the name of the IRQ balancer service is irq_balancer.
e) On RHEL 7, use systemctl instead of service command to stop irqbalance.
Recommended Linux boot-time settings
The Linux boot parameter "idle=poll" keeps the processing cores in C0 state when used in conjunction with "intel_idle.max_cstate=0." Without it, the processor will enter C1 state.

For RHEL 7, edit /etc/default/grub.cfg and add "idle=poll". This is in addition to the "nosoftlockup intel_idle.max_cstate=0 mce=ignore_ce" parameters that should have been added previously. After the edit, run the command:
# grub2-mkconfig -o /boot/grub2/grub.cfg (non-UEFI configurations) or
# grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg (UEFI configurations)

On RHEL 7, use “systemctl” to disable and monitor the status of the” irqbalance.service” process.
Consider the following:
• Consider changing the smp_affinity for the IRQs. For example, on a server on which you want to leave core 0 for the OS, the following masks off the other processors for all IRQs:
for MF in `find /proc/irq -name *smp_affinity` ; do awk -F, \
'{for(i=1;i<NF;i++)printf("00000000,");printf("%8.8x\n",and(0x00000001, strtonum("0x"$NF)))}' \
$MF > $MF ; done
• Consider using cset (http://code.google.com/p/cpuset/) to shield cores from the OS. For example, on a server on which you want to keep the OS from all cores except for 0, use the following command:
# cset shield --cpu 1-15 --kthread=on
• If running as root, the following command can then be used to move the current PID to the "user" set of cores:
# cset proc --move --pid=$$ --threads --toset=user