Server time slow

Another issue that popped up tonight. The time on a payroll server seem to be slower then usual. Futher troubleshooting on the box revealed that it took 4 seconds to move 1 second on the server. This caused the payroll servers to stop communicating between each other as time sync was part of a security measure built into the payroll software we run here.

Some googling later it seem to be a BUG with the kernel. The fix, suggested to either update the kernel, recompile the kernel or add some kernel parameters in GRUB to fix the issue. I decided to go with the kernel parameters because this was a production server and the downtime window was very slim.

The fix,

1. vi /etc/boot/grub.conf
2. Add to the end of the kernel line. “clock=pit noapic nolapic”
3. reboot and check time. “watch date”

# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE:  You have a /boot partition.  This means that
#          all kernel and initrd paths are relative to /boot/, eg.
#          root (hd0,0)
#          kernel /vmlinuz-version ro root=/dev/md7
#          initrd /initrd-version.img
#boot=/dev/sda
default=0
timeout=10
splashimage=(hd0,0)/grub/splash.xpm.gz
title Red Hat Linux (2.4.9-e.3smp)
root (hd0,0)
kernel /vmlinuz-2.4.9-e.3smp ro root=/dev/md7 clock=pit noapic nolapic
initrd /initrd-2.4.9-e.3smp.img
title Red Hat Linux Advanced Server (2.4.9-e.3)
root (hd0,0)
kernel /vmlinuz-2.4.9-e.3 ro root=/dev/md7 clock=pit noapic nolapic
initrd /initrd-2.4.9-e.3.img

Source: Rexiology::Work

Source: Linux Kernel Parameters