Mellanox OFED Installation With Lustre

The version used on Rocky Linux 8.9 (subject to change if kernel is upgraded)

MLNX_OFED_LINUX-5.8-3.0.7.0-rhel8.8-x86_64

Lustre client version lustre-client-2.12.9_ddn8-1.el8.x86_64

Original open sources from Whamcloud were not passing the compilation, client used on SOMHPC was provided by DDN.

DDN version DDN Customer Portal

https://www.ddn.com/solutions/lustre-multi-level-security/

( updated version of lustre client for DDN can be acquired by contacting DDN customer support if OS kernel changed for security reason)

Installation procedure:

1.     Log in to the installation machine as root.

2.     Mount the ISO image on your machine.

 mount -o ro,loop MLNX_OFED_LINUX-<ver>-<OS label>-<CPU arch>.iso /mnt

3.     Run the installation script.  /mnt/mlnxofedinstall

(Based on DDN solutions in our case) assumes non-standard compilation for the Mellanox OFED driver, don’t forget to add --add-kernel-support --kmp --without-fw-update

4.     rebuild the initramfs

cp /boot/initramfs-$(uname -r).img /boot/initramfs-$(uname -r).bak.$(date +%m-%d-%H%M%S).img

dracut -f -v

5.     # Extract Lustre sources into that directory

tar xvzf lustre-client-${version}.tar.gz -C lustre-${version}_MOFED-${mofed_version}_$(uname -r) --strip-components=1

# Lustre client RPMs build

cd lustre-${version}_MOFED-${mofed_version}_$(uname -r)

./configure --disable-server

make rpms