Omnia

Ansible playbook-based tools for deploying Slurm and Kubernetes clusters for High Performance Computing, Machine Learning, Deep Learning, and High-Performance Data Analytics

This project is maintained by dellhpc

Parameters in base_vars.yml

Parameters in base_vars.yml configure the common functionality of the control plane which includes information of the nics, dhcp ranges, pods cidr for k8s pod IP ranges, etc. This file is located in /control_plane/input_params

Variables [Required/ Optional] Default, Accepted values Description  
ansible_conf_file_path [Required] /etc/ansible Directory path with the Ansible configuration file (ansible.cfg). If Ansible is installed using pip, provide the directory path. If the path is not provided, it is set as /etc/ansible, by default.  
network_interface_type [Required] dedicated, lom This variable decides the network type set up by Omnia. LOM stands for LAN on Motherboard.
If lom is selected, idrac_support will be assumed to be true irrespective of user input.
 
awx_web_support [Required] false, true This variable decides whether AWX is to be installed by control_plane.yml. If the cluster is larger than 5 nodes, it is recommended to keep this variable false. If false, all playbooks will have to run via Ansible CLI. To toggle AWX support, set this variable to true (enable) or false (disable) and re-run control_plane.yml.  
device_config_support [Required] false, true This variable decides whether Management IPs are to be assigned to all devices.  
idrac_support [Required] false, true This boolean indicates that Management IPs are assigned and can be taken from device_list_path for communication.
If device_config_support is true, idrac_support has to be set to true.
 
device_ip_list_path [Optional]   File Path where all pre-assigned Management IPs are provided.
This field is mandatory if idrac_support is true.
A sample template is provided here: omnia/examples/device_ip_list.yml
Please ensure that all listed IPs are reachable to the control plane via the Public internet
 
ethernet_switch_support [Required] false, true This boolean decides whether Ethernet Switches are to be configured.
If network_interface_type=lom, Ethernet switch configuration is not supported.
 
ib_switch_support [Required] false, true This boolean decides whether Infiniband Switches are to be configured.
If network_interface_type=lom, Infiniband switch configuration is not supported. However, opensm will run if ib_switch_support=true
 
powervault_support [Required] false, true This boolean decides whether PowerVaults are to be configured.  
enable_security_support [Required] false, true This variable decides whether FreeIPA will be deployed and used on the Control Plane to control access and authorization. For more information, click here  
public_nic [Required] eno2 The NIC or Ethernet card connected to the public internet.  
appliance_k8s_pod_net_cidr [Required] 192.168.0.0/16 Kubernetes pod network CIDR for appliance k8s network. Ensure this value does not overlap with any of the host networks.  
snmp_trap_destination [Optional]   Enter an SNMP server IP address to receive SNMP traps from devices in the cluster. If this variable is left blank, SNMP will be disabled.  
snmp_community_name</br> [Required] public SNMP community name.  
awx_organization [Required] DellEMC Organization name configured in AWX.  
mount_location [Required] /opt/omnia/ The filepath where the grafana persistent volume will be created.
When telemetry.yml is run, both the timescaledb and the mysqldb are stored here.
 
provision_method [Required] PXE, idrac This variable is used to set node provisioning method
If device_config_support is true, PXE will be used.
Note : This parameter is case sensitive.
 
provision_state [Required] stateful If stateful is set, cobbler is used to provision disks. Stateless is currently not supported.  
provision_os [Required] rocky, centos , leap, redhat This is the operating system image that will be used for provisioning compute nodes in the cluster.  
iso_file_path [Required] /home/Rocky-8.x-x86_64-minimal.iso Provide the relevant ISO file path. This ISO file is used by Cobbler to provision the OS on the compute nodes.
The iso file should be ** Rocky8-Minimal ** or ** CentOS7-2009-minimal ** or ** openSUSE-Leap-15.3-DVD edition ** or ** RHEL-8.x-DVD edition ** .
Note: After running control_plane.yml, it is recommended that the ISO image file is not renamed. And, you must not change the path of this ISO image file as the provisioning of the OS on the compute nodes may be impacted.
 
timezone [Required] GMT Enter a timezone that is set during the provisioning of OS. GMT is set as the default time zone. You can set the time zone to EST, CET, MST, CST6CDT, or PST8PDT. For a list of available time zones, see the appliance/common/files/timezone.txt file.  
language [Required] en-US Set the language used during the provisioning of OS. By default, it is set to en-US.  
default_lease_time [Required] 86400 (24 hours), 21600 (6 hours)- 31536000 (1 Year). Measures (in seconds) the time period an IP is reserved for an assigned NIC.  
host_network_nic [Required] eno3 NIC or Ethernet card that is connected to the Host Network. By default, it is set to “eno3”.
If roce_network_nic is not available, provisioning information will be routed through this NIC.
 
host_network_dhcp_start_range [Required] 172.19.0.100 Start IP for the DHCP range used by the Host Network to assign IPv4 addresses.  
host_network_dhcp_end_range [Required] 172.19.0.200 End IP for the DHCP range used by the Host Network to assign IPv4 addresses.  
primary_dns [Optional]   The primary DNS host IP queried by Cobbler to provide Internet access to Compute Node (through DHCP routing)  
secondary_dns [Optional]   The secondary DNS host IP queried by Cobbler to provide Internet access to Compute Node (through DHCP routing)  
host_mapping_file_path [Optional]   Enter the file path containing a host mapping file with the MAC addresses, hostnames, IP addresses, and component role. A mapping_host_file.csv template file is provided here. Enter the details in the order: MAC address, Hostname, IP address, Component_role. For example, 10:11:12:13,server1,100.96.20.66,compute, 14:15:16:17,server2,100.96.22.199,manager, 18.19.20.21,server3,100.96.23.67,nfs_node, and 22.23.24.25,server4,100.96.23.75,login_node are all valid entries. The Hostname should not contain the following characters: , (comma), . (period), and - (hyphen). Ensure that you do not provide any duplicate entries in the file.
Another approach to filling this file maps IPs and MAC addresses based on the provided example.
If roce_network_nic is provided, the host_mapping_filepath is disregarded (That is, static IP assignment cannot take place).
 
mngmnt_network_nic [Optional] eno1 NIC or Ethernet card that is connected to the Management Network to provision the devices. By default, it is set to “eno1”.  
mngmnt_network_dhcp_start_range [Optional] 172.19.0.100 Start IP for the DHCP range used by the Management Network to assign IPv4 addresses.  
mngmnt_network_dhcp_end_range [Optional] 172.19.0.200 End IP for the DHCP range used by the Management Network to assign IPv4 addresses.  
mngmnt_mapping_file_path [Optional]   The file path containing a device mapping file with the MAC addresses and respective IP addresses. A mapping_device_file.csv template file is provided here Enter the details in the order: MAC address, IP address. For example, 10:11:12:13,1.2.3.4, 14:15:16:17,2.3.4.5, and 18.19.20.21,3.4.5.6 are all valid entries. Ensure that you do not provide any duplicate entries in the file.  
roce_network_nic [Optional]   The NIC/ Ethernet card used by Omnia for high speed pxe provisioning assuming a pre-existing RoCe network setup.  
roce_network_dhcp_start_range [Optional]   The start IP of the IP range used for DHCP provisioning of IPv4 addresses  
roce_network_dhcp_end_range [Optional]   The end IP of the IP range used for DHCP provisioning of IPv4 addresses  
ib_network_nic [Optional]   The NIC/Ethernet card to be used when configuring Infiniband switches.
This parameter is mandatory if ib_switch_support is true.
 
ib_network_dhcp_start_range [Optional]   The start IP for the range within which the Infiniband Switch IP is to be configured.
This parameter is mandatory if ib_switch_support is true.
 
ib_network_dhcp_end_range [Optional]   The end IP for the range within which the Infiniband Switch IP is to be configured.
This parameter is mandatory if ib_switch_support is true.