Network Configuration and Services

This section describes the basic network configuration and services of a Qlustar cluster.

Basic Network Configuration

The IP configuration of the head-node is defined in the file /etc/network/interfaces (type man interfaces for details), while /etc/resolv.conf is the key-control file regarding the DNS configuration (see DNS). Network interfaces can be brought up or down using the commands ifup and ifdown (see also the corresponding man pages). Typically, the head-node has at least two network interfaces, one for the external LAN and one for the internal cluster network. For all cluster-internal networks (boot/NFS, optional Infiniband and/or IPMI)Network Configuration cluster-internal networks, unofficial (not routed) IP addresses are used, usually in the range 192.168.x.0/255.255.255.0, while for larger clusters the range 172.16.y.0/255.255.0.0 is often used. In the latter case, y might indicate the rack number.

The cluster-internal network ranges can be conveniently chosen during installation.

DNS

This section describes how to configure the Domain Name System (DNS). The file /etc/resolv.conf contains the addresses of the DNS servers to use and a list of domain names to search when looking up a short (not fully qualified) hostname. Example:

search your.domain
nameserver 127.0.0.1

where 127.0.0.1 is the localhost address and your.domain is the DNS domain name.

The file /etc/resolv.conf is automatically generated. The DNS configuration must be configured directly in /etc/network/interfaces by adding dns- option lines to the relevant iface stanza. The following option names are supported: dns-nameservers, dns-search, and dns-sortlist. Get more details on these options by executing man resolvconf. The dns-nameservers entry is added and configured automatically to use 127.0.0.1 (the head-node’s dnsmasq instance) during installation.

Starting from Qlustar 11.0, all DNS configuration is done via QluMan and dnsmasq is used as a cluster-wide DNS server including proxy support.

DHCP

The basic IP configuration of the compute-nodes is done using the Dynamic Host Control Protocol (DHCP). The DHCP server is not only responsible to automatically supply the IP address and netmask to these nodes, but also additional configuration options like the gateway and DNS server addresses, DNS domain name and many other parameters. Starting from Qlustar 11.0, dnsmasq is also used to provide DHCP services. The corresponding configuration file is /etc/dnsmasq.d/01-dhcpd.conf and is auto-generated by QluMan.

If a cluster has additional internal networks (e.g. Infiniband), the IP address of a node in that network is derived from its basic DHCP address and set automatically during boot. The addresses of additional networks can be specified during installation and in QluMan. Check the QluMan Guide for more details.

IP Masquerading (NAT)

To allow direct TCP/IP connections for the compute-nodes to machines outside of the internal cluster network, IP masquerading (NAT) is configured by default on the head-node(s) during installation. This might be necessary e.g., when applications running on the compute-nodes need to contact a license server in the public LAN.

All IP packets with unofficial sender IP addresses and a destination in the public LAN are then translated by the head-node to packets with its own official IP address. When a reply packet arrives, it is translated back to the unofficial IP address of the originating node inside of the cluster. The head-node works as a router in this case. The following example section in /etc/network/interfaces (cluster network address is 192.168.97.0/24) shows, how masquerading is activated on boot and disabled on shutdown:

iface br-ext inet static
  address 192.168.55.44
  netmask 255.255.255.0
  broadcast 192.168.55.255
  gateway 192.168.55.254
  dns-nameservers 127.0.0.1
  bridge_ports enp0s4
  bridge_stp off
  bridge_fd 0
  bridge_maxwait 0
  up iptables -t nat -A POSTROUTING -s 192.168.97.0/24 \
    -o br-ext -j MASQUERADE
  down iptables -t nat -D POSTROUTING -s 192.168.97.0/24 \
    -o br-ext -j MASQUERADE

Time Server

Synchronized system time throughout the cluster is crucial for its flawless operation. It is achieved using the Network Time Protocol (NTP) daemon. If the head-node has direct Internet access, publicly available time-servers on the Internet can be contacted and used as an accurate time reference. In order to set a list of time-servers, edit the file /etc/ntp.conf and add a line for every ntp-server (example ntp-1) to be contacted:

server ntp-1