Objectives
- High Availability
- HA Modes
- FGCP (FortiGate Clustering Protocol)
- Heartbeat Interfaces and Virtual IP Interfaces
- HA Requirement
- Configure Primary FortiGate Firewall
- Configure Secondary FortiGate Firewall
- HA-Troubleshooting
What is High Availability?
High Availability (HA) is a feature of Firewalls in which two or more devices are grouped together to provide redundancy in the network. HA links and synchronizes two or more devices. In FortiGate HA one device will act as a primary device (also called Active FortiGate). The active device synchronizes its configuration with another device in the group. Other FortiGate devices are called Secondary or Standby devices.
Fortigate HA Modes
There are two Fortigate HA modes available:
- Active / Passive- Configuration of primary and secondary devices are in synchronization. In Active/Passive mode the primary device is the only equipment that can actively process the traffic. The secondary FortiGate device remains in Passive mode and monitors the status of the primary device. If the problem is detected in the Primary FortiGate, the secondary device takes over the primary role. This event is called HA Failover.
Active / Active-All HA configuration must be in synchronization. The only difference in Active / Active mode is that in A/A mode all the FortiGate devices are processing the traffic.
FGCP (FortiGate Clustering Protocol)
HA Protocol is used by FortiGate Cluster to communicate. FGCP travels between FortiGate cluster devices over the heartbeat links and uses TCP port 703 with Ethernet-type values:
- 0x8890 – NAT Mode
- 0x8891—transparent mode
FGCP uses TCP port 23 for configuration synchronization. The firewall cluster uses FGCP to elect the primary, synchronize configuration, discover another firewall that belongs to the same HA, and detect failover when any HA device fails.
In Active/Passive, Primary Firewall performs below tasks:
- Exchange heartbeat Hello messages with the secondary device over the control link
- Synchronizes routing table, DHCP information, running configuration
- Traffic sessions
The secondary Firewall performs below tasks:
- Monitor the Primary device to check if reachability is working in-between clusters or not
- If a problem is encountered with the Primary Firewall, a secondary device take-over the traffic sessions
- Maintain Data Plane Processes like Forwarding Table, NAT Table, Authentication record
Heartbeat Interfaces and Corresponding IP addresses
Virtual IP addresses are assigned to heartbeat Interfaces based on the serial number of the FortiGate Firewall
- 169.254.0.1—assigned to the highest serial number
- 169.254.0.2—assigned to the second-highest number
- 169.254.0.3—assigned to the third-highest number
The cluster uses these virtual IP addresses to differentiate cluster members and update configuration changes in clustered devices.
Fortigate HA Requirements
- Two to Four identical FortiGate Firewall (same Model )
- Same Licenses on all cluster member
- The physical link between Firewalls for a heartbeat
- DHCP and PPPoE interfaces are supported
Fortigate HA Configuration
Configuring Primary FortiGate for HA.
1. Go to System ->Select HA.
2. Select mode Active-Passive Mode.
3. Once Active-Passive mode is selected multiple parameters are required.
4. Mode- Active/ Passive
5. Set Device Priority -200. More numerical value higher the priority. Here Priority is set to 200, secondary devices must have a lower numerical value than Primary Firewall.
6. Device Group– The group name must be the same for both primary and secondary devices. Here we have given the name HA-GROUP. A device Group is used in HA to assign two or more devices to be part of the same HA Group.
7. Password – the same password must be provided to both primary and secondary Firewalls.
8. Heartbeat Interface—Add Port 3/HA1 and Port 4/ HA2 port in heartbeat interfaces through which both primary and secondary devices can interchange hello messages to check liveliness of the peer device.
9. Select OK.
- The FortiGate exchanges messages to peer devices to establish an HA cluster. When Admin selects OK connectivity can be lost with the FortiGate as the HA cluster negotiates and the FGCP initiates a new MAC address of the FortiGate interfaces.
- Power off the FortiGate.
- Repeat the steps in Secondary devices and connect Port 3 and Port 4 with Secondary FortiGate Firewall.
Configuring Secondary FortiGate for HA
Repeat Steps 1 to Step 9 in Secondary Firewall.
Check HA status in Secondary devices. Refresh the entries and check the sync status in the Primary and Secondary HA monitoring Dashboards.
FortiGate (global) # get sys ha status
HA Health Status: OK
Model: FortiGate-VM64-KVM
Mode: HA Active Passive
Group: HA-Group
Debug: 0
Cluster Uptime: 211 days 5:9:44
Cluster state change time: 2022-04-16 14:21:15
“Master selected using”
<2022/04/13 14:21:15> FGVMXXXXXXXXXX14 is selected as the master because it has the largest value of uptime.
<2022/04/13 14:15:46> FGVMXXXXXXXXXX16 is selected as the master because it has the largest value of uptime.
<2022/04/12 11:17:04> FGVMXXXXXXXXXX44 is selected as the master because it has the largest value of override priority.
ses_pickup: enable, ses_pickup_delay=disable
override: disable
“Configuration Status”:
FGVMXXXXXXXXXX14(updated 2 seconds ago): in-sync
FGVMXXXXXXXXXX16(updated 3 seconds ago): in-sync
“System Usage Stats”
FGVMXXXXXXXXXX14(updated 2 seconds ago):
sessions=12, average-cpu-user/nice/system/idle=0%/0%/0%/100%, memory=44%
FGVMXXXXXXXXXX16(updated 3 seconds ago):
sessions=2, average-cpu-user/nice/system/idle=0%/0%/0%/100%, memory=14%
HBDEV stats:
FGVMXXXXXXXXXX14(updated 2 seconds ago):
port3: physical/10000full, up, rx-bytes/packets/dropped/errors=2232258636/6463321/0/0, tx=3266257061/8035173/0/0
FGVMXXXXXXXXXX16(updated 3 seconds ago):
port3: physical/10000full, up, rx-bytes/packets/dropped/errors=3366612632/70886621/0/0, tx=1232321221/4564123/0/0
MONDEV stats:
FGVMXXXXXXXXXX14(updated 1 seconds ago):
port4: physical/10000full, up, rx-bytes/packets/dropped/errors=5543991879/3242247/0/0, tx=554325343/4321945/0/0
FGVMXXXXXXXXXX16(updated 3 seconds ago):
port1: physical/10000full, up, rx-bytes/packets/dropped/errors=22183223/2218321/0/0, tx=216832/1211/0/0
Master: Active-FW , FGVMXXXXXXXXXX14, cluster index = 1
Slave : Secondary-Fw , FGVMXXXXXXXXXX16, cluster index = 0
number of vcluster: 1
vcluster 1: work 169.254.0.2
Master: FGVMXXXXXXXXXX14, operating cluster index = 0
Slave : FGVMXXXXXXXXXX16, operating cluster index = 1
Check the checksum mismatch and compare for the cluster checksum. Run command to go in rough for discrepancy VDOM’s by using command:
diag sys ha checksum show <vdom>
diag sys ha checksum show <global>
Use grep to filter the configuration
diagnose sys ha checksum show root | grep system
diagnose sys ha checksum show global | grep log
Repeat the above commands on the secondary device to compare the mismatched output
Initiate and re-calculate checksum if no mismatch is found.
Command to re-calculate the checksum
diagnose sys ha checksum recalculate [<vdom-name> | global]
The above command re-calculates the checksum for all the devices.
Debug HA logs
diag debug app hasync 255
diag debug enable
execute ha synchronize start
diagnose debug application hatalk -1
communication between HA devices
Mismatch in HA can be calculated by using below command
1.diag debug config-error-log read
2. diag hardware device disk
3. show sys storage
4. show wanopt storage
