SSH Compromise Detection Using Flow Data

This article discusses how the flow-based compromise detection can be applied to SSH. It is based on the excellent thesis “Flow-based Compromise Detection” written by Richard Hofstede.

Picture 1: Phases of SSH Brute-force Attack

The attack is conducted from an IP address 10.10.10.10 to the network 195.165.10.0/24, a destination TCP port 22 (SSH). Each TCP SYN scan against a single IP address represents a single flow record, reaching a maximum PPF of 2. The majority of flows contains PPF 1, indicating a closed destination TCP port 22. In other words, the flow represents a TCP SYN packet sent from an attacker to the targeted host. In case an SSH daemon is running on the host, the TCP port 22 is open and PPF is 2. The flow counts an initial TCP SYN packet and consecutive TCP RST packet sent from an attacker to the targeted host. These findings are consistent with the thesis of Richard Hosfstede, who claims that during the scan phase the attack is conducted from a single IP address to a large number of targets, generating many small flows. Therefore, he defines an upper limit of 2 PPFs for the scan phase detection algorithm. Unlike the attack traffic, regular SSH connections will use the same TCP connection once established, which results in a higher value for the PPF metric. However, if only PPF is used for scan phase detection, there would be many false positives for SSH sessions involving sporadic activity. As the author explains, flow monitoring devices would export long-lived flows as multiple flow records due to the use of NetFlow timeouts. Based on his measurement, Hosfstede defines a threshold for the number of flow records per attack in the scan phase. It equals to a 200 flow per time interval of 1 minute that corresponds to 200 SSH connection attempts per minute.

As soon as the targets are identified, an attacker starts the brute-force phase, flooding SSH servers with many authentication attempts. Usernames and passwords are either generated and their length increases with the number of failed login attempts or they are read from dictionaries. In contrast to a scan phase, where the flow contains up to 2 PPF, the brute-force phase is characterized by a significantly higher number of PPF, due to the SSH connection initiation and one or more authentication attempts. This significantly distinguishes the brute-force phase from the scan phase. Hofstede suggests the PPF range from 11 to 51, where 11  is the minimum number of packets needed for a single authentication attempt, and 51 – the highest number of PPF observed for the brute-force phase traffic. These findings correspond to the measured PPF values of the brute-force phase of our simulated SSH attack (Picture 2).

Picture 2: Flow Records of the Brute-Force Phase of an SSH Attack

Hofstede’s algorithm starts with preselection of source and destination IP address pairs for which flow records have a PPF value of between 11 and 51. For each of these preselected address pairs, the most frequently used PPF value is taken as the baseline for determining brute-force behavior. This baseline is then used for comparing consecutive flow records with identical PPF values. If at least a number of consecutive flows feature the baseline number of PPF, a brute-force attack is recognized. He sets the threshold number to five, so five consecutive flow records with the same number of PPF would represent 15 failed login attempts. If more than 5 flow records feature the same number of PPF, a brute-force attack is detected. Picture 3 depicts the exported flow records after the 15 failed login attempts. In this case, Hofstede’s algorithm is successfully used for the detection of SSH brute-force phase.

Picture 3: Detected Brute-force Phase Against an SSH Server

Hofstede’s method, however, could lead to undetected attacks or false negatives when PPFs differ and the threshold is not reached. PPF deviation is the effect of a TCP re-transmissions caused by delays and double acknowledgments due to packets reordering. In this case, additional packets and bytes are counted to flows, so the same failed login attempts generate different PPFs. For this reason, deviations in the number of PPF must be taken into account. Therefore, he measures TCP re-transmissions and control information packets, storing related counters in the flow cache. The compensated number of PPF is then the difference between the total number of PPF per flow minus the number of packets of all re-transmissions and TCP control information fields.

When an attacker gains access to the targeted host using a correct combination of username and password, the compromise phase is launched.  The phase distinguishes from the brute-force phase by either a higher volume of PPF when the system is actively misused (e.g for DDoS) or lower PPF when connection to the targeted host is only maintained. In both cases, the threshold for the detection of compromise phase depends on the baseline set for the brute-force phase.

Conclusion:

Flow-based compromise detection definitely applies to SSH. It represents a viable solution in terms of the accuracy and scalability as NetFlow aggregates individual packets into flows. Moreover, NetFlow is cost-effective, as it is very likely that it is already enabled on your network devices so no additional capturing devices are needed. However, we need to compensate deviations in the number of PPF for SSH brute-force phase due to the fact that TCP traffic is not flat. For this reason, NetFlow must be modified to store the relevant counter of re-transmissions and TCP control information so a compensated number of PPF can be used for detecting SSH compromises.