Fail2nft: Difference between revisions

From Coolscript
Jump to navigation Jump to search
No edit summary
 
(37 intermediate revisions by the same user not shown)
Line 1: Line 1:
=Why Fail2Nft=
=Why Fail2Nft=
<br>'''Fail2Nft tracks your logs against unauthorized logins and blocks the sender ip address by condition for a specified amount of time.<br><br>'''
<br>'''Fail2Nft offers a simple Intrusion Prevention System (IPS) and more and can be easily installed<br><br>'''
Fail2Nft is a lightweight perl script solution which aims to keep out of unwanted ssh login attemps from people or robots on your system. <br>
Fail2Nft is a lightweight [https://www.perl.org/ perl] solution which aims to keep out of unwanted ssh login attemps from people or robots on your system. <br>
If ssh logins attempt to continue with wrong user/password combinations then fail2nft can block the ip address for a specified amount of time.<br>
If ssh logins attempt to continue with wrong user/password combinations then Fail2Nft can block the ip address for a specified amount of time.<br>
*Fail2Nft keeps the setup as simple as possible, the internet installer can setup your server with a very few steps with pre-configured packages.
*Fail2Nft keeps the setup as simple as possible, the internet installer can setup your server with a very few steps with pre-configured templates.
*Fail2Nft has been prooven to run on recent Linux Platforms which are on [https://en.wikipedia.org/wiki/APT_(software) APT] but Fail2Nft can run on any other recent [https://en.wikipedia.org/wiki/Linux_Standard_Base LSB] distros shipped with nftables as well.
*Fail2Nft has been prooven to run on recent Linux Platforms which are on [https://en.wikipedia.org/wiki/APT_(software) APT] but Fail2Nft can run on any other recent [https://en.wikipedia.org/wiki/Linux_Standard_Base LSB] distros shipped with nftables as well.
*Fail2Nft has been designed to run on single instances such as single cloud machines or application servers running ssh, mail or ftp services,<br>Fail2Nft is currently not made for firewalls or routers but can be adapted to work on those environments as well.This is because the setup process would becomes too complex
*Fail2Nft has been designed to run on single instances such as single cloud machines or application servers running ssh, mail or ftp services,<br>Fail2Nft is currently not made for firewalls or routers but can be converted to work on customized environments too.
*Fail2Nft is an update of [http://test.coolscript.org/index.php/Syslog_to_Firewall Syslog_to_Firelwall]  it follows the same idea but instead of iptables we use nftables in combination with named sets. <br>
*Fail2Nft is an update of [http://test.coolscript.org/index.php/Syslog_to_Firewall Syslog_to_Firelwall]  it follows the same idea but instead of iptables we use nftables in combination with named sets. <br>
<br><br>
<br><br>
Line 12: Line 12:
*Fail2Nft is based on [https://wiki.nftables.org/wiki-nftables/index.php/Main_Page nftables] together with [https://wiki.nftables.org/wiki-nftables/index.php/Sets#Named_sets_specifications  named sets].
*Fail2Nft is based on [https://wiki.nftables.org/wiki-nftables/index.php/Main_Page nftables] together with [https://wiki.nftables.org/wiki-nftables/index.php/Sets#Named_sets_specifications  named sets].
*Fail2Nft handles ip [https://en.wikipedia.org/wiki/IPv4 v4] and [https://en.wikipedia.org/wiki/IPv6 ip v6] addresses automatically
*Fail2Nft handles ip [https://en.wikipedia.org/wiki/IPv4 v4] and [https://en.wikipedia.org/wiki/IPv6 ip v6] addresses automatically
*Fail2Nft can open all ports to authenticated login IP's, the idea is similar to [https://en.wikipedia.org/wiki/Port_knocking '''Port Knocking'''] but using ssh authentication instead of port knocking to allow access.
*Fail2Nft can open all ports to authenticated login IP's, the idea is similar to [https://en.wikipedia.org/wiki/Port_knocking '''Port Knocking'''] but using ssh authentication instead of port knocking to allow unlimited access.
*Dynamic increasing of lock times, optional based on [https://en.wikipedia.org/wiki/Country_code Country] or [https://en.wikipedia.org/wiki/Autonomous_system_(Internet) ASN]
*Dynamic increasing of lock times, optional based by [https://en.wikipedia.org/wiki/Country_code Country] or [https://en.wikipedia.org/wiki/Autonomous_system_(Internet) ASN]
*Plugin based development, current available plugins:
*Plugin based development, current available plugins:
**[https://de.wikipedia.org/wiki/Secure_Shell Auth]
**[https://de.wikipedia.org/wiki/Secure_Shell Auth]
**Mail - Imap/pop
**Mail - Imap/pop
**FTP
**FTP
**Grafana
*Syslog forwarding to [https://www.splunk.com/ Splunk] friendly key/value messages
*Syslog forwarding to [https://www.splunk.com/ Splunk] friendly key/value messages
*Automaic reinitialization to the previous last known state, for example in case of a reboot
*Automaic reinitialization to the previous last known state, for example in case of a reboot
*Whitelist support
*Whitelist ip address support
*XML configuration schema  
*XML configuration schema  
*SQLite Database
*SQLite Database, we create the database and tables automatically and run periodic maintenance on it. No need to install tools for this.
**'''No database maintenance required, everything will happen automatically'''
*Optional event mail  
*Optional event mail  
*Sressless installer available
*Sressless installer available
<br>
<br>


=Proven Platforms=
=Tested Platforms=
Fail2Nft works currently on Linux [https://en.wikipedia.org/wiki/APT_(software) APT] platforms and has been tested on  
The Fail2Nft installer works currently on Linux [https://en.wikipedia.org/wiki/APT_(software) APT] platforms only and has been tested on  
*Debian 9 Stretch  
*Debian  
*Debian 10 Buster
**9 (Stretch)
*Ubuntu 18
**10 (Buster)
**11 (Bullseye)
**12 (Bookworm)
*Ubuntu  
**18 (Bionic Beaver)
**20 (Focal Fossa)
**21 (Hirsute Hippo)
**22 (Jammy Jellyfish)
**24 (Noble)
<br>
 
 
Attention Ubuntu 24 and Debian 12 Users, please check the date time format in your logs, we require the traditional log time format,
to activate you may set to '''/etc/rsyslog.con''':
'''$ActionFileDefaultTemplate  RSYSLOG_TraditionalFileFormat'''
 
*Raspbian
**10
**11
 
<br>
<br>


=Internet Installer=
=Tested Cloud Environments=
'''Caution''' If you run already a netfilter solution on your server then watch your choice during the installation as this could lead into a malfunction system<br><br>
*AWS
The installer performs the entire steps needed to get a working fail2nft setup.<br>
*Azure
This includes the download of all files needed, installing all packages needed, create directories, create sym links, create the iinit script and setup crontab as well.<br>
*Digital Ocean
*Hetzner
 
=Easy Installer=


Syntax:
  fail2nft-installer.sh
  -h Optional Flag, Display this help
  -a Mandatory Flag, Automatic installation
  -i Optional Flag, Allow icmp
  -s Optional Flag, Install advanced syslog modules
  -t Optional String, tcp ports (comma seperated)
  -u Optional String, udp ports (comma seperated)
  -v Mandatory String, IP version (4 or 6 or both 4,6) valid only if -i or -t or -u is given
  -o Optional Flag,  OpenVPN/Enable
  -e Optional String, OpenVPN Interface Name eg eth0
  -k Optional String, OpenVPN Protocol (tcp or udp)
  -m Optional Int,    OpenVPN Port to masquerade (snat)
  -n Optional String, OpenVPN Network, eg 10.8.0.1\/24


*Run the following installer command to install Fail2Nft on your system:
*Run the following installer command to install Fail2Nft on your system, that is the '''Default Installation Method'''
  wget -q https://coolscript.org/download/fail2nft-installer.pl -O /tmp/fail2nft-installer.pl && perl /tmp/fail2nft-installer.pl
wget -q https://coolscript.org/download/scripts/fail2nft/fail2nft-installer.sh -O /tmp/fail2nft-installer.sh && bash /tmp/fail2nft-installer.sh -a
*Manual Sample
bash fail2nft-installer.sh -a -t 80,443,25 -u 53  -v 4  -n 10.0.8.0\\/24  -e eth0 -m 1994 -k tcp -o
 
==Sample: Advanced Installation==
configure additional tcp ports (80,443), udp port (1194), icmp and prepare the usage for OpenVPN<br>
additional install syslog modules and set the reinstall flag
  wget -q https://coolscript.org/download/scripts/fail2nft/fail2nft-installer.sh -O \
/tmp/fail2nft-installer.sh && bash \
/tmp/fail2nft-installer.sh \
-a \
-e eth0 \
-n 192.168.200.0\\/24 \
-o \
-r \
-s \
-t 80,443,1194 \
-u 1194  \
-v 4


<br><br><br>
=Command Line Parameters=
=Command Line Parameters=


Line 202: Line 256:
  Nov  9 20:22:07 myserver.com fail2nft[26563]: M=F2N LOCK=1 IP=a.b.c.d TIMER=600 COUNTRY=XX ASN=ASxxxx LOG=AUTH
  Nov  9 20:22:07 myserver.com fail2nft[26563]: M=F2N LOCK=1 IP=a.b.c.d TIMER=600 COUNTRY=XX ASN=ASxxxx LOG=AUTH


==Geo Functions==
===Rsyslog===
Fail2Nft is using an optional third party service to resolve the country and asn of the sender ip, if this function is depreciated then simply set the value to '''0''' or delete the entire element<br>
 
Note that the geo server runs on limited resources, response time can be high as this is a private and small equipped cloud server.
To log the Splunk optimized message in rsyslog you may add a rule for this
*Geoip_URL="https://xml.coolgeo.org/?myip=%IP%"
 
You may build this service by your own too, Fail2Nft expects the following xml schema to resolve ip to geo data:
*/etc/rsyslog.conf
#Enable udp listening
module(load="imudp")
input(type="imudp" port="514")
...
...
#Redirect nftables losg  to it's own log
:msg,regex,"IN=.*OUT=.*SRC=.*DST="     -/var/log/firewall.log
& stop
...
#Redirect Splunk optimized messages to it's own log
:msg,regex,"M=F2N.*"     -/var/log/fail2nft-splunk.log
 
*Required date format
Make sure that your date format in /var/log/auth.log is set to something like '''Jun 16 18:08:07'''
otherwise try the following setting in rsyslog.conf:
 
 
 
###########################
#### GLOBAL DIRECTIVES ####
###########################
# Use traditional timestamp format.
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
 
*Then Reset rsyslog
systemctl stop rsyslog
rm /var/log/auth.log
systemctl start rsyslog


<Client>
==IP2Country / IP2ASN==
  <IP>a.b.c.d</IP>
Fail2Nft supports a few IP to Geo provider to allow to retrieve the IP Country or ASN, <br>
  <COUNTRY>XX</COUNTRY>
this information can be optional used to setup individual lock times. <br>
  <ASN>ASxxxxxx</ASN>
</Client>


Note that this function has some kind of DOS protection inside, on the client site the service gets automatically disabled if timeouts occure three times in row.<br>
'''Available provider:'''
The service becomes than automatically active again after the daily maintenance task at midnight.<br><br>
 
{| class="wikitable " border="1"
|+ IP2Country Provider
|-
! scope="col" | Provider Name
! scope="col" | Limitation
! scope="col" | Require Registration
! scope="col" | Require API-KEY
! scope="col" | List ASN
! scope="col" | Is Accurate
|-
|  [https://app.abstractapi.com abstractapi] || 20.000/Month<br>Max 1 per second || Yes || Yes || Yes || Yes
|-
|  [http://wiki.coolgeo.org coolgeo.org] || 200/Day || No || No || Yes || Inhouse DB <br>From 2022
|-
|  [https://ipapi.co/ ipapi.co] || 30.000/Month || No || No || Yes || Yes
|-
|  [https://ipstack.com ipstack.com] || 5.000/Month || Yes || Yes || No || Yes
|-
|  [https://ip2loc.com ip2loc.com] || 15.000/Month || Yes || Yes || No || Yes
|-
 
|}
 
Note that the default is set to '''ipapi.co''' while on older installations '''coolgeo.org''' is used


==Maintenance==
==Maintenance==
Daily and monthly maintenance is automatically performed. this is, of course if Fail2Nft runs out of crontab frequently. <br>
Daily and monthly maintenance is automatically performed. This is, of course if Fail2Nft runs out of crontab frequently. <br>
Maintenance happens daily at midnight (day change). The Maintenance includes:
Maintenance happens daily at midnight (day change). The Maintenance includes:
* Removing old ip records, the condition is set within fail2nft.xml - Delete_Inactive_Records (in seconds)
* Removing old ip records, the condition is set within fail2nft.xml - Delete_Inactive_Records (in seconds)
Line 226: Line 330:


=Performance/Tweaks=
=Performance/Tweaks=
Fail2Nft has been tested with up to 50000 Records with no noticeable impacts on cpu or memory usage, that applies even to a Raspberry v4.<br>
Fail2Nft has been tested with up to 50000 Records with no noticeable impacts on cpu or memory usage, that applies even to a Raspberry-V4.<br>
However, if performance matters then consider to reduce the size of logs, for example /var/log/auth.log is kept up to 6 days by default.<br>
However, if performance matters then consider to reduce the size of logs, for example /var/log/auth.log is kept up to 6 days by default.<br>
Depending on ssh logon frequency this log can grow up which causes the delays for Fail2Nft. You can mitigate this by reducing the archiving time<br>.
Depending on ssh logon frequency this log can grow up which causes then delays for Fail2Nft. You can mitigate this by reducing the archiving time<br>.
*Example for '''/var/log/syslog'''
*Example for '''/etc/logrotate.d/syslog'''
  {
  {
         rotate 1
         rotate 1
Line 291: Line 395:
|-
|-


! style="font-size:13px;text-align: left;background-color:#efefef; color:#000000;" | Geoip_URL
! style="font-size:13px;text-align: left;background-color:#efefef; color:#000000;" | GeoIP_NAME
! style="font-size:13px;text-align: left;background-color:#efefef; color:#000000;" | If specified then we use the URL / Service to resolve the Country and ASN of the IP address
! style="font-size:13px;text-align: left;background-color:#efefef; color:#000000;" | If set then we use the specified service to resolve the Country and ASN of the IP sender address <br>
Available Provider to set: <br>
* abstractapi.com
* ipapi.co
* ipstack.com
* ip2loc.com
* coolgeo.org
! style="font-size:13px;text-align: left;background-color:#efefef; color:#000000;" | String
|-
 
! style="font-size:13px;text-align: left;background-color:#efefef; color:#000000;" | GeoIP_KEY
! style="font-size:13px;text-align: left;background-color:#efefef; color:#000000;" | Specify the API Key for the above (GeoIP_NAME) service, keys are required for:<br>
* abstractapi.com
* ipstack.com
* ip2loc.com
! style="font-size:13px;text-align: left;background-color:#efefef; color:#000000;" | String
! style="font-size:13px;text-align: left;background-color:#efefef; color:#000000;" | String
|-
|-
! style="font-size:13px;text-align: left;background-color:#efefef; color:#000000;" | GeoIP_Connect_Failure_Max
! style="font-size:13px;text-align: left;background-color:#efefef; color:#000000;" | Specify the amount of max failures per day for this provider, if not specified then we set the default to 3
! style="font-size:13px;text-align: left;background-color:#efefef; color:#000000;" | Integer
|-


! style="font-size:13px;text-align: left;background-color:#efefef; color:#000000;" | Delete_Inactive_Records
! style="font-size:13px;text-align: left;background-color:#efefef; color:#000000;" | Delete_Inactive_Records
Line 378: Line 503:


{| class="wikitable"
{| class="wikitable"
|+ Attributes for the Logs Element
|+ Attributes for the Logs Element, Note that auth.log is always enabled
! style="font-size:14px;text-align: left;background-color:#f8ff00; color:#000000;" | Atrribute
! style="font-size:14px;text-align: left;background-color:#f8ff00; color:#000000;" | Atrribute
! style="font-size:14px;text-align: left;background-color:#f8ff00; color:#000000;" | Description
! style="font-size:14px;text-align: left;background-color:#f8ff00; color:#000000;" | Description
Line 393: Line 518:
! style="font-size:13px;text-align: left;background-color:#efefef; color:#000000;" | Boolean (0/1)
! style="font-size:13px;text-align: left;background-color:#efefef; color:#000000;" | Boolean (0/1)
|-
|-
! style="font-size:13px;text-align: left;background-color:#efefef; color:#000000;" | grafana
! style="font-size:13px;text-align: left;background-color:#efefef; color:#000000;" | Set to read /var/log/grafana/grafana.log
! style="font-size:13px;text-align: left;background-color:#efefef; color:#000000;" | Boolean (0/1)
|-


|}
|}
Line 539: Line 670:


<br>
<br>
=Full Configuration Sample=
=Full Configuration Sample=


Line 552: Line 684:
   Login_Fail_Counter="3"
   Login_Fail_Counter="3"
   Max_Reverse_Time="172800"
   Max_Reverse_Time="172800"
  Geoip_URL="https://xml.coolgeo.org/?myip=%IP%"
   Delete_Inactive_Records="2592000"
   Delete_Inactive_Records="2592000"
   Reset_Record_Counter="259200"
   Reset_Record_Counter="259200"
   Process_Timeout="3600"
   Process_Timeout="3600"
  GeoIP_NAME="ipapi.co"
  GeoIP_KEY="0"
  GeoIP_Connect_Failure_Max="5"
  />   
  />   
   
   
Line 575: Line 709:
   vsftp="1"
   vsftp="1"
   mail="1"
   mail="1"
  grafana="1"
  />   
  />   
   
   
Line 599: Line 734:
    
    
  </CONFIG>
  </CONFIG>
=Amazon Linux PoC=
*Manual installation of required modules:
yum install perl-Module-Load-Conditional.noarch -y
yum install perl-DBI.x86_64 -y
yum install perl-XML-Simple.noarch -y
yum install perl-JSON.noarch -y
yum install perl-Net-IP.noarch -y
yum install perl-App-cpanminus.noarch -y
yum install gcc -y
cpanm Proc::ProcessTable
cpanm Mail::Sendmail
yum install nftables.x86_64 -y
yum install perl-DBD-SQLite.x86_64 -y
[root@ip-172-31-26-244 fail2nft]# perl -v
This is perl 5, version 16, subversion 3 (v5.16.3) built for x86_64-linux-thread-multi
[root@ip-172-31-26-244 fail2nft]# lsb_release -a
LSB Version:    :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
Distributor ID: Amazon
Description:    Amazon Linux release 2 (Karoo)
Release:        2
Codename:      Karoo
[root@ip-172-31-26-244 fail2nft]# uname -a
Linux ip-172-31-26-244.eu-west-1.compute.internal 5.10.198-187.748.amzn2.x86_64 #1 SMP Tue Oct 24 19:49:54 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
*auth.pm
  if ( /ssh_dispatch_run_fatal/ ) {  #1.0.xxxx
    if (m/^([^ ]*) *([^ ]*) *([^ ]*) *([^ ]*) *([^ ]*)/){  #Get the epoche date from the log record
    $tmpEpoche=str2time("$1 $2 $3");
    $tmpDate="$1 $2 $3";
    if ($5 =~ /\[(\d+)\]/) {
      $tmpPID=$1;
    }
    }
    if (/from (\d+)\.(\d+)\.(\d+)\.(\d+)/){  #Get IP
    $tmpIP="$1.$2.$3.$4";
    }
    print "OK $tmpDate/$tmpEpoche/$tmpIP => $_ \n";
    $formatdate = strftime($syslog_dateformat, localtime($tmpEpoche));
    $AUTH_ReturnHash->{'SYSLOG'}.="$formatdate Failure SSH Login: SRV=$tmpIP M=S2F SUC=1 LOU=$1 TGE=$tmpDate PROTO=AUTH LCK=1 USR=$tmpUser\n";
    $AUTH_ReturnHash->{'IP_DENY'}->{$tmpIP}=$AUTH_ReturnHash->{'IP_DENY'}->{$tmpIP}+1;
    $cnt_allow++;
  }


=Download=
=Download=
Line 604: Line 785:
|style="width:10%; vertical-align: top;"|
|style="width:10%; vertical-align: top;"|
'''Download Fail2Nft'''<br>
'''Download Fail2Nft'''<br>
[[File:Download.png|60px|link=http://coolscript.org/download/fail2nft.tar.gz|Download]]
[[File:Download.png|60px|link=https://coolscript.org/download/scripts/fail2nft/fail2nft.tar.gz|Download]]
<br>
<br>
|style="width:90%; vertical-align: top;"|
|style="width:90%; vertical-align: top;"|

Latest revision as of 16:56, 6 July 2024

Why Fail2Nft


Fail2Nft offers a simple Intrusion Prevention System (IPS) and more and can be easily installed

Fail2Nft is a lightweight perl solution which aims to keep out of unwanted ssh login attemps from people or robots on your system.
If ssh logins attempt to continue with wrong user/password combinations then Fail2Nft can block the ip address for a specified amount of time.

  • Fail2Nft keeps the setup as simple as possible, the internet installer can setup your server with a very few steps with pre-configured templates.
  • Fail2Nft has been prooven to run on recent Linux Platforms which are on APT but Fail2Nft can run on any other recent LSB distros shipped with nftables as well.
  • Fail2Nft has been designed to run on single instances such as single cloud machines or application servers running ssh, mail or ftp services,
    Fail2Nft is currently not made for firewalls or routers but can be converted to work on customized environments too.
  • Fail2Nft is an update of Syslog_to_Firelwall it follows the same idea but instead of iptables we use nftables in combination with named sets.



Fail2Nft Features

  • Fail2Nft is based on nftables together with named sets.
  • Fail2Nft handles ip v4 and ip v6 addresses automatically
  • Fail2Nft can open all ports to authenticated login IP's, the idea is similar to Port Knocking but using ssh authentication instead of port knocking to allow unlimited access.
  • Dynamic increasing of lock times, optional based by Country or ASN
  • Plugin based development, current available plugins:
    • Auth
    • Mail - Imap/pop
    • FTP
    • Grafana
  • Syslog forwarding to Splunk friendly key/value messages
  • Automaic reinitialization to the previous last known state, for example in case of a reboot
  • Whitelist ip address support
  • XML configuration schema
  • SQLite Database, we create the database and tables automatically and run periodic maintenance on it. No need to install tools for this.
  • Optional event mail
  • Sressless installer available


Tested Platforms

The Fail2Nft installer works currently on Linux APT platforms only and has been tested on

  • Debian
    • 9 (Stretch)
    • 10 (Buster)
    • 11 (Bullseye)
    • 12 (Bookworm)
  • Ubuntu
    • 18 (Bionic Beaver)
    • 20 (Focal Fossa)
    • 21 (Hirsute Hippo)
    • 22 (Jammy Jellyfish)
    • 24 (Noble)



Attention Ubuntu 24 and Debian 12 Users, please check the date time format in your logs, we require the traditional log time format,

to activate you may set to /etc/rsyslog.con:
$ActionFileDefaultTemplate  RSYSLOG_TraditionalFileFormat
  • Raspbian
    • 10
    • 11


Tested Cloud Environments

  • AWS
  • Azure
  • Digital Ocean
  • Hetzner

Easy Installer

Syntax:
 fail2nft-installer.sh
 -h Optional Flag, Display this help
 -a Mandatory Flag, Automatic installation
 -i Optional Flag, Allow icmp
 -s Optional Flag, Install advanced syslog modules
 -t Optional String, tcp ports (comma seperated)
 -u Optional String, udp ports (comma seperated)
 -v Mandatory String, IP version (4 or 6 or both 4,6) valid only if -i or -t or -u is given
 -o Optional Flag,   OpenVPN/Enable
 -e Optional String, OpenVPN Interface Name eg eth0
 -k Optional String, OpenVPN Protocol (tcp or udp)
 -m Optional Int,    OpenVPN Port to masquerade (snat)
 -n Optional String, OpenVPN Network, eg 10.8.0.1\/24
  • Run the following installer command to install Fail2Nft on your system, that is the Default Installation Method
wget -q https://coolscript.org/download/scripts/fail2nft/fail2nft-installer.sh -O /tmp/fail2nft-installer.sh && bash /tmp/fail2nft-installer.sh -a
  • Manual Sample
bash fail2nft-installer.sh -a -t 80,443,25 -u 53  -v 4  -n 10.0.8.0\\/24  -e eth0 -m 1994 -k tcp -o

Sample: Advanced Installation

configure additional tcp ports (80,443), udp port (1194), icmp and prepare the usage for OpenVPN
additional install syslog modules and set the reinstall flag

wget -q https://coolscript.org/download/scripts/fail2nft/fail2nft-installer.sh -O \
/tmp/fail2nft-installer.sh && bash \
/tmp/fail2nft-installer.sh \
-a \
-e eth0 \
-n 192.168.200.0\\/24 \
-o \
-r \
-s \
-t 80,443,1194 \
-u 1194  \
-v 4

Command Line Parameters

Manual adding a IP Address to Fail2Nft, the syntax applies for v4 and v6:

  • Allow a.b.c.d for 24 hours
#fail2nft -add -a -ip a.b.c.d -time 24
  • Deny a.b.c.d for 48 hours
#fail2nft -add -d -ip a.b.c.d -time 48
  • Check logs and apply violations to nft (this is what you want to run through crontab):
#fail2nft -c
  • Same than above but be verbose
#fail2nft -c -v
  • Delete a.b.c.d from Fail2Nft
#fail2nft -delete a.b.c.d
  • Print all options
#fail2nft -h
  • Initialize Fail2Nft, this will read the records out of our database and apply this to nft. Typically used after an reboot.
#fail2nft -i 
  • List all known records
#fail2nft -l 
  • List all known records, filtered by allow
#fail2nft -l -a
  • List all known records, filtered by deny
#fail2nft -l -d
  • Check/create the sqlite database and exit
#fail2nft -s
  • Testing Fail2Nft
#fail2nft -t
  • Testing Fail2Nft, print json
#fail2nft -t -json    
  • Send a Testmail (if configured)
#fail2nft -testmail
  • Print the version only
#fail2nft -version

Internals

Operating Principle

Note that the Internet installer is using preconfigured templates which are based on the below sample schema.

Failnft is using named sets which gets assigned within the table, in this example we use ip4 but the same applies to ip6,
two named sets which are named fail2nft_drop and fail2nft_drop.
The set name can be individual but must match with the name within fail2nft.xml


table ip filter_v4 {
     set fail2nft_drop {
       type ipv4_addr;
       flags timeout
      }
      set fail2nft_accept {
        type ipv4_addr
        flags timeout
      }
 ....

Once specified then the named set must be assigned to a chain, in this case we use Input.

chain INPUT {
               type filter hook input priority 0; policy drop;
               ip saddr @fail2nft_accept counter accept comment "accept by log2nft_accpet"
               ip saddr @fail2nft_drop counter jump my_drop comment "drop by log2nft_drop"
               #Carry on with your configuration from here
...

Process Database

Fail2Nft stores the data to a SQLite database, the database is created automatically at startup, the database contains

  • All seen ip(4/6) addresses from the auth, mail and ftp logs
  • Keeps tracking of events such as first seen, last seen, blocking state, lock times, country, asn, etc
  • Keeps tracking of the last executed time and the state information of geo searches

Fail2Nft should get intialized at startup using the -I option which is applying the neccesary commands to nftables
Fail2Nft should get executed by crontab frequently, the sample shows to run Fail2Nft every minute:

*/1 * * * * (/usr/bin/fail2nft -c )



Process Affinity

Process affinity is a build in function to make sure that Fail2Nft runs only once, with one instance on your system, this is to avoid race conditions.
The function can be controlled by setting Process_Timeout, this is the time which will force Fail2Nft to wait for previous processes to end.


Commands

For your information - Possible commands being executed by Fail2Nft

  • Add an element to a named set
    • The sample is using the filter filter_v4 which is specified in fail2nft.xml
    • The sample is using the named set log2nft_drop which is specified in fail2nft.xml
/usr/sbin/nft add set filter_v4 log2nft_drop \{type ipv4_addr \; flags timeout \; elements=\{a.b.c.d timeout xxxs  comment \"...." \} \;\}
  • Remove a element
    • The sample is using the filter filter_v4 which is specified in fail2nft.xml
    • The sample is using the named set log2nft_drop which is specified in fail2nft.xml
/usr/sbin/nft delete element ip filter_v4 fail2nft_accept \{a.b.c.d\}
  • List rulesets and return json (undocumented)
/usr/sbin/nft -j list ruleset

Log

Log

Traditional logging is enabled by default to /var/log/syslog2nft/syslog2nft.log, this can be changed within the configuration (fail2nft.xml) withinin the element

  • Log
    • Enable="1"
    • Path="/var/log/fail2nft/"

It is recommended to set an logrotate file to allow log maintenance. Note that the online installer is doing this automatically.

Syslog

Syslog is an optional function of Fail2Nft, it is optional because Debian Systems do not distribute the Net::Syslog perl module by default, therefore it is required to
install the required compiler modules ( apt-get install build-essential) in order to allow the cpan module to compile the Net::Syslog module.
Note that the online installer is taking care about this step.
Once installed then it can be activated through the configuration (fail2nft.xml), the configuration represents an array like:

<Syslog Enable="1" IP="192.168.x.y"/> 
<Syslog Enable="1" IP="192.168.x.z"/> 


If the above steps are completed then Fail2Nft should send Syslog/UDP messages in the following format:

Key Value Format
Key Value Description
M F2N The modus being used, this is always F2B (Fail2Nft)
Lock Boolean Indicates if the ip record gets denied (1) or allowed (0)
IP String The ip address, this could be v4 or v6
TIMER Integer The time for how long the record is being blocked
Country String The ISO country code of the ip origin (needs Geoip_URL to be configured)
ASN String The ASN code of the ip origin (needs Geoip_URL to be configured)
LOG Integer The origin of the source, this could be AUTH, MAIL or FTP


  • Syslog Sample
Nov  9 20:22:07 myserver.com fail2nft[26563]: M=F2N LOCK=1 IP=a.b.c.d TIMER=600 COUNTRY=XX ASN=ASxxxx LOG=AUTH

Rsyslog

To log the Splunk optimized message in rsyslog you may add a rule for this

  • /etc/rsyslog.conf
#Enable udp listening
module(load="imudp")
input(type="imudp" port="514")
...
...
#Redirect nftables losg  to it's own log
:msg,regex,"IN=.*OUT=.*SRC=.*DST="      -/var/log/firewall.log
& stop
...
#Redirect Splunk optimized messages to it's own log
:msg,regex,"M=F2N.*"      -/var/log/fail2nft-splunk.log
  • Required date format

Make sure that your date format in /var/log/auth.log is set to something like Jun 16 18:08:07 otherwise try the following setting in rsyslog.conf:


########################### 
#### GLOBAL DIRECTIVES ####
###########################
# Use traditional timestamp format.
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
  • Then Reset rsyslog
systemctl stop rsyslog
rm /var/log/auth.log
systemctl start rsyslog

IP2Country / IP2ASN

Fail2Nft supports a few IP to Geo provider to allow to retrieve the IP Country or ASN,
this information can be optional used to setup individual lock times.

Available provider:

IP2Country Provider
Provider Name Limitation Require Registration Require API-KEY List ASN Is Accurate
abstractapi 20.000/Month
Max 1 per second
Yes Yes Yes Yes
coolgeo.org 200/Day No No Yes Inhouse DB
From 2022
ipapi.co 30.000/Month No No Yes Yes
ipstack.com 5.000/Month Yes Yes No Yes
ip2loc.com 15.000/Month Yes Yes No Yes

Note that the default is set to ipapi.co while on older installations coolgeo.org is used

Maintenance

Daily and monthly maintenance is automatically performed. This is, of course if Fail2Nft runs out of crontab frequently.
Maintenance happens daily at midnight (day change). The Maintenance includes:

  • Removing old ip records, the condition is set within fail2nft.xml - Delete_Inactive_Records (in seconds)
  • Reset the login counter, the condition is set within fail2nft.xml - Reset_Record_Counter (in seconds)

Furthermore we do a SQLite Vaccum command every first day of the month

Performance/Tweaks

Fail2Nft has been tested with up to 50000 Records with no noticeable impacts on cpu or memory usage, that applies even to a Raspberry-V4.
However, if performance matters then consider to reduce the size of logs, for example /var/log/auth.log is kept up to 6 days by default.
Depending on ssh logon frequency this log can grow up which causes then delays for Fail2Nft. You can mitigate this by reducing the archiving time
.

  • Example for /etc/logrotate.d/syslog
{
        rotate 1
...
}
...
/var/log/auth.log
...

Email Notification

Fail2nft can send emails to inform you about ssh events, there are three levels of events:

  • 1 - On success
  • 2 - On failure
  • 3 - Always

Note: In case of an success login there will be only mails being sent if the connection is new to Fail2nft. There are no emails for already pending or known connections.
This is because you don't want to get bombed with emails in case of - for example video streaming via ssh where a continuous logon /logoff events are occurring on your server.
Please note that there is no SMTP-TLS support yet, only plain login is supported for SMTP.
Please note that the SMTP Authentication is optional, if your SMTP server allows relaying with no login then please leave MailUser and MailPassword blank.
You may test the SMTP function with fail2nft -testmail

Configuration

Attributes for the Setup Element
Atrribute Description Type
Discard_Private_IPAddress Discard Private IP Addresses if they were found in any log Boolean (0/1)
On_Success_Timer If set, then this will be the amount of time in seconds which the IP address will remain within the input/accept set Interger/Seconds
On_Success_Renew Update an already whitelisted record Boolean (0/1)
On_Fail_Timer This will be the amount of time in seconds which the IP address will remain within the input/drop set Interger/Seconds
On_Fail_Double_Timer If set then we double the previous On_Fail_Time every time when a known IP address gets blocked Boolean (0/1)
Login_Fail_Counter Maximum count of failed login per IP address before we block it when a known IP address gets blocked Integer
Max_Reverse_Time Used for the very first startup when we create the database, at this time we don't know the time of the last check.

Max_Reverse_Time is then used to limit the time delta which we use for reading the logs.

Integer
GeoIP_NAME If set then we use the specified service to resolve the Country and ASN of the IP sender address

Available Provider to set:

  • abstractapi.com
  • ipapi.co
  • ipstack.com
  • ip2loc.com
  • coolgeo.org
String
GeoIP_KEY Specify the API Key for the above (GeoIP_NAME) service, keys are required for:
  • abstractapi.com
  • ipstack.com
  • ip2loc.com
String
GeoIP_Connect_Failure_Max Specify the amount of max failures per day for this provider, if not specified then we set the default to 3 Integer
Delete_Inactive_Records Specify the amount of time for how long we keep inactive IP addresses in our database Integer/Seconds
Reset_Record_Counter Specify the amount of time before we reset the counter for failed logins Integer/Seconds
Process_Timeout If fail2nft starts multiple times (eg bad performance, misconfiguration) then the follow up process will for for the

specified amount of time before it exits without results.

Integer/Seconds


Attributes for the NFT Element
Atrribute Description Type
Table_IPV4 The name of the ip table String
Set_IPV4_drop The name of the named set to drop packets String
Set_IPV4_accept The name of the named set to accept packets String
Table_IPV6 The name of the ip6 table String
Set_IPV6_drop The name of the named set to drop packets String
Set_IPV6_accept The name of the named set to accept packets String



Attributes for the Logging Element
Atrribute Description Type
Enable The name of the named set to accept packets Boolean (0/1)
Path The name of the named set to accept packets String


Attributes for the Logs Element, Note that auth.log is always enabled
Atrribute Description Type
vsftp Set to read /var/log/vsftp.log Boolean (0/1)
mail Set to read /var/log/mail.log Boolean (0/1)
grafana Set to read /var/log/grafana/grafana.log Boolean (0/1)


Attributes for the Mail Element
Atrribute Description Type
Level 0 = No Mail, 1=Success only, 2=Error only, 3=Always Integer
MailTo Sender Email Address String
MailFrom Sender From Email Address String
MailSMTP SMTP Address String
MailUser SMTP User Authentication (Only AUTH PLAIN support) String
MailPassword SMTP User Password String


Attributes for the Country Element - NOTE: This is an Array
Atrribute Description Type
Enable Enable the setting Boolean (0/1)
Code Country Code String (2)
On_Fail_Timer This will be the amount of time in seconds which the IP address will remain within the input/drop set Integer/Seconds
On_Fail_Double_Timer If set then we double the previous On_Fail_Time every time when a known IP address gets blocked Boolean(0/1)


Attributes for the ASN Element - NOTE: This is an Array
Atrribute Description Type
Enable Enable the setting Boolean (0/1)
Name ASN Code String (7)
On_Fail_Timer This will be the amount of time in seconds which the IP address will remain within the input/drop set Integer/Seconds
On_Fail_Double_Timer If set then we double the previous On_Fail_Time every time when a known IP address gets blocked Boolean(0/1)


Attributes for the Syslog Element - NOTE: This is an Array
Atrribute Description Type
Enable Enable the setting Boolean (0/1)
IP IP Address of the Syslog Server String


Attributes for the Whitelist Element - NOTE: This is an Array
Atrribute Description Type
IP IP Address to whitelits String


Full Configuration Sample

<?xml version="1.0"?>
<CONFIG>

<Setup 
 Discard_Private_IPAddress="1"
 On_Success_Timer="86400"
 On_Success_Renew="1"
 On_Fail_Timer="300"
 On_Fail_Double_Timer ="1"
 Login_Fail_Counter="3"
 Max_Reverse_Time="172800"
 Delete_Inactive_Records="2592000"
 Reset_Record_Counter="259200"
 Process_Timeout="3600"
 GeoIP_NAME="ipapi.co"
 GeoIP_KEY="0" 
 GeoIP_Connect_Failure_Max="5"
/>   

<NFTABLES
 Table_IPV4="filter_v4"
 Set_IPV4_drop="log2nft_drop"
 Set_IPV4_accept="log2nft_accept"
 Table_IPV6="filter_v6"
 Set_IPV6_drop="log2nft_drop"
 Set_IPV6_accept="log2nft_accept"
/>
 
<Logging
 Enable="1" 
 Path="/var/log/fail2nft/"
/>   

<Logs
 vsftp="1"
 mail="1"
 grafana="1"
/>   

<Email
  Level="0"
  MailTo = "receiver@mail.com"
  MailFrom="sender@mail.com"
  MailSMTP = "smtp.mail.com"
  MailUser = "user"
  MailPassword="password"
/>   

<Country Enable="1" Code="XX" On_Fail_Timer="310" On_Fail_Double_Timer ="1"/> 
<Country Enable="0" Code="YY" On_Fail_Timer="86400" On_Fail_Double_Timer ="1"/> 
<ASN Enable="0" Name="AS366XX" On_Fail_Timer="360" On_Fail_Double_Timer ="1"/> 
<ASN Enable="0" Name="AS244XX" On_Fail_Timer="370" On_Fail_Double_Timer ="0"/> 

<Syslog Enable="0" IP="127.0.0.1"/>  
<Syslog Enable="0" IP="192.168.1.1"/>  

<Whitelist IP="8.8.8.8"/> 
<Whitelist IP="1.2.3.4"/> 

 
</CONFIG>

Amazon Linux PoC

  • Manual installation of required modules:
yum install perl-Module-Load-Conditional.noarch -y
yum install perl-DBI.x86_64 -y
yum install perl-XML-Simple.noarch -y
yum install perl-JSON.noarch -y
yum install perl-Net-IP.noarch -y
yum install perl-App-cpanminus.noarch -y
yum install gcc -y
cpanm Proc::ProcessTable
cpanm Mail::Sendmail
yum install nftables.x86_64 -y
yum install perl-DBD-SQLite.x86_64 -y

[root@ip-172-31-26-244 fail2nft]# perl -v
This is perl 5, version 16, subversion 3 (v5.16.3) built for x86_64-linux-thread-multi

[root@ip-172-31-26-244 fail2nft]# lsb_release -a
LSB Version:    :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
Distributor ID: Amazon
Description:    Amazon Linux release 2 (Karoo)
Release:        2
Codename:       Karoo

[root@ip-172-31-26-244 fail2nft]# uname -a
Linux ip-172-31-26-244.eu-west-1.compute.internal 5.10.198-187.748.amzn2.x86_64 #1 SMP Tue Oct 24 19:49:54 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
  • auth.pm
  if ( /ssh_dispatch_run_fatal/ ) {   #1.0.xxxx
   if (m/^([^ ]*) *([^ ]*) *([^ ]*) *([^ ]*) *([^ ]*)/){  #Get the epoche date from the log record
    $tmpEpoche=str2time("$1 $2 $3");
    $tmpDate="$1 $2 $3";
    if ($5 =~ /\[(\d+)\]/) {
     $tmpPID=$1;
    }
   }
   if (/from (\d+)\.(\d+)\.(\d+)\.(\d+)/){  #Get IP
    $tmpIP="$1.$2.$3.$4";
   }
   print "OK $tmpDate/$tmpEpoche/$tmpIP => $_ \n";
   $formatdate = strftime($syslog_dateformat, localtime($tmpEpoche));
   $AUTH_ReturnHash->{'SYSLOG'}.="$formatdate Failure SSH Login: SRV=$tmpIP M=S2F SUC=1 LOU=$1 TGE=$tmpDate PROTO=AUTH LCK=1 USR=$tmpUser\n";
   $AUTH_ReturnHash->{'IP_DENY'}->{$tmpIP}=$AUTH_ReturnHash->{'IP_DENY'}->{$tmpIP}+1;
   $cnt_allow++;
  }

Download

Download Fail2Nft
Download



Please drop us comments, feedbacks, wishes, criticism, or for future announcements - Welcome to contact: fail2nft at coolscript.org