Fail2nft

From Coolscript
Revision as of 21:21, 12 November 2019 by Admin (talk | contribs)
Jump to navigation Jump to search

Why Fail2Nft


Fail2Nft tracks your logs against unauthorized logins and blocks the sender ip address by condition for a specified amount of time.

 Fail2Nft is a lightweight perl script solution which aims to keep out of unwanted ssh login attemps from people or robots on your system.
If ssh logins attempt to continue with wrong user/password combinations then Fail2Nft can block the ip address for a specified amount of time.

  • Fail2Nft keeps the setup as simple as possible, the internet installer can setup your server with a very few steps with pre-configured templates.
  • Fail2Nft has been prooven to run on recent Linux Platforms which are on APT but Fail2Nft can run on any other recent LSB distros shipped with nftables as well.
  • Fail2Nft has been designed to run on single instances such as single cloud machines or application servers running ssh, mail or ftp services,
    Fail2Nft is currently not made for firewalls or routers but can be adapted to work on those environments as well.This is because the setup process would becomes too complex
  • Fail2Nft is an update of Syslog_to_Firelwall it follows the same idea but instead of iptables we use nftables in combination with named sets.



Fail2Nft Features

  • Fail2Nft is based on nftables together with named sets.
  • Fail2Nft handles ip v4 and ip v6 addresses automatically
  • Fail2Nft can open all ports to authenticated login IP's, the idea is similar to Port Knocking but using ssh authentication instead of port knocking to allow unlimited access.
  • Dynamic increasing of lock times, optional based by Country or ASN
  • Plugin based development, current available plugins:
    • Auth
    • Mail - Imap/pop
    • FTP
  • Syslog forwarding to Splunk friendly key/value messages
  • Automaic reinitialization to the previous last known state, for example in case of a reboot
  • Whitelist ip address support
  • XML configuration schema
  • SQLite Database, we create the database and tables automatically and run periodic maintenance on it. No need to install tools for this.
  • Optional event mail
  • Sressless installer available


Proven Platforms

Fail2Nft works currently on Linux APT platforms and has been tested on

  • Debian 9 Stretch
  • Debian 10 Buster
  • Ubuntu 18


Internet Installer

Caution If you run already a netfilter solution on your server then watch your choice during the installation as this could lead into a malfunction system

The installer performs the entire steps needed to get a working fail2nft setup.
This includes the download of all files needed, installing all packages needed, create directories, create sym links, create the iinit script and setup crontab as well.


  • Run the following installer command to install Fail2Nft on your system:
wget -q https://coolscript.org/download/fail2nft-installer.pl -O /tmp/fail2nft-installer.pl && perl /tmp/fail2nft-installer.pl




Command Line Parameters

Manual adding a IP Address to Fail2Nft, the syntax applies for v4 and v6:

  • Allow a.b.c.d for 24 hours
#fail2nft -add -a -ip a.b.c.d -time 24
  • Deny a.b.c.d for 48 hours
#fail2nft -add -d -ip a.b.c.d -time 48
  • Check logs and apply violations to nft (this is what you want to run through crontab):
#fail2nft -c
  • Same than above but be verbose
#fail2nft -c -v
  • Delete a.b.c.d from Fail2Nft
#fail2nft -delete a.b.c.d
  • Print all options
#fail2nft -h
  • Initialize Fail2Nft, this will read the records out of our database and apply this to nft. Typically used after an reboot.
#fail2nft -i
  • List all known records
#fail2nft -l
  • List all known records, filtered by allow
#fail2nft -l -a
  • List all known records, filtered by deny
#fail2nft -l -d
  • Check/create the sqlite database and exit
#fail2nft -s
  • Testing Fail2Nft
#fail2nft -t
  • Testing Fail2Nft, print json
#fail2nft -t -json
  • Send a Testmail (if configured)
#fail2nft -testmail
  • Print the version only
#fail2nft -version

Internals

Operating Principle

Note that the Internet installer is using preconfigured templates which are based on the below sample schema.

Failnft is using named sets which gets assigned within the table, in this example we use ip4 but the same applies to ip6,
two named sets which are named fail2nft_drop and fail2nft_drop.
The set name can be individual but must match with the name within fail2nft.xml 


table ip filter_v4 {
     set fail2nft_drop {
       type ipv4_addr;
       flags timeout
      }
      set fail2nft_accept {
        type ipv4_addr
        flags timeout
      }
 ....

Once specified then the named set must be assigned to a chain, in this case we use Input. 

chain INPUT {
               type filter hook input priority 0; policy drop;
               ip saddr @fail2nft_accept counter accept comment "accept by log2nft_accpet"
               ip saddr @fail2nft_drop counter jump my_drop comment "drop by log2nft_drop"
               #Carry on with your configuration from here
...

Process Database

Fail2Nft stores the data to a SQLite database, the database is created automatically at startup, the database contains

  • All seen ip(4/6) addresses from the auth, mail and ftp logs
  • Keeps tracking of events such as first seen, last seen, blocking state, lock times, country, asn, etc
  • Keeps tracking of the last executed time and the state information of geo searches

Fail2Nft should get intialized at startup using the -I option which is applying the neccesary commands to nftables
Fail2Nft should get executed by crontab frequently, the sample shows to run Fail2Nft every minute: 

*/1 * * * * (/usr/bin/fail2nft -c )



Process Affinity

Process affinity is a build in function to make sure that Fail2Nft runs only once, with one instance on your system, this is to avoid race conditions.
The function can be controlled by setting Process_Timeout, this is the time which will force Fail2Nft to wait for previous processes to end.


Commands

For your information - Possible commands being executed by Fail2Nft

  • Add an element to a named set
    • The sample is using the filter filter_v4 which is specified in fail2nft.xml
    • The sample is using the named set log2nft_drop which is specified in fail2nft.xml
/usr/sbin/nft add set filter_v4 log2nft_drop \{type ipv4_addr \; flags timeout \; elements=\{a.b.c.d timeout xxxs  comment \"...." \} \;\}
  • Remove a element
    • The sample is using the filter filter_v4 which is specified in fail2nft.xml
    • The sample is using the named set log2nft_drop which is specified in fail2nft.xml
/usr/sbin/nft delete element ip filter_v4 fail2nft_accept \{a.b.c.d\}
  • List rulesets and return json (undocumented)
/usr/sbin/nft -j list ruleset

Log

Log

Traditional logging is enabled by default to /var/log/syslog2nft/syslog2nft.log, this can be changed within the configuration (fail2nft.xml) withinin the element

  • Log
    • Enable="1"
    • Path="/var/log/fail2nft/"

It is recommended to set an logrotate file to allow log maintenance. Note that the online installer is doing this automatically.

Syslog

Syslog is an optional function of Fail2Nft, it is optional because Debian Systems do not distribute the Net::Syslog perl module by default, therefore it is required to
install the required compiler modules ( apt-get install build-essential) in order to allow the cpan module to compile the Net::Syslog module.
Note that the online installer is taking care about this step.
Once installed then it can be activated through the configuration (fail2nft.xml), the configuration represents an array like: 

<Syslog Enable="1" IP="192.168.x.y"/> 
<Syslog Enable="1" IP="192.168.x.z"/>


If the above steps are completed then Fail2Nft should send Syslog/UDP messages in the following format:

Key Value Format
Key Value Description
M F2N The modus being used, this is always F2B (Fail2Nft)
Lock Boolean Indicates if the ip record gets denied (1) or allowed (0)
IP String The ip address, this could be v4 or v6
TIMER Integer The time for how long the record is being blocked
Country String The ISO country code of the ip origin (needs Geoip_URL to be configured)
ASN String The ASN code of the ip origin (needs Geoip_URL to be configured)
LOG Integer The origin of the source, this could be AUTH, MAIL or FTP


  • Syslog Sample
Nov  9 20:22:07 myserver.com fail2nft[26563]: M=F2N LOCK=1 IP=a.b.c.d TIMER=600 COUNTRY=XX ASN=ASxxxx LOG=AUTH

Geo Functions

Fail2Nft is using an optional third party service to resolve the country and asn of the sender ip, if this function is depreciated then simply set the value to 0 or delete the entire element
Note that the geo server runs on limited resources, response time can be high as this is a private and small equipped cloud server.

You may build this service by your own too, Fail2Nft expects the following xml schema to resolve ip to geo data: 

<Client>
 <IP>a.b.c.d</IP>
 <COUNTRY>XX</COUNTRY>
 <ASN>ASxxxxxx</ASN>
</Client>

Note that this function has some kind of DOS protection inside, on the client site the service gets automatically disabled if timeouts occure three times in row.
The service becomes than automatically active again after the daily maintenance task at midnight.

Maintenance

Daily and monthly maintenance is automatically performed. this is, of course if Fail2Nft runs out of crontab frequently.
Maintenance happens daily at midnight (day change). The Maintenance includes:

  • Removing old ip records, the condition is set within fail2nft.xml - Delete_Inactive_Records (in seconds)
  • Reset the login counter, the condition is set within fail2nft.xml - Reset_Record_Counter (in seconds)

Furthermore we do a SQLite Vaccum command every first day of the month

Performance/Tweaks

Fail2Nft has been tested with up to 50000 Records with no noticeable impacts on cpu or memory usage, that applies even to a Raspberry v4.
However, if performance matters then consider to reduce the size of logs, for example /var/log/auth.log is kept up to 6 days by default.
Depending on ssh logon frequency this log can grow up which causes the delays for Fail2Nft. You can mitigate this by reducing the archiving time
.

  • Example for /var/log/syslog
{
        rotate 1
...
}
...
/var/log/auth.log
...

Email Notification

Fail2nft can send emails to inform you about ssh events, there are three levels of events:

  • 1 - On success
  • 2 - On failure
  • 3 - Always

Note: In case of an success login there will be only mails being sent if the connection is new to Fail2nft. There are no emails for already pending or known connections.
This is because you don't want to get bombed with emails in case of - for example video streaming via ssh where a continuous logon /logoff events are occurring on your server.
Please note that there is no SMTP-TLS support yet, only plain login is supported for SMTP.
Please note that the SMTP Authentication is optional, if your SMTP server allows relaying with no login then please leave MailUser and MailPassword blank.
You may test the SMTP function with fail2nft -testmail

Configuration

Attributes for the Setup Element
Atrribute Description Type
Discard_Private_IPAddress Discard Private IP Addresses if they were found in any log Boolean (0/1)
On_Success_Timer If set, then this will be the amount of time in seconds which the IP address will remain within the input/accept set Interger/Seconds
On_Success_Renew Update an already whitelisted record Boolean (0/1)
On_Fail_Timer This will be the amount of time in seconds which the IP address will remain within the input/drop set Interger/Seconds
On_Fail_Double_Timer If set then we double the previous On_Fail_Time every time when a known IP address gets blocked Boolean (0/1)
Login_Fail_Counter Maximum count of failed login per IP address before we block it when a known IP address gets blocked Integer
Max_Reverse_Time Used for the very first startup when we create the database, at this time we don't know the time of the last check.

Max_Reverse_Time is then used to limit the time delta which we use for reading the logs.

Integer
Geoip_URL If specified then we use the URL / Service to resolve the Country and ASN of the IP address String
Delete_Inactive_Records Specify the amount of time for how long we keep inactive IP addresses in our database Integer/Seconds
Reset_Record_Counter Specify the amount of time before we reset the counter for failed logins Integer/Seconds
Process_Timeout If fail2nft starts multiple times (eg bad performance, misconfiguration) then the follow up process will for for the

specified amount of time before it exits without results.

Integer/Seconds


Attributes for the NFT Element
Atrribute Description Type
Table_IPV4 The name of the ip table String
Set_IPV4_drop The name of the named set to drop packets String
Set_IPV4_accept The name of the named set to accept packets String
Table_IPV6 The name of the ip6 table String
Set_IPV6_drop The name of the named set to drop packets String
Set_IPV6_accept The name of the named set to accept packets String



Attributes for the Logging Element
Atrribute Description Type
Enable The name of the named set to accept packets Boolean (0/1)
Path The name of the named set to accept packets String


Attributes for the Logs Element
Atrribute Description Type
vsftp Set to read /var/log/vsftp.log Boolean (0/1)
mail Set to read /var/log/mail.log Boolean (0/1)


Attributes for the Mail Element
Atrribute Description Type
Level 0 = No Mail, 1=Success only, 2=Error only, 3=Always Integer
MailTo Sender Email Address String
MailFrom Sender From Email Address String
MailSMTP SMTP Address String
MailUser SMTP User Authentication (Only AUTH PLAIN support) String
MailPassword SMTP User Password String


Attributes for the Country Element - NOTE: This is an Array
Atrribute Description Type
Enable Enable the setting Boolean (0/1)
Code Country Code String (2)
On_Fail_Timer This will be the amount of time in seconds which the IP address will remain within the input/drop set Integer/Seconds
On_Fail_Double_Timer If set then we double the previous On_Fail_Time every time when a known IP address gets blocked Boolean(0/1)


Attributes for the ASN Element - NOTE: This is an Array
Atrribute Description Type
Enable Enable the setting Boolean (0/1)
Name ASN Code String (7)
On_Fail_Timer This will be the amount of time in seconds which the IP address will remain within the input/drop set Integer/Seconds
On_Fail_Double_Timer If set then we double the previous On_Fail_Time every time when a known IP address gets blocked Boolean(0/1)


Attributes for the Syslog Element - NOTE: This is an Array
Atrribute Description Type
Enable Enable the setting Boolean (0/1)
IP IP Address of the Syslog Server String


Attributes for the Whitelist Element - NOTE: This is an Array
Atrribute Description Type
IP IP Address to whitelits String


Full Configuration Sample

<?xml version="1.0"?>
<CONFIG>

<Setup 
 Discard_Private_IPAddress="1"
 On_Success_Timer="86400"
 On_Success_Renew="1"
 On_Fail_Timer="300"
 On_Fail_Double_Timer ="1"
 Login_Fail_Counter="3"
 Max_Reverse_Time="172800"
 Geoip_URL="https://xml.coolgeo.org/?myip=%IP%"
 Delete_Inactive_Records="2592000"
 Reset_Record_Counter="259200"
 Process_Timeout="3600"
/>   

<NFTABLES
 Table_IPV4="filter_v4"
 Set_IPV4_drop="log2nft_drop"
 Set_IPV4_accept="log2nft_accept"
 Table_IPV6="filter_v6"
 Set_IPV6_drop="log2nft_drop"
 Set_IPV6_accept="log2nft_accept"
/>
 
<Logging
 Enable="1" 
 Path="/var/log/fail2nft/"
/>   

<Logs
 vsftp="1"
 mail="1"
/>   

<Email
  Level="0"
  MailTo = "receiver@mail.com"
  MailFrom="sender@mail.com"
  MailSMTP = "smtp.mail.com"
  MailUser = "user"
  MailPassword="password"
/>   

<Country Enable="1" Code="XX" On_Fail_Timer="310" On_Fail_Double_Timer ="1"/> 
<Country Enable="0" Code="YY" On_Fail_Timer="86400" On_Fail_Double_Timer ="1"/> 
<ASN Enable="0" Name="AS366XX" On_Fail_Timer="360" On_Fail_Double_Timer ="1"/> 
<ASN Enable="0" Name="AS244XX" On_Fail_Timer="370" On_Fail_Double_Timer ="0"/> 

<Syslog Enable="0" IP="127.0.0.1"/>  
<Syslog Enable="0" IP="192.168.1.1"/>  

<Whitelist IP="8.8.8.8"/> 
<Whitelist IP="1.2.3.4"/> 

 
</CONFIG>

Download

Download Fail2Nft
Download



Please drop us comments, feedbacks, wishes, criticism, or for future announcements - Welcome to contact: fail2nft at coolscript.org

Retrieved from "https://coolscript.org/index.php?title=Fail2nft&oldid=141"