Setting up a firewall with AIX TCP/IP filtering

Setting up a firewall with AIX TCP/IP filtering

Create a configuration for a sample scenario

by Leonardo Cabral | Published October 4, 2011

Systems

---

Introduction

A single POWER server has the processing power to run hundreds of LPARs or partitions (a partition as an independent instance of an operating system installation) which means you can have hundreds of servers within a single physical box thanks to integrated virtualization capabilities. If these hundreds of servers are handling Internet or other sensitive traffic, you probably want to setup some firewall rules in them as an extra security layer (along with your network firewalls and intrusion prevention devices).

This article begins with a short review of TCP/IP networks. If you’re familiar with TCP/IP and the way it works, you can skip the next section. Second, you will review the packages, steps, and commands required to enable and setup TCP/IP filtering capabilities in an AIX box. Finally, you’ll use the concepts presented in the first sections to create a configuration for a sample scenario.

TCP/IP review

TCP/IP is the essence that gives life to networks. Data transmitted over a network is broken into small pieces called “packets” or “IP packets” (for Internet Protocol packets). When information is transmitted, it is broken into packets at the sender of that data, labeled with information about the source and destination computers, and sent over the network.

Now let’s see at a high level how a network packet is structured. Depending on the protocol being used for the communication, you could be dealing with one of the following IP packets:

• Plain IP packet, or IP datagram: IP packets are used to encapsulate any of the following protocols. Under normal circumstances, you do not see a plain IP packet traveling over a network. You can, however, use information transmitted in IP packets to create your firewall filters.
• ICMP message: ICMP is a protocol designed to report and diagnose network problems. The most common command that generates an ICMP message is ping, used to test if there is connectivity to a certain device. When filtering some ICMP packets when configuring filters, for example, you may want to prevent certain ICMP traffic from being sent to your Internet connected servers.
• UDP packet or user datagram: UDP is an unreliable, connectionless protocol. UDP Datagrams can be lost or delivered in a different order to the receiving application; the error handling must be done at the application layer as well. The advantages are lower processing, lower latency, and quicker data transfer due to less controls being performed by the protocol stack. Some known services that rely on top of UDP are DNS queries, TFTP, and VoIP.
• TCP packet or segment: TCP is a reliable, connection oriented protocol. TCP segments have guaranteed delivery in the same order they are sent, with the cost of some extra packets being required to establish and finish connections and more processing done by the protocol stack to ensure this. Many known services rely on TCP: HTTP (web traffic), SMTP (mail traffic), FTP (file transfer), SSH (Secure Shell), Telnet, and many others.

Typically you’ll have servers dedicated to a single or a few of these services, hence you’ll want to ensure that you only allow traffic for the services you provide.

X-ray of ICMP, TCP and UDP

The following tables show how different types of network packets are structured, and more importantly, you can see which is the command line option for AIX filtering management commands that allow us to filter packets based on the contents of this field. You will use these command line options later when you analyze the AIX commands to operate with filters.

Table 1 – IP Packet Format

bits	0-15			16-31
0	Version (-v)	IHL	Type of Service	Total Length
32	Identification			Flags (-f)	Fragment Offset
64	Time to Live		Protocol (-c)	Header Checksum
96	Source Address (-s)
128	Destination Address (-d)
160	Options					Padding

 

Table 4 – TCP Format

bits	0-15			16-31
0	Source Port (-o)			Destination Port (-O)
32	Sequence Number
64	Acknowledgement Number
96	Offset	Reserved	Flags	Window
128	Checksum			Urgent Pointer
160	Options			Padding
192	Data

 

Introducing AIX filters

In some situations, you’ll want to prevent certain packets from arriving to their destination, or you might be interested in allowing only certain packets to do so.

Required packages

Some common scenarios include allowing connections from trusted sources only, blocking connections to services that are not provided by the server, and allowing connections only to specific services. In these situations, you can use the TCP/IP filtering capabilities bundled with the AIX operating system.

TCP/IP filters are included in the IPSec packages, so the easiest way to check for their presence is using lslpp –l to list installed packages and verify that you have at least the following two packages present:

 

bos.net.ipsec.keymgt
bos.net.ipsec.rte

 

IPsec is a protocol for creating encrypted communication channels between servers, often referred to as tunnels or VPN tunnels. Ipsec is not part of this article, but if you plan to use IPSec in your environment you want to make sure that you also have these packages installed:

 

bos.msg.LANG.net.ipsec 
bos.net.ipsec.websm 
bos.crypto‑priv

 

For a more complete reference on IPSec, you can refer to the Security Guide document in the resources section on the right.

Commands reference

To work with TCP/IP filters you only need a few commands, which is explained here and then used in the next section. If you’re familiar with AIX commands you see that these follow the same logic of having descriptive prefixes in their names, like mk, ls, and rm, followed by the filt suffix.

• lsfilt: List filters rules present in the table. When created, each rule is assigned a number, which can be easily seen using this command.
• genfilt: Adds a filter rule to the table. This is the one you use to create new filters. If you do not specify a position with the –n parameter, the new rule is added at the end of the table.
• chfilt: Used to change existing filter rules. You need to provide the rule ID to indicate which rule you want to modify. Rule 1 is the default rule and can’t be changed with this command.
• rmfilt: The rm suffix should sound familiar with any UNIX administrator. You use this command whenever you have to remove a filter rule providing its rule ID.
• mkfilt: This is a key command that allows us to activate or deactivate the filter rules in the table, enable or disable logging for filters, and change the default rules. For the changes done to the filters table to take effect, you’ll have to run this command with some arguments.

Approaches to filtering policies

When you refer to policies in TCP/IP filtering, you are referring to two possible approaches to security:

• To deny all traffic by default and allow only what you want to be permitted.
• To allow all traffic by default and deny only what you want to restrict.

From a security standpoint, the deny all policy is the most secure one. However, if you’re configuring a server already in production, you have to be very cautious before applying this policy to avoid generating unnecessary service outages. For new servers, it is easier to use this policy from the beginning since you have enough time for testing before moving them to production.

The allow all policy is more permissive, and it is a good policy for cases when you have production services and want to protect a specific service.

Sample scenario

In this section, I present a simple sample scenario that will give you a good starting point to implement other filtering configurations in your servers.

Scenario

Your company has deployed a new intranet application in an AIX LPAR of a p710 Express server, and the security area of your company has requested to restrict access to web applications allowing only traffic coming from the internal proxy server with the address 172.16.10.5. The web application listens in TCP ports 80 and 443. The security area has also made a second requirement to allow only SSH connections to the interface connected in the administration network 10.1.1.x. The security area has expressed that these requirements are mandatory and independent to any service configurations done to the services, as an extra layer of protection. The IP addresses of your server are 172.16.10.45 and 10.1.1.45 (the first being the address where services are provided to users, and the second being used by administrators to login to the server and perform maintenance tasks). The following graphic shows this scenario.

Figure 1: Sample scenario

Implementation – Setting up rules

To begin, enable IP filtering by enabling IPSec in the server. For this, use the ipsec4 smitty fastpath:

 

#smitty ipsec4

 

Select Start/Stop IP Security and press Enter:

 

Move cursor to desired item and press Enter
    Start/Stop IP Security
    Basic IP Security Congiguration
    Advanced IP Security Configuration

 

Select Start IP Security and press Enter:

 

Move cursor to desired item and press Enter
    Start IP Security
    Stop IP Security

 

From the following select Start IP Security and press Enter:

 

Type or select values in the entry field
Press Enter AFTER making all desired changes              EntryFields    Start IP Security                    [Now and AfterReboot]                  
    Deny ALL Non0Secure IP Packets                        [No]

 

The previous sequence shows how to enable IPSec to take advantage of the IP filtering feature. On successfully completion the following displays:

 

Command: OK    stdout: yes    stderr: no
 
Before command completion, additional instructions may appear below.

ipsec_v4 Available
Default rule for IPv4 in ODM has been changed.
Successfully set default action to PERMIT

 

If you see any error at this point, it might be possible that your system is missing some of the required packages mentioned earlier in the Required packages section. One way to confirm if IPSec has already been enabled in a system is to run the following:

 

#lsdev ‑l ipsec_v4
ipsec_v4 Available  IP Version 4 Security Extension

 

For logging, be very careful when deciding which filtering events are going to be logged. A policy like “log everything” soon becomes a nightmare filling the logging filesystem and even causing some undesirable side effects to your sever. Always be rational about the logging policies. In this scenario, you are only going to log those packets that are rejected by the filters, thus the log will help you identify unauthorized sources of connections. Logging is controlled with the -l (lowercase L) parameter in each rule, followed by Y or N (Yes or No).

You can now use the lsfilt command to see the default rules that are created. Do not rush into modifying the default rules until you have a clear understanding of their purpose and impact.

Now, you are going to create the rules to comply with the demands of the security area.

Let’s begin by adding the rules to restrict administrative accesses to the administration network only.

 

##Rules to allow authorized administration traffic
#genfilt ‑v 4 ‑a P ‑s 10.1.1.0 ‑m 255.255.255.0 ‑d 10.1.1.45 ‑M
255.255.255.255 ‑g N ‑c tcp ‑o gt ‑p 1023 ‑O eq ‑P 22 ‑r L ‑w I ‑l 
N ‑f Y ‑i all
Filter rule 3 for IPv4 has been added successfully.
#genfilt ‑v 4 ‑a P ‑s 10.1.1.45 ‑m 255.255.255.255 ‑d 10.1.1.0 ‑M 
255.255.255.0 ‑g N ‑c tcp/ack ‑o eq ‑p 22 ‑O gt ‑P 1023 ‑r L ‑w O ‑l
N ‑f Y ‑i en1
Filter rule 4 for IPv4 has been added successfully.

##Rules to deny and log unauthorized administration traffic. 
##Note that we add one rule per server address to have 
##independence from service configuration.
#genfilt ‑v 4 ‑a D ‑s 0 ‑m 0 ‑d 10.1.1.45 ‑M 255.255.255.255 ‑g N ‑c 
tcp ‑O eq ‑P 22 ‑r L ‑w I ‑l Y ‑f Y ‑i all
Filter rule 5 for IPv4 has been added successfully.
#genfilt ‑v 4 ‑a D ‑s 0 ‑m 0 ‑d 172.16.10.45 ‑M 255.255.255.255 ‑g N 
‑c tcp ‑O eq ‑P 22 ‑r L ‑w I ‑l Y ‑f Y ‑i all
Filter rule 6 for IPv4 has been added successfully.

Show more

 

The previous rules achieve our goal with administration connections. Note however that I intentionally left a gap in our first rule, rule 3, by not restricting the interface to en1. I did this to show you how to use the chfilt command to alter an existing rule, as you can see below:

 

#chfilt ‑v 4 ‑n 3 ‑i en1
Filter rule 3 for IPv4 has been changed successfully.
#

 

Now let’s add the rules to restrict access to the intranet application to connections originated in the internal proxy server 172.16.10.5.

 

##Rules to allow traffic between Proxy and Web Application
#genfilt ‑v 4 ‑a P ‑s 172.16.10.5 ‑m 255.255.255.255 ‑d 172.16.10.45 
‑M 255.255.255.255 ‑g N ‑c tcp ‑o gt ‑p 1023 ‑O eq ‑P 80 ‑r L ‑w I ‑l 
N ‑f Y ‑i en2
Filter rule 7 for IPv4 has been added successfully.
#genfilt ‑v 4 ‑a P ‑s 172.16.10.45 ‑m 255.255.255.255 ‑d 172.16.10.5 
‑M 255.255.255.255 ‑g N ‑c tcp/ack ‑o eq ‑p 80 ‑O gt ‑P 1023 ‑r L ‑w O 
‑l N ‑f Y ‑i en2
Filter rule 8 for IPv4 has been added successfully.
#genfilt ‑v 4 ‑a P ‑s 172.16.10.5 ‑m 255.255.255.255 ‑d 172.16.10.45 
‑M 255.255.255.255 ‑g N ‑c tcp ‑o gt ‑p 1023 ‑O eq ‑P 443 ‑r L ‑w I ‑l N 
‑f Y ‑i en2
Filter rule 9 for IPv4 has been added successfully.
#genfilt ‑v 4 ‑a P ‑s 172.16.10.45 ‑m 255.255.255.255 ‑d 172.16.10.5 
‑M 255.255.255.255 ‑g N ‑c tcp/ack ‑o eq ‑p 443 ‑O gt ‑P 1023 ‑r L ‑w 
O ‑l N ‑f Y ‑i en2
Filter rule 10 for IPv4 has been added successfully.

##Rules to reject traffic to the Web Application not coming from the Proxy
#genfilt ‑v 4 ‑a D ‑s 0 ‑m 0 ‑d 172.16.10.45 ‑M 255.255.255.255 ‑g N 
‑c tcp ‑O eq ‑P 80 ‑r L ‑w I ‑l Y ‑f Y ‑i all
Filter rule 11 for IPv4 has been added successfully.
#genfilt ‑v 4 ‑a D ‑s 0 ‑m 0 ‑d 10.1.1.45 ‑M 255.255.255.255 ‑g N 
‑c tcp ‑O eq ‑P 80 ‑r L ‑w I ‑l Y ‑f Y ‑i all
Filter rule 12 for IPv4 has been added successfully.
#genfilt ‑v 4 ‑a D ‑s 0 ‑m 0 ‑d 172.16.10.45 ‑M 255.255.255.255 ‑g N
‑c tcp ‑O eq ‑P 443 ‑r L ‑w I ‑l Y ‑f Y ‑i all
Filter rule 13 for IPv4 has been added successfully.
#genfilt ‑v 4 ‑a D ‑s 0 ‑m 0 ‑d 10.1.1.45 ‑M 255.255.255.255 ‑g N 
‑c tcp ‑O eq ‑P 443 ‑r L ‑w I ‑l Y ‑f Y ‑i all
Filter rule 14 for IPv4 has been added successfully.

Show more

 

The previous commands have created all the rules required to comply with the demands of the scenario. Note that the order of the rules is important in achieving our objective.

Next, you are going to perform a few more steps required to enable logging of rejected packets and activating your configuration.

Implementation – Setting up logging

Now, you are going to configure the syslog daemon to log entries coming from the IP filters in a file that you specify.

 

##Backup syslog.conf file before modifying it.
#cp /etc/syslog.conf /etc/syslog.conf.bak
##Append entry for IP filters logs.
#echo "local4.debug /var/adm/ipsec.log" >> /etc/syslog.conf
##Create log file and set permissions (permissions may depend on 
##company policies)
#touch /var/adm/ipsec.log
#chmod 644 /var/adm/ipsec.log
##Refresh the syslog subsystem to activate the new configuration.
#refresh ‑s syslogd
0513‑095 The request for subsystem refresh was completed successfully.

 

Implementation – Activating everything

At this point, you have created the filtering rules and prepared syslog to record logs of rejected packets. It is time to activate your configuration and, for this, you will use the mkfilt command.

 

 #Start the log functionality of the filter rule module
#mkfilt ‑g start
##Activates the filter rules
#mkfilt –u

 

Panic mode: Let’s assume that you have logged in to your server through the console and have created and activated your new firewall rules. Suddenly you receive a phone call from someone saying that he is no longer able to access one of the services running in the server. You realize that probably one of the rules has a typo or is incorrect, but it’s not the moment to review all the rules on by one, you have to restore the service immediately. If this happens, you just need to run mkfilt –d to deactivate the rules. Don’t fall in the trap of using rmfilt, this command will only affect the rules in the table but won’t take effect until you run mkfilt.

Once you have completed the configuration, you can periodically check the /var/adm/ipsec.log file to view which packets are being dropped and detect either potential attacks or some adjustments to be done to the configuration.

Conclusion

In this article I have presented the AIX IP filtering capabilities, a useful built-in feature that will allow system administrators to implement an extra layer of security in their servers. Some basic networking concepts were presented, as well as a summary of required packages and commands to operate with filters. As always, any firewalling configuration has to be carefully planned and designed to avoid secondary side effects that are usually very hard to troubleshoot.

UNIX administrators willing to use this module now have a good starting point for their configurations.