suricata mysql_Suricata 配置文件阅读 以及标注

Suricata.yaml

Suricata uses the Yaml format for configuration. The Suricata.yaml

file included in the source code, is the example configuration of

Suricata. This document will explain each option.

At the top of the YAML-file you will find % YAML 1.1. Suricata

reads the file and identifies the file as YAML.

Max-pending-packets

With the

max-pending-packets(最大等待的数据包数量)

setting you can set the number of packets you allow Suricata to

process simultaneously(同时地).

This can range

from one packet to tens of thousands/hundreds of thousands of

packets.

It is a trade of higher performance and the use of more memory

(RAM), or lower performance and less use of memory. A high number

of packets being processed results in a higher performance and the

use of more memory. A low number of packets, results in lower

performance and less use of memory.

Choosing a low number of packets being processed while having many

CPU's/CPU cores, can result in not making use of the whole

computer-capacity(能力 性能). (For instance: using one core

while having three waiting for processing packets.)

max-pending-packets: 1024

Runmodes(运行模式)

By default

the runmode option is disabled

With the

runmodes setting you can set the runmode you would like to use. For

all runmodes available, enter--list-runmodesin your

command line. For more information,

seeRunmodes.(auto, single,

autofp.)

runmode: autofp

Default-packet-size(设置默认的数据包大小)

For the max-pending-packets option, Suricata has to keep packets in

memory. With the default-packet-size option, you can set the size

of the packets on your network. It is possible that bigger packets

have to be processed sometimes. The engine can still process these

bigger packets, but processing it will lower the performance.

default-packet-size: 1514

User and

group(设置用户组)

It is possible to set the user and group to run Suricata

as:

run-as: user: suri group: suri

Action-order(事件处理顺序默认为：pass drop alert

reject)

All signatures

have different properties. One of those is the Action

property(属性). This one

determines(确定) what will happen when a signature

matches.

There are

four types of Action. A summary of what will happen when a

signature matches and contains one of those Actions:

1) Pass

If a signature matches and contains pass, Suricata stops scanning

the packet and skips to the end of all rules (only for the current

packet).

2)

Drop

This only concerns the IPS/inline mode.(这个动作只对IPS或者inline模式用影响)

If the

program finds a signature that matches, containing drop, it stops

immediately. The packet will not be sent any

further.

Drawback:

The receiver does not receive a message of what is going on,

resulting in a time-out (certainly with TCP). Suricata generates an

alert for this packet.

3)

Reject

This is an active rejection of the packet. Both receiver and sender

receive a reject packet(该模式，可以给发送者和接受者返回一个reset数据包来断开连接). There are

two types of reject packets that will be automatically selected. If

the offending packet concerns TCP, it will be a Reset-packet. For

all other protocols it will be an ICMP-error packet. Suricata also

generates(产生) an alert. When in Inline/IPS mode,

the offending packet will also be dropped like with the 'drop'

action.

4) Alert

If a signature matches and contains alert, the packet will be

treated like any other non-threatening packet, except for this one

an alert will be generated by Suricata. Only the system

administrator can notice this alert.

Inline/IPS can block network traffic in two ways. One way is by

drop and the other by reject.

Rules will be

loaded in the order of which they appear in

files(规则的加载顺序是按照它们出现在文件中的顺序). But they will be processed in

a different order. Signatures have different

priorities(但是它们会按照不同的顺序进行处理，因为它们具有不同的优先级). The most important signatures

will be scanned first. There is a possibility to change the order

of priority. The default order is:pass, drop,

reject, alert.

action-order: - pass - drop - reject - alert

This means a pass rule is considered before a drop rule, a drop

rule before a reject rule and so on.

Splitting

configuration in multiple files(将配置文件分成多个文件)

Some users might have a need or a wish to split their suricata.yaml

file in to seperate files, this is available vis the 'include' and

'!include' keyword. The first example is of taking the contents of

the outputs section and storing them in outputs.yaml

# outputs.yaml- fast enabled: yes filename: fast.log append: yes- unified2-alert: enabled: yes...

# suricata.yaml...outputs: !include outputs.yaml...

The second scenario is where multiple sections are migrated to a

different YAML file.

# host_1.yamlmax-pending-packets: 2048 outputs: - fast enabled: yes filename: fast.log append: yes - unified2-alert: enabled: yes

# suricata.yamlinclude: host_1.yaml...

If the same section, say outputs is later redefined after the

include statement it will overwrite the included file. Therefor any

include statement at the end of the document will overwrite the

already configured sections.

Event output

Default logging directory(默认日志文件夹)

In the /var/log/suricata directory, all of Suricata's output

(alerts and events) will be stored.

default-log-dir: /var/log/suricata

This directory

can be overridden by entering the -l command line parameter or by

changing the directory directly in Yaml. To change it with the -l

command line parameter, enter the following:(该默认log文件夹可以被重写，通过-l

命令来改变默认的log文件夹)

suricata -c suricata.yaml -i eth0 -l /var/log/suricata-logs/

Outputs

There are several types of output. The general structure is:

outputs: -fast: enabled: yes filename: fast.log append: yes/no

Enabling all of the logs, will result in a much lower performance

and the use of more disc space, so enable only the outputs you

need.

(打开所有的输出日志，将会导致性能降低 和

占用更多磁盘空间。因此只需要输出自己需要的就行了。)

Line based alerts log (fast.log)

This log

contains alerts consisting of a single line.(fast.log

只包含一些报警的必要信息)

Example of

the appearance of a single fast.log-file line:

10/05/10-10:08:59.667372  [**] [1:2009187:4] ET WEB_CLIENT ACTIVEX iDefense COMRaider ActiveX Control Arbitrary File Deletion [**] [Classification: Web Application Attack] [Priority: 3] {TCP} xx.xx.232.144:80 -> 192.168.1.4:56068

-fast:(日志名称)#The log-name. enabled:yes #This log is enabled. Set to 'no' to disable. filename: fast.log #The name of the file in the default logging directory. append: yes/no #If this option is set to yes, the last filled fast.log-file will not be #overwritten while restarting Suricata.

Eve (Extensible Event Format)

This is an JSON output for alerts and events. It allows for easy

integration with tools like logstash.

# Extensible Event Format (nicknamed EVE) event log in JSON format - eve-log: enabled: yes type: file #file|syslog|unix_dgram|unix_stream filename: eve.json # the following are valid when type: syslog above #identity: "suricata" #facility: local5 #level: Info ## possible levels: Emergency, Alert, Critical, ## Error, Warning, Notice, Info, Debug types: - alert - http: extended: yes # enable this for extended logging information - dns - tls: extended: yes # enable this for extended logging information - files: force-magic: no # force logging magic on all logged files force-md5: no # force logging of md5 checksums #- drop - ssh

For more advanced configuration options,

seeEveJSONOutput.

The format is documented in

Log output for

use with Barnyard (unified.log)(使用barnyard

进行日志输出，这样就可以很好的性能)

This log only supports IPv4. Its information will be stored in the

default logging directory.

This log is designed to be stored in a binary format on the hard

disc, where it will be further processed by Barnyard. Barnyard can

store the output in a database, so Suricata can work on other

important tasks. Barnyard can add the files in the Mysql-database,

send them to Sguil or several other output options.

There is a size-limit to the log-file: If Suricata generates an

alert, it stores this alert in a unified-file. Suricata keeps

continuing doing that, until the file has reached its limit. Which

in the default case is at 32 MB. At that point Suricata generates a

new file and the process starts all over again. Barnyard keeps on

processing these files. To prevent Suricata from filling up the

hard disc, a size limit is enforced. When the limit is reached, the

file will 'role-over', creating a new file. Barnyard removes old

files. To every file, Suricata adds a time stamp, so it is easy to

see which one came first and which one is the latter.

-Unified-log: #The log-name. enabled: no #This log is not enabled. Set 'yes' to enable. filename: unified.log #The name of the file in the default logging directory. limit: 32 #The file size limit in megabytes.

This output option has been removed in Suricata 1.1rc1 (see

ticket #353).(在suricata1.1就已经去掉了该配置)

Alert output for use with Barnyard (unified.alert)

This log only supports IPv4. Its information will be stored in the

default logging directory. For further information read the above information about ( 2)

unified.log)

-Unified-alert: #The log-name. enabled: no #This log is not enabled. Set 'yes' to enable. filename: unified.alert #The name of the file in the default logging directory. limit: 32 #The file size limit in megabytes.

This output option has been removed in Suricata 1.1rc1 (see

ticket

Alert output for use with Barnyard2 (unified2.alert)

This log also supports IPv6 in addition to IPv4. It's information

will be stored in the default logging

directory. For further information read the above information about 2.

unified.log.

- unified2-alert: #The log-name. enabled: yes #This log is enabled. Set 'no' to disable. filename: unified2.alert #The name of the file in the default logging directory. limit: 32 #The file size limit in megabytes.

This alert output needs Barnyard2.

A line based log of HTTP requests (http.log)

This log keeps track of all HTTP-traffic events. It contains the

HTTP request, hostname, URI and the User-Agent. This information

will be stored in the http.log (default name, in the suricata log

directory). This logging can also be performed through the use of

the

Example of a HTTP-log line with non-extended logging:

07/01/2014-04:20:14.338309 vg.no [**] / [**] Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2)AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36 [**]192.168.1.6:64685 -> 195.88.54.16:80

Example of a HTTP-log line with extended logging:

07/01/2014-04:21:06.994705 vg.no [**] / [**] Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2)AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36 [**]

[**]GET [**] HTTP/1.1 [**] 301 => http://www.vg.no/ [**] 239 bytes [**] 192.168.1.6:64726 -> 195.88.54.16:80

- http-log: #The log-name. enabled: yes #This log is enabled. Set 'no' to disable. filename: http.log #The name of the file in the default logging directory. append: yes/no #If this option is set to yes, the last filled http.log-file will not be # overwritten while restarting Suricata. extended: yes # If set to yes more information is written about the event.

A line based log of DNS queries and replies (dns.log)

This log keeps track of all DNS events (queries and replies). It

contains the type of DNS activity that has been performed, the

requested / replied domain name and relevant data suck as client,

server, ttl, resource record data. This logging can also be

performed through the use of the

Example of the apperance of a DNS log of a query with a preceding

reply:

07/01/2014-04:07:08.768100 [**] Query TX 14bf [**] zeustracker.abuse.ch [**] A [**] 192.168.1.6:37681 -> 192.168.1.1:5307/01/2014-04:07:08.768100 [**] Response TX 14bf [**] zeustracker.abuse.ch [**] A [**] TTL 60 [**] 205.188.95.206 [**] 192.168.1.1:53 -> 192.168.1.6:37681

Configuration options:

- dns-log: # The log-name enabled: yes # If this log is enabled. Set 'no' to disable filename: dns.log # Name of this file this log is written to in the default logging directory append: yes # If this option is set to yes, the (if any exists) dns.log file wil not be overwritten while restarting Suricata. filetype: regular / unix_stream / unix_dgram

Packet log (pcap-log)

With the pcap-log option you can save all packets, that are

registered by Suricata, in a log file

named log.pcap. This way, you can take

a look at all packets whenever you want.

In the normal mode a pcap file is created in the default-log-dir.

It can also be created elsewhere if a absolute path is set in the

yaml-file.

The file that is saved in example the default -log-dir

/var/log/suricata, can be be opened with every program which

supports the pcap file format. This can be Wireshark, TCPdump,

Suricata, Snort and many others.

The pcap-log option can be enabled and disabled.

There is a size limit for the pcap-log file that can be set. The

default limit is 32 MB. If the log-file reaches this limit, the

file will be rotated and a new one will be created.

The pcap-log option has an extra functionality

for Sguil that

can be enabled in the 'mode' option.

In the sguil mode the "sguil_base_dir" indicates the base

directory. In this base dir the pcaps are created in a

Sguil-specific directory structure that is based on the day:

$sguil_base_dir/YYYY-MM-DD/$filename.

If you would like to use Suricata with Sguil, do not forget to

enable (and if necessary modify) the base dir in the suricata.yaml

file.

Remember that in the 'normal' mode, the file will be saved in

default-log-dir or in the absolute path (if set).

By default all packets are logged

except: - TCP streams beyond

stream.reassembly.depth - encrypted streams after the key exchange

- pcap-log: enabled: yes filename: log.pcap # Limit in MB. limit: 32 mode: sguil # "normal" (default) or sguil. sguil_base_dir: /nsm_data/

Verbose Alerts Log (alert-debug.log)

This is a log type that gives supplementary information about an

alert. It is particularly convenient for people who investigate

false positives and who write signatures. However, it lowers the

performance because of the amount of information it has to

store.

- alert-debug: #The log-name. enabled: no #This log is not enabled. Set 'yes' to enable. filename: alert-debug.log #The name of the file in the default logging directory. append: yes/no #If this option is set to yes, the last filled fast.log-file will not be # overwritten while restarting Suricata.

Alert output to prelude (alert-prelude)

To be able to use this type, you have to connect with the prelude

manager first.

Prelude alerts contain a lot of information and fields, including

the IPfields in of the packet which triggered the alert. This

information can be divided in three parts:

- The alert description (sensor name, date, ID (sid) of the rule,

etc). This is always included

- The packets headers (almost all IP fields, TCP UDP etc. if

relevant)

- A binary form of the entire packet.

Since the last two parts can be very big (especially since they are

stored in the Prelude SQL database), they are optional and

controlled by the two options 'log_packet_header' and

'log_packet_content'. The default setting is to log the headers,

but not the content.

The profile name is the name of the Prelude profile used to connect

to the prelude manager. This profile must be registered using an

external command (prelude-admin), and must match the uid/gid of the

user that will run Suricata. The complete procedure is detailed in

the

- alert-prelude: #The log-name. enabled: no #This log is not enabled. Set 'yes' to enable. profile: suricata #The profile-name used to connect to the prelude manager. log_packet_content: no #The log_packet_content is disabled by default. log_packet_header: yes #The log _packet_header is enabled by default.

Stats

In stats you can set the options for stats.log.

When enabling stats.log you can set the amount of time in seconds

after which you want the output-data to be written to the log

file.

- stats: enabled: yes #By default, the stats-option is enabled filename: stats.log #The log-name. Combined with the default logging directory #(default-log-dir) it will result in /var/log/suricata/stats.log. #This directory can be overruled with a absolute path. (A #directory starting with / ). interval: 8 #The default amount of time after which the file will be #refreshed. append: yes/no #If this option is set to yes, the last filled fast.log-file will not be #overwritten while restarting Suricata.

Syslog

With this option it is possible to send all alert and event output

to syslog.

- syslog: #This is a output-module to direct log-output to several directions. enabled: no #The use of this output-module is not enabled. facility: local5 #In this option you can set a syslog facility. level: Info #In this option you can set the level of output. The possible levels are: #Emergency, Alert, Critical, Error, Warning, Notice, Info and Debug.

Drop.log, a line based information for dropped packets

If Suricata works in IPS mode, it can drop packets based on rules.

Packets that are being dropped are saved in the drop.log file, a

Netfilter log format.

- drop: enabled: yes #The option is enabled. filename: drop.log #The log-name of the file for dropped packets. append: yes #If this option is set to yes, the last filled drop.log-file will not be #overwritten while restarting Suricata. If set to 'no' the last filled drop.log file will be overwritten.

Detection

engine(检测引擎)

Inspection configuration

The

detection-engine builds internal(内置) groups of signatures. Suricata

loads signatures, with which the network traffic will be compared.

The fact is, that many rules certainly will not be necessary. (For

instance: if there appears a packet with the UDP-protocol, all

signatures for the TCP-protocol won't be

needed.)

For that

reason, all signatures will be divided in groups. However, a

distribution(分布) containing many groups will make use of a lot of

memory(把特征进行分组，把组的数量设的越高，占得内存越高). Not every type of signature gets

its own group. There is a possibility that different signatures

with several properties in common(不同的特征也会有很多相似的地方), will be placed

together in a group.

The

quantity of groups will determine the balance between memory and

performance.A

small amount of groups will lower the performance yet uses little

memory(分组的质量决定了效率和内存的平衡，越少的分组将导致低效率、但是占用的内存也较低). The opposite

counts for a higher amount of groups. The engine allows you to

manage the balance between memory and performance. To manage this,

(by determining the amount of groups) there are several general

options:high for good performance and more use of memory, low for

low performance and little use of memory. The option medium is the

balance between performance and memory usage. This is the default

setting.The option custom is for advanced users. This option has

eight values which can be managed by the user.

detect-engine: -profile: medium #The balance between performance and memory usage. This is the default setting. - custom-values: toclient_src_groups: 2 toclient_dst_groups: 2 toclient_sp_groups: 2 toclient_dp_groups: 3 toserver_src_groups: 2 toserver_dst_groups: 4 toserver_sp_groups: 2 toserver_dp_groups: 25 - sgh-mpm-context: auto - inspection-recursion-limit: 3000

At all of these options, you can add (or change) a value.

Most signatures have the adjustment to focus on one direction,

meaning focusing exclusively on the server, or exclusively(专有的) on

the client.

If

you take a look at example 4, the

Detection-engine grouping tree, you see it has many branches.

At the end of each branch, there is actually a 'sig group

head'. Within that sig group head there is

a container which contains a list with signatures that are

significant for that specific group/that specific end of the

branch. Also within the sig group head the settings for

Multi-Pattern-Matcher (MPM)(多模匹配) can be found: the

MPM-context.

As will be described again at the part 'Pattern matching settings',

there are several MPM-algorithms of which can be chosen

from(有很多的多模匹配算法可供选择). Because every sig group head has its own

MPM-context, some algorithms use a lot of memory. For that reason

there is the option sgh-mpm-context to set whether the groups share

one MPM-context, or to set that every group has its own

MPM-context.

For setting

the option sgh-mpm-context, you can choose from auto, full or

single. The default setting is 'auto', meaning Suricata selects

full or single based on the algorithm you use. 'Full' means that

every group has its own MPM-context, and 'single' that all groups

share one MPM-context.

The two algorithmsacandac-gfbsare

new in 1.03.These algorithms use a single

MPM-context if the Sgh-MPM-context setting is

'auto'. The rest of the algorithms use full

in that case.

The inspection-recursion-limit(递归检查限制) option has to mitigate(减轻

缓和) that possible bugs in Suricata cause big problems. Often

Suricata has to deal with complicated issues. It could end up in an

'endless loop' due to a bug, meaning it will repeat its actions

over and over again. With the option inspection-recursion-limit you

can limit this action.

Example 4 Detection-engine grouping tree

src Stands for source IP-address.dst Stands for destination IP-address.sp Stands for source port.dp Stands for destination port.

Example 5 Detail grouping tree

CUDA (Compute United Device Architecture)(通用并行计算架构)

Suricata

utilizes(利用) CUDA for offloading(卸货 分担) CPU intensive(密集

集中的 ) tasks to the (NVIDIA) GPU (graphics

processing unit). Suricata supports an experimental

multi-pattern-matcher using CUDA.

Only if you have compiled Suricata with CUDA (by entering

--enable-cuda in the configure stage)(configure时 加上 --enable-cuda

该选项)you can make use of

these features.

There are

several options for CUDA.

The option

'packet_buffer_limit' designates (指派、指定)how many packets will be

send to the GPU at the same time. Suricata sends packets in

'batches', meaning it sends multiple packets at once. As soon as

Suricata has collected the amount of packets set in the

'packet_buffer_limit' option, it sends them to the GPU. The default

amount of packets is 2400.

The option 'packet_size_limit' makes sure that packets with

payloads(载荷) bigger than a certain amount of bytes will not be send

to the GPU. Other packets will be send to the GPU. The default

setting is 1500 bytes.

The option 'packet_buffers' designates the amount of buffers that

will be filled with packets and will be processed. Buffers contain

the batches(一组、一批、批量) of packets. During the time these filled

buffers are being processed, new buffers will be filled.

The option

'batching_timeout(超时限制)' can have all values higher

than 0. If a buffers is not fully filled after a period of time

(set in this option 'batching_timeout'), the buffer will be send to

the GPU anyway.(可以设置成大于0的数字，

如果缓冲区没有被填满，在一定时间之后，该缓冲区的数据包一样会发送到GPU进行计算)

The option 'page_locked' designates whether the page locked memory

will or will not be used. The advantage of page locked memory is

that it can not be swapped out to disk. You would not want your

computer to use your hard disk for Suricata, because it lowers the

performance a lot. In this option you can set whether you still

want this for CUDA or not.

The option

'device_id' is an option within CUDA to determine which GPU should

be turned to account.(If there is only one GPU present at your

computer, there is no benefit making use of the 'device-id'

option.)(可以不用该选项)

To detect

the id of your GPU's, enter the following in your command

line:

suricata --list-cuda-cards

With the option 'cuda_streams' you can determine how many

cuda-streams should be used for asynchronous processing. All values

> 0 are valid. For this option you need a device with Compute

Capability > 1.0 and page_locked enabled to have any

effect.()

cuda: -mpm: packet_buffer_limit: 2400 packet_size_limit: 1500 packet_buffers: 10 batching_timeout: 1 page_locked: enabled device_id: 0 cuda_streams: 2

Pattern matcher settings

The multi-pattern-matcher (MPM) is a part of the detection engine

within Suricata that searches for multiple patterns at once.

Generally(通常), signatures have one ore more patterns. Of each

signature, one pattern is used by the multi-pattern-matcher. That

way Suricata can exclude(排除) many signatures from being examined,

because a signature can only match when all its patterns

match.

These are the proceedings: 1)A packet comes in.

2)The packed will be analysed by the Multi-pattern-matcher in

search of patterns that match.

3)All patterns that match, will be further processed by Suricata

(signatures).

Example 8 Multi-pattern-matcher

Suricata offers various implementations of different

multi-pattern-matcher algorithm's. These can be found below.

To set the multi-pattern-matcher algorithm:

mpm-algo: b2gc

After

'mpm-algo', you can enter one of the following algorithms: b2g,

b2gc, b2gm, b3g, wumanber, ac and ac-gfbs (These last two are new

in 1.0.3). For more information about these last

two,please read again

the the end of the part 'Detection engine'.These algorithms

have no options, so the fact that below there is no option being

mentioned (提取)is no omission(忽略).

Subsequently(然后), you can set the options for the

mpm-algorithm's.

Thehash_sizeoption

determines the size of the hash-table that is internal used by the

pattern matcher.A low hash-size

(small table) causes lower memory usage, but decreases(减少性能) the

performance.The opposite counts for a high

hash-size: higher memory usage, but (generally) higher performance.

The memory settings for hash size of the algorithms can vary from

lowest (2048) - low (4096) - medium (8192) - high (16384) - higher

(32768) – max (65536). (Higher is 'highest' in YAML 1.0

-1.0.2)

Thebf_sizeoption determines the size of

the bloom filter, that is used with the final step of the pattern

matcher, namely the validation of the pattern. For this option the

same counts as for the hash-size option: setting it to low will

cause lower memory usage, but lowers the performance. The opposite

counts for a high setting of the bf_size: higher memory usage, but

(generally) higher performance.

The

bloom-filter sizes can vary from low (512) - medium (1024) - high

(2048).

pattern-matcher: - b2gc: search_algo: B2gSearchBNDMq hash_size: low #Determines the size of the hash-table. bf_size: medium #Determines the size of the bloom- filter. - b3g: search_algo: B3gSearchBNDMq hash_size: low #See hash-size -b2gc. bf_size: medium #See bf-size -b2gc. - wumanber: hash_size: low #See hash-size -b2gc. bf_size: medium #See bf-size -b2gc.

Threading(线程配置)

Suricata is multi-threaded. Suricata uses multiple CPU' s/CPU cores

so it can process a lot of network packets simultaneously. (In a

single-core engine, the packets will be processed one at a

time.)

There

are four thread-modules: Packet acquisition(数据包捕获), decode(解码) and

stream application layer(数据流层), detection(检测), and

outputs(输出).

The

packet acquisition module reads packets from the network.

The

decode module decodes the packets and the stream application layer

performs its job here. The stream application layer has three

tasks:

First: it performs stream-tracking, meaning it is making sure all steps will be taken to make a correct network-connection. Second: TCP-network traffic comes in as packets. The Stream-Assembly engine reconstructs the original stream. Finally: the application layer will be inspected. HTTP and DCERPC will be analyzed.

The

detection threads will compare signatures. There can be several

detection threads so they can operate simultaneously.

In

Outputs all alerts and events will be processed.

Example 6 Threading

Packet  : Reads packets from the network Decode: Decodes packets.Stream app. Layer: Performs stream-tracking and reassembly.Detect: Compares signatures.Outputs: Processes all events and alerts.

Most computers have multiple CPU's/ CPU cores. By default the

operating system determines which core works on which thread. When

a core is already occupied, another one will be designated to work

on the thread. So, which core works on which thread, can differ

from time to time.

There is an option within threading:

set-cpu-affinity: no(线程所占用的cpu核心)

With this option you can cause Suricata setting fixed(固定的) cores

for every thread.

In that case 1, 2 and 4 are at core 0 (zero).

Each core has its own detect thread. The detect thread running on

core 0 has a lower priority than the other threads running on core

0. If these other cores are to occupied, the detect thread on core

0 has not much packets to process. De detect threads running on

other cores will process more packets.

This is only the case after setting the option at 'yes'.

Example 7 Balancing workload

You can set the detect-thread-ratio:

detect-thread-ratio: 1.5

The detect

thread-ratio will determine the amount of detect threads. By

default it will be1.5 x the amount of

CPU's/CPUcores present at

your computer. This will result in having more detection threads

then CPU's/ CPU cores. Meaning you are oversubscribing(超额订购) the

amount of cores. This may be convenient(方便) at times when there

have to be waited for a detection thread. The remaining detection

thread can become active.

In the option 'cpu affinity' you can set which CPU's/cores work on

which thread. In this option there are several sets of threads. The

management-, receive-, decode-, stream-, detect-, verdict-, reject-

and outputs-set. These are fixed names and can not be changed. For

each set there are several options: cpu, mode, and

prio. In the option 'cpu' you can set the numbers of the CPU's/cores

which will run the threads from that set. You can set this option

to 'all', use a range (0-3) or a comma separated list

(0,1).

The option 'mode' can be set to 'balanced'(平衡模式) or

'exclusive(专一模式)'. When set to 'balanced', the individual threads

can be processed by all cores set in the option 'cpu'. If the

option 'mode' is set to 'exclusive', there will be fixed cores for

each thread.

As mentioned before, threads can have different priority's. In the

option 'prio' you can set a priority for each thread. This priority

can be low, medium, high or you can set the priority to 'default'.

If you do not set a priority for a CPU, than the settings in

'default' will count.

By default Suricata creates one 'detect' thread per available

CPU/CPU core.

cpu-affinity: - management-cpu-set: cpu: [ 0 ] # include only these cpus in affinity settings - receive-cpu-set: cpu: [ 0 ] # include only these cpus in affinity settings - decode-cpu-set: cpu: [ 0, 1 ] mode: "balanced" - stream-cpu-set: cpu: [ "0-1" ] - detect-cpu-set: cpu: [ "all" ] mode: "exclusive" # run detect threads in these cpus # Use explicitely 3 threads and don't compute number by using # detect-thread-ratio variable: # threads: 3 prio: low: [ 0 ] medium: [ "1-2" ] high: [ 3 ] default: "medium" - verdict-cpu-set: cpu: [ 0 ] prio: default: "high" - reject-cpu-set: cpu: [ 0 ] prio: default: "low" - output-cpu-set: cpu: [ "all" ] prio: default: "medium"

IP

Defrag(IP 分片)

Occasionally network packets appear fragmented(网络数据包有可能出现分片). On

some networks it occurs more often than on others. Fragmented

packets exist of many parts. Before Suricata is able to inspect

these kind of packets accurately, the packets have to be

reconstructed(重组). This will be done by a component(组件) of

Suricata; the defragment-engine. After a fragmented packet is

reconstructed by the defragment-engine, the engine sends on the

reassembled packet to rest of Suricata.

There are

three options within defrag:max-frags, prealloc

and timeout.

At the

moment Suricata receives a fragment of a packet, it keeps in memory

that other fragments of that packet will appear soon to complete

the packet. However, there is a possibility that one of the

fragments does not appear. To prevent Suricata for keeping waiting

for that packet (thereby using memory) there is a timespan after

which Suricata discards the fragments. This occurs by default after

60 seconds.

defrag: max-frags: 65535 prealloc: yes timeout: 60

Flow and Stream handling(数据流处理)

Flow Settings

Within

Suricata, Flows are very important. They play a big part in the way

Suricata organizes data internally. A flow is a bit similar to a

connection, except a flow is more general.All packets having the

sameTuple(五元组)(protocol, source IP,

destination IP, source-port, destination-port), belong to the same

flow. Packets belonging to a flow are connected to it

internally(内部的).

Example 9 Flow

Example 10 Tuple

Keeping

track of all these flows, uses memory. The more flows, the more

memory it will cost.

To

keep control over memory usage, there are several

options:

The optionmemcapfor setting

the maximum amount of bytes the flow-engine will use, hash-size for

setting the size of the hash-table and prealloc for the

following:

For

packets not yet belonging to a flow, Suricata creates a new flow.

This is a relative expensive action. The risk coming with it, is

that attackers /hackers can a attack the engine system at this

part. When they make sure a computer gets a lot of packets with

different tuples, the engine has to make a lot of new flows. This

way, an attacker could flood the system. To mitigate(使缓和...) the

engine from being overloaded, this option instructs Suricata to

keep a number of flows ready in memory. This way Suricata is less

vulnerable to these kind of attacks.

The

flow-engine has a management thread that operates independent from

the packet processing. This thread is called

theflow-manager. This thread ensures

that wherever possible and within the memcap. there will be 10000

flows prepared.

flow: memcap: 33554432 (设置存储数据流的内存大小) #The maximum amount of bytes the flow-engine will make use of. hash_size: 65536 (存储数据回话的hash大小) #Flows will be organized in a hash-table. With this option you can set the #size of the hash-table. Prealloc: 10000 #The amount of flows Suricata has to keep ready in memory.

At the point the memcap will still be reached, despite prealloc,

the flow-engine goes into the emergency-mode. In this mode, the

engine will make use of shorter time-outs. It lets flows expire in

a more aggressive manner(激进的方式) so there will be more space for new

Flows. There are two options: emergency_recovery and prune_flows. The

emergency recovery is set on 30. This is the percentage of

prealloc'd flows after which the flow-engine will be back to normal

(when 30 percent of the 10000 flows is completed).

If during the emergency-mode, the aggressive time-outs do not have

the desired result, this option is the final resort(手段). It ends

some flows even if they have not reached their time-outs yet. The

prune-flows option shows how many flows there will be terminated at

each time a new flow is set up.

emergency_recovery: 30 #Percentage of 1000 prealloc'd flows.prune_flows: 5 #Amount of flows being terminated during the emergency mode.

Flow

Time-Outs(数据流的超时时间设置)

The amount of time Suricata keeps a flow in memory is determined by

the Flow time-out.

There are different states in which a flow can be. Suricata

distinguishes three flow-states for TCP and two for UDP. For TCP,

these are: New, Established and Closed,for UDP only new and

established. For each of these states Suricata can employ different

timeouts.

The state new in a TCP-flow, means the period during the three way

handshake. The state established is the state when the three way

handshake is completed. The state closed in the TCP-flow: there a

several ways to end a flow. This is by means of Reset or the

Four-way FIN handshake.

New in a UDP-flow: the state in which packets are send from only

one direction. Established in a UDP-flow: packets are send from both

directions.

In the example configuration the are settings for each protocol.

TCP, UDP, ICMP and default (all other protocols).

flow-timeouts: default: new: 30 #Time-out in seconds after the last activity in this flow in a New state. established: 300 #Time-out in seconds after the last activity in this flow in a Established #state. emergency_new: 10 #Time-out in seconds after the last activity in this flow in a New state #during the emergency mode. emergency_established: 100 #Time-out in seconds after the last activity in this flow in a Established #state in the emergency mode. tcp: new: 60 established: 3600 closed: 120 emergency_new: 10 emergency_established: 300 emergency_closed: 20 udp: new: 30 established: 300 emergency_new: 10 emergency_established: 100 icmp: new: 30 established: 300 emergency_new: 10 emergency_established: 100

Stream-engine(数据流处理引擎)

The Stream-engine keeps track of the TCP-connections. The engine

exists of two parts: The stream tracking- and the

reassembly-engine.

The stream-tracking engine monitors the state of a connection. The

reassembly-engine reconstructs the flow as it used to be, so it

will be recognised by Suricata.

The stream-engine has two memcaps that can be set. One for the

stream-tracking-engine and one for the reassembly-engine.

The

stream-tracking-engine keeps information of the flow in memory.

Information about the state,TCP-sequence-numbers

and the TCP window. For keeping this information, it can

make use of the capacity the memcap

allows.

TCP

packets have a so-called checksum. This is an internal code which

makes it possible to see if a packet has arrived in a good state.

The stream-engine will not process packets with a wrong checksum.

This option can be set off by entering 'no' instead of

'yes'.

stream: memcap: 33554432 #Amount of flow-information (in bytes) to keep in memory. checksum_validation: yes (校验和检查) #Validate packet checksum, reject packets with invalid checksums.

The option

'max_sessions' is the limit for concurrent sessions. It prevents

Suricata from using all memory for sessions.

To mitigate Suricata from being overloaded by fast session

creation, the option prealloc_sessions instructs Suricata to keep a

number of sessions ready in memory.

A TCP-session starts with the three-way-handshake. After that, data

can be send en received. A session can last a long time. It can

happen that Suricata will be running after a few sessions have

already been started. This way, Suricata misses the original setup

of those sessions. This setup always includes a lot of information.

If you want Suricata to check the stream from that time on, you can

do so by setting the option 'midstream' to 'true'. The default setting

is 'false'.

Normally

Suricata is able to see all packets of a connection. Some networks

make it more complicated though. Some of the network-traffic

follows a different route than the other part, in other words: the

traffic goes asynchronous. To make sure Suricata will check the one

part it does see, instead of getting confused, the option

'async-oneside' is brought to life. By default the option is set to

'false'.

Suricata

inspects content in the normal/IDS mode in chunks. In the

inline/IPS mode it does that on the sliding window way (see example

..) In the case Suricata is set in inline mode, it has to inspect

packets immediately before sending it to the receiver. This way

Suricata is able to drop a packet directly if needed.(see example

…) It is important for Suricata to note which operating system it

is dealing with, because operating systems differ in the way they

process anomalies in streams. See Host-os-policy in

thissuricata.yaml.

max_sessions: 262144 # 256k concurrent sessions prealloc_sessions: 32768 # 32k sessions prealloc'd midstream: false # do not allow midstream session pickups async_oneside: false # do not enable async stream handling inline: no # stream inline mode

Example 11 Normal/IDS mode

Suricata inspects traffic in chunks.

Example 12 Inline/IPS Sliding Window

Suricata inspects traffic in a sliding window manner.

Example 13 Normal/IDS (reasembly on ACK'D data)

Example 14 Inline/IPS (reassembly on UNACK'D data)

The reassembly-engine has to keep packets in memory to be able to

make a reassembled stream. It can make use of the amount of bytes

set below. Reassembling a stream is an expensive operation. In the

option depth you can set the depth (in a stream) of the

reassembling. By default this is 1MB.

reassembly: memcap: 67108864 #Amount of packets (in bytes) to keep in memory. depth: 1048576 #The depth of the reassembling. toserver_chunk_size: 2560 # inspect raw stream in chunks of at least this size toclient_chunk_size: 2560 # inspect raw stream in chunks of at least

Example 15 Stream reassembly

Application Layer Parsers(应用层解析)

Asn1_max_frames (new in 1.0.3 and 1.1)

Asn1 (Abstract

Syntax One) is a standard notation(符号) to structure and

describe data.

Within Asn1_max_frames there are several frames. To protect itself,

Suricata will inspect a maximum of 256. You can set this amount

differently if wanted. Application layer protocols such as X.400 electronic mail, X.500

and LDAP directory services, H.323 (VoIP), BACnet and SNMP, use

ASN.1 to describe the protocol data units (PDUs) they exchange. It

is also extensively used in the Access and Non-Access Strata of

UMTS.

Limit for the maximum number of asn1 frames to decode (default

256):

asn1_max_frames: 256

Configure HTTP (libhtp)

The library Libhtp is being used by Suricata to parse

HTTP-sessions. While processing HTTP-traffic, Suricata has to deal with different

kind of servers which each process anomalies in HTTP-traffic

differently. The most common web-server is Apache. This is a open

source web -server program.

Beside Apache, IIS (Internet Information Services/Server)a

web-server program of Microsoft is also well-known.

Like with host-os-policy, it is important for Suricata to which

IP-address/network-address is used by which server. In Libhtp this

assigning of web-servers to IP-and network addresses is called

personality.

Currently Available Personalities(定制): Minimal Generic IDS (default) IIS_4_0 IIS_5_0 IIS_5_1 IIS_6_0 IIS_7_0 IIS_7_5 Apache Apache_2_2

You can assign names to each block of settings. Which in this case

is -apache and -iis7. Under these names you can set IP-addresses,

network-addresses the personality and the

request-body-limit.

The version-specific personalities know exactly how web servers

behave, and emulate(效仿) that. The IDS personality (will be GENERIC

in the future) would try to implement a best-effort approach that

would work reasonably well in the cases where you do not know the

specifics. The default configuration also applies to every IP-address for

which no specific setting is available.

HTTP request body's are often big, so they take a lot of time to

process which has a significant impact on the performance. With the

option 'request-body-limit' you can set the limit (in bytes) of the

client-body that will be inspected. Setting it to 0 will inspect

all of the body.

HTTP response body's are often big, so they take a lot of time to

process which has a significant impact on the performance. With the

option 'response-body-limit' you can set the limit (in bytes) of

the server-body that will be inspected. Setting it to 0 will

inspect all of the body.

libhtp: default-config: personality: IDS request-body-limit: 3072 response-body-limit: 3072 server-config: - apache: address: [192.168.1.0/24, 127.0.0.0/8, "::1"] personality: Apache_2_2 request-body-limit: 0 response-body-limit: 0 - iis7: address: - 192.168.0.0/24 - 192.168.10.0/24 personality: IIS_7_0 request-body-limit: 4096 response-body-limit: 8192

As of 1.4, Suricata makes available the whole set of libhtp

customisations(量身打造) for its users.

You can now use these parameters in the conf to customise

suricata's use

of libhtp.

# Configures whether backslash characters are treated as path segment # separators. They are not on Unix systems, but are on Windows systems. # If this setting is enabled, a path such as "/one\two/three" will be # converted to "/one/two/three". Accepted values - yes, no. #path-backslash-separators: yes # Configures whether consecutive path segment separators will be # compressed. When enabled, a path such as "/one//two" will be normalized # to "/one/two". The backslash_separators and decode_separators # parameters are used before compression takes place. For example, if # backslash_separators and decode_separators are both enabled, the path # "/one\\/two\/\threefour" will be converted to # "/one/two/three/four". Accepted values - yes, no. #path-compress-separators: yes # This parameter is used to predict how a server will react when control # characters are present in a request path, but does not affect path # normalization. Accepted values - none or status_400 */ #path-control-char-handling: none # Controls the UTF-8 treatment of request paths. One option is to only # validate path as UTF-8. In this case, the UTF-8 flags will be raised # as appropriate, and the path will remain in UTF-8 (if it was UTF-8 in # the first place). The other option is to convert a UTF-8 path into a # single byte stream using best-fit mapping. Accepted values - yes, no. #path-convert-utf8: yes # Configures whether encoded path segment separators will be decoded. # Apache does not do this, but IIS does. If enabled, a path such as # "/one/two" will be normalized to "/one/two". If the # backslash_separators option is also enabled, encoded backslash # characters will be converted too (and subseqently normalized to # forward slashes). Accepted values - yes, no. #path-decode-separators: yes # Configures whether %u-encoded sequences in path will be decoded. Such # sequences will be treated as invalid URL encoding if decoding is not # desireable. Accepted values - yes, no. #path-decode-u-encoding: yes # Configures how server reacts to invalid encoding in path. Accepted # values - preserve_percent, remove_percent, decode_invalid, status_400 #path-invalid-encoding-handling: preserve_percent # Configures how server reacts to invalid UTF-8 characters in path. # This setting will not affect path normalization; it only controls what # response status we expect for a request that contains invalid UTF-8 # characters. Accepted values - none, status_400. #path-invalid-utf8-handling: none # Configures how server reacts to encoded NUL bytes. Some servers will # terminate path at NUL, while some will respond with 400 or 404. When # the termination option is not used, the NUL byte will remain in the # path. Accepted values - none, terminate, status_400, status_404. # path-nul-encoded-handling: none # Configures how server reacts to raw NUL bytes. Some servers will # terminate path at NUL, while some will respond with 400 or 404. When # the termination option is not used, the NUL byte will remain in the # path. Accepted values - none, terminate, status_400, status_404. path-nul-raw-handling: none # Sets the replacement characater that will be used to in the lossy # best-fit mapping from Unicode characters into single-byte streams. # The question mark is the default replacement character. #set-path-replacement-char: ? # Controls what the library does when it encounters an Unicode character # where only a single-byte would do (e.g., the %u-encoded characters). # Conversion always takes place; this parameter is used to correctly # predict the status code used in response. In the future there will # probably be an option to convert such characters to UCS-2 or UTF-8. # Accepted values - bestfit, status_400 and status_404. #set-path-unicode-mapping: bestfit

Engine output(输出引擎)

Logging configuration

The logging subsystem can display all output except alerts and

events. It gives information at runtime about what the engine is

doing. This information can be displayed during the engine startup,

at runtime and while shutting the engine down. For informational

messages, errors, debugging, etc.

The log-subsystem has several log levels:

Error, warning, informational and debug. Note that debug level

logging will only be emitted if Suricata was compiled with the

--enable-debug configure option.

The first option within the logging configuration is the

default-log-level. This option determines the severity/importance

level of information that will be displayed. Messages of lower

levels than the one set here, will not be shown. The default

setting is Info. This means that error, warning and info will be

shown and the other levels won't be. There are more levels: emergency, alert, critical and notice, but

those are not used by Suricata yet. This option can be changed in

the configuration, but can also be overridden in the command line

by the environment variable: SC_LOG_LEVEL .

Logging: default-log-level: info

Default log format

A logging line exists of two parts. First it displays meta

information (thread id, date etc.), and finally the actual log

message. Example:

[27708] 15/10/2010 -- 11:40:07 - (suricata.c:425)

(main) – This is Suricata version 1.0.2

(Here the part until the – is the meta info, “This is Suricata

1.0.2” is the actual message.)

It is possible to determine which information will be displayed in

this line and (the manner how it will be displayed) in which format

it will be displayed.

This option is the so called format string:

default-log-format: "[%i] %t - (%f:%l)