siftr(4) 맨 페이지 - 윈디하나의 솔라나라

개요

섹션
맨 페이지 이름
검색(S)

siftr(4)

To  load  the  driver  as a module at run-time, run the following
command as root: kldload siftr Alternatively, to load the  driver
as  a  module at boot time, add the following line into the file:
siftr_load="YES" The kernel module logs a range of statistics  on
active TCP connections to a log file.  It provides the ability to
make  highly granular measurements of TCP connection state, aimed
at system administrators, developers and  researchers.   The  de‐
fault  operation  of  is  to capture IPv4 TCP/IP packets.  can be
configured  to   support   IPv4   and   IPv6   by   uncommenting:
CFLAGS+=-DSIFTR_IPV6  in  and recompiling.  In the IPv4-only (de‐
fault)   mode,   standard   dotted   decimal    notation    (e.g.
"136.186.229.95")  is  used to format IPv4 addresses for logging.
In IPv6 mode, standard dotted decimal notation is used to  format
IPv4  addresses,  and  standard colon-separated hex notation (see
RFC 4291) is used to format IPv6  addresses  for  logging.   Note
that  SIFTR  uses uncompressed notation to format IPv6 addresses.
For example,  the  address  "fe80::20f:feff:fea2:531b"  would  be
logged  as  "fe80:0:0:0:20f:feff:fea2:531b".  utilises the inter‐
face to export its configuration variables  to  user-space.   The
following  variables  are  available: controls whether the module
performs its measurements or not.  By default, the value  is  set
to 0, which means the module will not be taking any measurements.
Having the module loaded with set to 0 will have no impact on the
performance  of  the network stack, as the packet filtering hooks
are only inserted when is  set  to  1.   controls  how  many  in‐
bound/outbound  packets  for  a given TCP connection will cause a
log message to be generated for the connection.  By default,  the
value  is set to 1, which means the module will log a message for
every packet of every TCP connection.  The value can  be  set  to
any  integer  in  the  range  [1,2^32], and can be changed at any
time, even while the module is enabled.  controls the path to the
file that the module writes its log messages to.  By default, the
file /var/log/siftr.log is used.  The path can be changed at  any
time,  even while the module is enabled.  controls whether a hash
is generated for each TCP packet seen by By default, the value is
set to 0, which means no hashes are generated.   The  hashes  are
useful  to correlate which TCP packet triggered the generation of
a particular log message, but calculating  them  adds  additional
computational  overhead  into  the fast path.  A typical log file
will contain 3 different types of log message.  All messages  are
written  in  plain  ASCII text.  Note: The present in the example
log messages in this section indicates a line continuation and is
not part of the actual log message.  The first type of  log  mes‐
sage is written to the file when the module is enabled and starts
collecting data from the running kernel.  The text below shows an
example  module  enable  log.   The fields are tab delimited key-
value pairs which describe some basic information about the  sys‐
tem.   enable_time_secs=1238556193     enable_time_usecs=462104 \
siftrver=1.2.2    hz=1000    tcp_rtt_scale=32  \  sysname=FreeBSD
sysver=604000    ipmode=4 Field descriptions are as follows: time
at which the module was enabled, in seconds since the UNIX epoch.
time  at  which the module was enabled, in microseconds since en‐
able_time_secs.  version of tick rate of the kernel in ticks  per
second.   smoothed RTT estimate scaling factor.  operating system
name.  operating system version.  IP mode as defined  at  compile
time.   An  ipmode  of "4" means IPv6 is not supported and IP ad‐
dresses are logged in regular dotted quad format.  An  ipmode  of
"6"  means IPv6 is supported, and IP addresses are logged in dot‐
ted quad or hex format, as described in the subsection.  The sec‐
ond type of log message is written to the file when  a  data  log
message  is  generated.  The text below shows an example data log
triggered by an IPv4 TCP/IP packet.  The data is  CSV  formatted.
o,0xbec491a5,1238556193.463551,172.16.7.28,22,172.16.2.5,55931, \
1073725440,172312,6144,66560,66608,8,1,4,1448,936,1,996,255,    \
33304,208,66608,0,208,0 Field descriptions are as follows: Direc‐
tion of packet that triggered the log message.  Either for in, or
for out.  Hash of the packet  that  triggered  the  log  message.
Time  at  which  the  packet  that  triggered the log message was
processed by the hook function, in seconds and microseconds since
the UNIX epoch.  The IPv4 or IPv6 address of the local  host,  in
dotted  quad  (IPv4  packet) or colon-separated hex (IPv6 packet)
notation.  The TCP port that the local host is communicating via.
The IPv4 or IPv6 address of the  foreign  host,  in  dotted  quad
(IPv4 packet) or colon-separated hex (IPv6 packet) notation.  The
TCP  port  that  the foreign host is communicating via.  The slow
start threshold for the flow, in bytes.  The  current  congestion
window  for the flow, in bytes.  The current bandwidth-controlled
window for the flow, in bytes.  The current  sending  window  for
the  flow,  in  bytes.  The post scaled value is reported, except
during the initial handshake (first few  packets),  during  which
time  the unscaled value is reported.  The current receive window
for the flow, in bytes.  The post  scaled  value  is  always  re‐
ported.   The  current window scaling factor for the sending win‐
dow.  The current window scaling factor for the receiving window.
The current state of the TCP finite state machine, as defined  in
The  maximum  segment  size  for the flow, in bytes.  The current
smoothed RTT estimate for the flow, in units of  TCP_RTT_SCALE  *
HZ, where TCP_RTT_SCALE is a define found in tcp_var.h, and HZ is
the kernel's tick timer.  Divide by TCP_RTT_SCALE * HZ to get the
RTT in secs.  TCP_RTT_SCALE and HZ are reported in the enable log
message.  SACK enabled indicator. 1 if SACK enabled, 0 otherwise.
The  current state of the TCP flags for the flow.  See for infor‐
mation about the various flags.  The current retransmission time‐
out length for the flow, in units of HZ, where HZ is the kernel's
tick timer.  Divide by HZ to get the timeout length  in  seconds.
HZ  is  reported  in the enable log message.  The current size of
the socket send buffer in bytes.  The current number of bytes  in
the  socket  send buffer.  The current size of the socket receive
buffer in bytes.  The current number of bytes in the  socket  re‐
ceive  buffer.   The  current  number of unacknowledged bytes in-
flight.  Bytes acknowledged via SACK are not excluded  from  this
count.   The  current number of segments in the reassembly queue.
Flowid for the connection.  A caveat: Zero '0' either  represents
a valid flowid or a default value when it's not being set.  There
is no easy way to differentiate without looking at actual network
interface card and drivers being used.  Flow type for the connec‐
tion.   Flowtype defines which protocol fields are hashed to pro‐
duce the flowid.  A complete listing is available  in  under  The
third  type of log message is written to the file when the module
is disabled and ceases collecting data from the  running  kernel.
The  text  below shows an example module disable log.  The fields
are tab delimited key-value pairs which provide statistics  about
operations  since  the  module  was  most recently enabled.  dis‐
able_time_secs=1238556197    disable_time_usecs=933607 \  num_in‐
bound_tcp_pkts=356        num_outbound_tcp_pkts=627     \     to‐
tal_tcp_pkts=983    num_inbound_skipped_pkts_malloc=0 \  num_out‐
bound_skipped_pkts_malloc=0     num_inbound_skipped_pkts_mtx=0  \
num_outbound_skipped_pkts_mtx=0    num_inbound_skipped_pkts_tcb=0
\           num_outbound_skipped_pkts_tcb=0               num_in‐
bound_skipped_pkts_icb=0 \ num_outbound_skipped_pkts_icb=0    to‐
tal_skipped_tcp_pkts=0                                          \
flow_list=172.16.7.28;22-172.16.2.5;55931, Field descriptions are
as follows: Time at which the module  was  disabled,  in  seconds
since  the UNIX epoch.  Time at which the module was disabled, in
microseconds since disable_time_secs.  Number of TCP packets that
traversed up the network stack.  This only includes  inbound  TCP
packets during the periods when was enabled.  Number of TCP pack‐
ets  that  traversed  down the network stack.  This only includes
outbound TCP packets during the periods when  was  enabled.   The
summation   of  num_inbound_tcp_pkts  and  num_outbound_tcp_pkts.
Number of inbound packets that  were  not  processed  because  of
failed  malloc() calls.  Number of outbound packets that were not
processed because of failed malloc() calls.   Number  of  inbound
packets  that  were  not  processed because of failure to add the
packet to the packet processing queue.  Number of outbound  pack‐
ets  that were not processed because of failure to add the packet
to the packet processing queue.  Number of inbound  packets  that
were  not  processed  because  of failure to find the TCP control
block associated with the packet.   Number  of  outbound  packets
that  were  not processed because of failure to find the TCP con‐
trol block associated with the packet.  Number of inbound packets
that were not processed because of failure to find the IP control
block associated with the packet.   Number  of  outbound  packets
that were not processed because of failure to find the IP control
block  associated  with the packet.  The summation of all skipped
packet counters.  A CSV list of TCP flows that triggered data log
messages to be generated since the module was loaded.  Each  flow
entry  in the CSV list is formatted as If there are no entries in
the list (i.e., no data log messages were generated),  the  value
will  be  blank.   If  there is at least one entry in the list, a
trailing comma will always be present.  The total number of  data
log  messages  found  in the log file for a module enable/disable
cycle should equate to total_tcp_pkts  -  total_skipped_tcp_pkts.
hooks into the network stack using the interface.  In its current
incarnation,  it hooks into the AF_INET/AF_INET6 (IPv4/IPv6) fil‐
tering points, which means it sees packets at the IP layer of the
network stack.  This means that TCP packets inbound to the  stack
are intercepted before they have been processed by the TCP layer.
Packets  outbound  from the stack are intercepted after they have
been processed by the TCP layer.  The diagram  below  illustrates
how        inserts       itself       into       the       stack.
----------------------------------
           Upper Layers ----------------------------------
    ^                       |
    |                       |
    |                       |
    |                       v
 TCP            in                             TCP            out
----------------------------------
    ^                      |
    |________     _________|
            |     |
            |     v
           ---------
           | SIFTR |
           ---------
            ^     |
    ________|     |__________
    |                       |
    |                       v IPv{4/6} in            IPv{4/6} out
----------------------------------
    ^                       |
    |                       |
    |                        v Layer 2 in             Layer 2 out
----------------------------------
          Physical Layer ----------------------------------  uses
the  interface  to manage writing data to disk.  At first glance,
you might mistakenly think that extracts information  from  indi‐
vidual  TCP  packets.   This  is  not  the case.  uses TCP packet
events (inbound and outbound) for each TCP flow originating  from
the  system  to  trigger  a  dump of the state of the TCP control
block for that flow.  With the PPL set to 1,  we  are  in  effect
sampling  each  TCP  flow's  control block state as frequently as
flow packets enter/leave the system.  For example, setting PPL to
2 halves the sampling rate i.e., every second  flow  packet  (in‐
bound OR outbound) causes a dump of the control block state.  The
distinction  between  interrogating individual packets versus in‐
terrogating the control block is important, because does not  re‐
move  the need for packet capturing tools like allows you to cor‐
relate and observe the cause-and-affect relationship between what
you see on the wire (captured using a tool like  and  changes  in
the  TCP control block corresponding to the flow of interest.  It
is therefore useful to use and a tool like to gather  the  neces‐
sary  data to piece together the complete picture.  Use of either
tool on its own will not be able to provide all of the  necessary
data.   As  a  result  of  needing to interrogate the TCP control
block, certain packets during the lifecycle of a  connection  are
unable  to  trigger  a  log message.  The initial handshake takes
place without the existence of a control block and the final  ACK
is  exchanged  when the connection is in the TIMEWAIT state.  was
designed to minimise the delay introduced to  packets  traversing
the network stack.  This design called for a highly optimised and
minimal  hook  function that extracted the minimal details neces‐
sary whilst holding the packet up, and passing these  details  to
another  thread  for  actual processing and logging.  This multi‐
threaded design does introduce some contention  issues  when  ac‐
cessing  the  data structure shared between the threads of opera‐
tion.  When the hook function  tries  to  place  details  in  the
structure,  it  must  first acquire an exclusive lock.  Likewise,
when the processing thread tries to read details from the  struc‐
ture,  it  must  also acquire an exclusive lock to do so.  If one
thread holds the lock, the other must wait before it  can  obtain
it.   This  does introduce some additional bounded delay into the
kernel's packet processing code path.  In some cases  (e.g.,  low
memory,  connection termination), TCP packets that enter the hook
function will not trigger a log message to be generated.   refers
to  this  outcome  as a Note that always ensures that packets are
allowed to continue through the stack, even  if  they  could  not
successfully  trigger a data log message.  will therefore not in‐
troduce any packet loss for TCP/IP packets traversing the network
stack.  The behaviour of a log file path change whilst the module
is enabled is as follows: Attempt to open the new file  path  for
writing.  If this fails, the path change will fail and the exist‐
ing  path  will  continue  to  be used.  Assuming the new path is
valid and opened successfully: Flush all pending log messages  to
the  old  file path.  Close the old file path.  Switch the active
log file pointer to point at the new file path.  Commence logging
to the new file.  During the time between the  flush  of  pending
log  messages  to  the old file and commencing logging to the new
file, new log messages will still be generated and buffered.   As
soon  as  the new file path is ready for writing, the accumulated
log messages will be written out to the file.  To enable the mod‐
ule's operations, run  the  following  command  as  root:  sysctl
net.inet.siftr.enabled=1  To  change  the granularity of log mes‐
sages such that 1 log message is generated for every 10 TCP pack‐
ets per connection, run the following  command  as  root:  sysctl
net.inet.siftr.ppl=10   To   change  the  log  file  location  to
/tmp/siftr.log,  run  the  following  command  as  root:   sysctl
net.inet.siftr.logfile=/tmp/siftr.log  Development  of this soft‐
ware was made possible in part by grants from the  Cisco  Univer‐
sity  Research  Program Fund at Community Foundation Silicon Val‐
ley, and the FreeBSD Foundation.  first appeared in and was first
released in 2007 by Lawrence Stewart and James Healy whilst work‐
ing on the NewTCP research project  at  Swinburne  University  of
Technology's  Centre  for  Advanced  Internet Architectures, Mel‐
bourne, Australia, which was made possible in  part  by  a  grant
from  the  Cisco  University  Research  Program Fund at Community
Foundation  Silicon  Valley.   More  details  are  available  at:
http://caia.swin.edu.au/urp/newtcp/  Work on v1.2.x was sponsored
by the FreeBSD Foundation as part of the project 2008-2009.  More
details  are  available   at:   http://www.freebsdfoundation.org/
http://caia.swin.edu.au/freebsd/etcp09/  was  written by and This
manual page was written by Current known limitations and any rel‐
evant workarounds are outlined below: The internal queue used  to
pass  information  between  the threads of operation is currently
unbounded.  This allows to cope with bursty network traffic,  but
sustained  high packet-per-second traffic can cause exhaustion of
kernel memory if the processing thread cannot keep  up  with  the
packet  rate.   If  using on a machine that is also running other
modules utilising the framework e.g.  the order in which you load
the modules is important.  You should kldload the  other  modules
first,  as this will ensure TCP packets undergo any necessary ma‐
nipulations before and processes them.  There is a  known,  harm‐
less  lock  order  reversal warning between the mutex and tcbinfo
TCP lock reported by when is enabled in a  kernel  compiled  with
support.   There  is no way to filter which TCP flows you wish to
capture data for.  Post processing is required  to  separate  out
data  belonging to particular flows of interest.  The module does
not detect deletion of the log file path.  New log messages  will
simply  be  lost  if the log file being used by is deleted whilst
the module is set to use the file.  Switching to a new  log  file
using  the  variable  will create the new file and allow log mes‐
sages to begin being written to disk again.   The  new  log  file
path must differ from the path to the deleted file.  The hash ta‐
ble  used  within the code is sized to hold 65536 flows.  This is
not a hard limit, because chaining is used to  handle  collisions
within  the  hash table structure.  However, we suspect (based on
analogies with other hash table performance data) that  the  hash
table look up performance (and therefore the module's packet pro‐
cessing performance) will degrade in an exponential manner as the
number  of  unique flows handled in a module enable/disable cycle
approaches and surpasses 65536.  There is no  garbage  collection
performed  on  the  flow  hash  table.  The only way currently to
flush it is to disable The PPL variable applies to  packets  that
make it into the processing thread, not total packets received in
the  hook  function.  Packets are skipped before the PPL variable
is applied, which means there may be a slight discrepancy in  the
triggering  of  log messages.  For example, if PPL was set to 10,
and the 8th packet since the last log  message  is  skipped,  the
11th  packet  will  actually trigger the log message to be gener‐
ated.  This is discussed in greater depth in CAIA  technical  re‐
port 070824A.  At the time of writing, there was no simple way to
hook  into  the  TCP layer to intercept packets.  use of IP layer
hook points means all IP traffic will be processed  by  the  hook
function,  which  introduces  minor,  but nonetheless unnecessary
packet delay and processing overhead on the  system  for  non-TCP
packets  as  well.   Hooking in at the IP layer is also not ideal
from the data gathering point of view.  Packets traversing up the
stack will be intercepted and cause a log message generation  BE‐
FORE  they  have  been processed by the TCP layer, which means we
cannot observe the cause-and-affect relationship between  inbound
events  and  the  corresponding TCP control block as precisely as
could be.  Ideally, should intercept packets after they have been
processed by the TCP layer i.e.  intercept packets coming up  the
stack  after  they have been processed by tcp_input(), and inter‐
cept packets coming down the stack after they have been processed
by tcp_output().  The current code still gives satisfactory gran‐
ularity though,  as  inbound  events  tend  to  trigger  outbound
events,  allowing  the cause-and-effect to be observed indirectly
by capturing the state on outbound events  as  well.   The  value
logged  by does not take into account bytes that have been by the
receiving host.  Packet hash generation does not  currently  work
for  IPv6 based TCP packets.  Compressed notation is not used for
IPv6 address representation.  This consumes more  bytes  than  is
necessary in log output.





맨 페이지 내용의 저작권은 맨 페이지 작성자에게 있습니다.
RSS ATOM XHTML 5 CSS3