Site Map - skip to main content

Hobby Public Radio

Your ideas, projects, opinions - podcasted.

New episodes Monday through Friday.


hpr2944 :: ONICS Basics Part 4: Network Flows and Connections

I try to add a bit more basic networking info while writing a quick script for Dave Morris

<< First, < Previous, Latest >>

Host Image
Hosted by Gabriel Evenfire on 2019-11-14 is flagged as Clean and is released under a CC-BY-SA license.
Tags: command line networking.
Listen in ogg, spx, or mp3 format. | Comments (1)

Part of the series: Networking

This series will try and explain the basics of networking to the listener as well as introduce more detailed topics.

Terminology

  • connection - a bi-directional communication channel between two programs over a network
  • client - the initiator of a connection
  • server - the receiver of the connection
  • port - a common term for the address of a program or service on a given machine
  • 5-tuple - the combination of protocol, client machine network address, client port, server machine network address, server port that uniquely identifies a connection
  • flow - a grouping of packets to be treated in a common way
  • microflow - a flow with a fine level of granularity such as the packets from one direction of traffic in a connection

The topflow.sh Script

#!/bin/sh

# Start a capture in the background that drops the packets
# and just reports the flow events
pktin $1 | nftrk -d -f /tmp/flows.txt &
PID=$!

# On CTRL-C clean kill the capture and clean up
trap "kill $PID ; rm -f /tmp/flows.txt /tmp/topflows.txt /tmp/namecache.txt ; exit 0" INT TERM

# Once per second do
#   look at the last 100 flows
#   sort them by 5-tuple
#   remove duplicates
#   convert ports, protocols and addresses to names
#   sort by data usage per flow in reverse order (highest first)
#   a little more pretty printing
#   only take the top 20 lines
#   clear the screen and print the result
while [ 1 ] ; do
    tail -100 /tmp/flows.txt |
            sort -s -t '|' -k 3,3 |
            awk -f uniqflows.awk  |
            awk -f prflow.awk  |
            sort -s -t ',' -k 3 -r |
            awk -f columns.awk |
            head -20 > /tmp/topflows.txt
    clear
    cat /tmp/topflows.txt
    sleep 1
done

You can find the complete code at: https://gitlab.com/onics/onics-examples


Comments

Subscribe to the comments RSS feed.

Comment #1 posted on 2019-11-27T22:07:16Z by Dave Morriss

This is wonderful

Hi Gabriel,

It's been a busy month and I have only just caught up with this show.

I'm amazed by what you have done here. I was running the script while my family were visiting and could see them checking Reddit and YouTube, etc! I could see my main router doing its thing, and my secondary router (being used mainly as a wireless access point and Ethernet switch) also doing what it does. I was impressed the display showed the names I had allocated in /etc/hosts ;-)

Now they have left it's all a lot quieter with my mail client checking various mail feeds and Mastodon updating itself. Fascinating!

I had a fairly detailed look at your Bash and Awk scripts. Impressive. I shall look further later. As you say, Perl would perhaps be better, but it's great to see how powerful (and lightweight) sed and awk can be.

Thanks for putting this together. I really enjoyed this episode.

Dave

<< First, < Previous, Latest >>

Leave Comment

Note to Verbose Commenters
If you can't fit everything you want to say in the comment below then you really should record a response show instead.

Note to Spammers
All comments are moderated. All links are checked by humans. We strip out all html. Feel free to record a show about yourself, or your industry, or any other topic we may find interesting. We also check shows for spam :).

Provide feedback
Your Name/Handle:
Title:
Comment:
Anti Spam Question: What does the P in HPR stand for ?
Are you a spammer →
Who hosted this show →
What does HPR mean to you ?