We started producing shows as Today with a Techie on 2005-09-19, 13 years, 11 months, 14 days ago. Our shows are produced by listeners like you and can be on any topic that "are of interest to hobbyists". If you listen to HPR then please consider contributing one show a year. If you record your show now it could be released in 14 days.
In this episode we'll talk about filtering and dissecting packet traces and streams and introduce diffing. Remember that most tools have very flexible options for a variety of use cases. So check their manpages. Each man page also has multiple examples of how to use each tool.
Lets start with grabbing a trace from the unit tests:
$ mkdir /tmp/packets
$ cd /tmp/packets
$ cp /path/to/onics/tests/data/packets/sample.xpkt .
Lets see what we have inside. First, lets see how many packets there are. We'll use a new tool 'pcount'.
$ pcount sample.xpkt
90 total packets and 19082 total bytes.
Good thing we looked first. Don't want to walk through all the packets.
Scanning Packet Flows
Well, lets look at the connections or "flows" in the trace. We'll do this by using the 'nftrk' command for "network flow tracker".
Like 'pcount' this utility (and many or most ONICS utilities), this program can run on a live stream or a trace file. We'll run:
I demonstrate the use of a vintage home studio device
Hosted by Jon Kulp on 2019-08-22 is flagged as Clean and released under a CC-BY-SA license. Tags:Multi-Track Recording, Recording Devices, Home Recording Studios, Cassette Tapes, Vintage Recording .
Listen in ogg, spx, or mp3 format.
I discuss and demonstrate the latest retro gadget I found at the flea market last weekend, a TASCAM Porta 02 MiniStudio 4-Track Cassette Recorder. It was in a bin full of junk—filthy, lacking its power supply, and I got it for only $5. I hacked a power supply, disassembled it completely, washed everything thoroughly, and put it back together. It worked perfectly with the exception of the pause button. This has been one of the most fun projects I can remember, especially because my daughter is into it too, and she's learning how to make multi-track recordings. I always wanted a 4-track when I was in high school but never had one. Now I do!
It's been about 6 years since I talked about my project ONICS in HPR 1350
ONICS stands for Open Network Inpection Command Suite
I created ONICS as because I thought it would be neat to have a suite of tools that could manipulate packets on the command line in a way similar to how tools lik sed, awk, grep, cut, and so forth manipulate text.
Not currently maintained in any package distributions
Maintainers who are interested in doing so are welcome
Install by source
$ git clone https://gitlab.com/catlib/catlib
$ cd catlib
$ cd ..
$ git clone https://gitlab.com/onics/onics
$ cd onics
$ make test
$ sudo make install
$ make veryclean
Can always uninstall cleanly from the source directory
$ make uninstall
Alternate to installation is to stop at 'make test' and then add to 'onics/bin' and 'onics/scripts' to your path.
Manpages are available in onics/doc directory if you aren't installing locally. They are quite extensive.
If installed locally, starting with:
$ man onics
PCAP format is outdated and not very extensible
I want to be able to annotate with interface IDs, flow IDs, packet numbers, classification info, header offsets, etc...
First and foremost, the file header prevents just cating files together.
it makes merging live streams more difficult
pcapng improves things but still has global file header
Let's first capture in the traditional way
$ sudo tcpdump -i eth0 -c 5 -w file1.pcap
First program is to capture packets from the wire:
$ sudo pktin eth0 > file2.xpkt
If not running as root
$ sudo chown myname file1.pcap file2.xpkt
Let's dump them:
$ tcpdump -r file1.pcap
$ xpktdump file2.xpkt
Now lets convert the PCAP to XPKT
$ pc2xpkt file1.pcap file1.xpkt
$ pc2xpkt file1.pcap > file1.xpkt
$ pc2xpkt < file1.pcap > file1.xpkt
$ cat file1.pcap | pc2xpkt > file1.xpkt
Inspired by a Jon Kulp show, Ken splits a large recording based on silence between tracks
Hosted by Ken Fallon on 2019-08-19 is flagged as Explicit and released under a CC-BY-SA license. Tags:Audacity, hpr1771, Detect Silence, Split Track, Label.
Listen in ogg, spx, or mp3 format.
We take the ideas we have developed over the previous episodes and use them to evaluate a a study I found online. These are things anyone can do with just a little work on Google, and the payoff is to have a good idea of whether or not you are looking at a quality study
Type classes are Haskell’s way of doing ad hoc polymorphics or overloading. They are used to defined set of functions that can operate more than one specific type of data.
In Haskell there’s no default equality, it has to be defined.
There’s two parts to the puzzle. First is type class Eq that comes with the standard library and defines function signatures for equality and non-equality comparisons. There’s type parameter a in the definition, which is filled by user when they define instance of Eq for their data. In that instance definition, a is filled with concrete type.
class Eq a where
(==) :: a -> a -> Bool
(/=) :: a -> a -> Bool
x /= y = not (x == y)
Definition above can be read as “class Eq a that has two functions with following signatures and implementations”. In other words, given two a, this function determines are they equal or not (thus Bool as return type). /= is defined in terms of ==, so it’s enough to define one and you get other one for free. But you can still define both if you’re so included (maybe some optimization case).
If we define our own Size type, like below, we can compare sizes:
data Size = Small | Medium | Large
deriving (Show, Read)
instance Eq Size where
Small == Small = True
Medium == Medium = True
Large == Large = True
_ == _ = False
And here’s couple example comparisons.
> Small == Small
> Large /= Large
Writing these by hand is both tedious and error prone, so we usually use automatic derivation for them. Note how the second line now reads deriving (Show, Read, Eq).
data Size = Small | Medium | Large
deriving (Show, Read, Eq)
Hierarchy between type classes
There can be hierarchy between type classes, meaning one requires presence of another. Common example is Ord, which is used to order data.
class Eq a => Ord a where
compare :: a -> a -> Ordering
(<) :: a -> a -> Bool
(>=) :: a -> a -> Bool
(>) :: a -> a -> Bool
(<=) :: a -> a -> Bool
max :: a -> a -> a
min :: a -> a -> a
This definition can be read as “class Ord a, where a has instance of Eq, with pile of functions as follows”. Ord has default implementation for quite many of these, in terms of others, so it’s enough to implement either compare or <=.
For our Size, instance of Ord could be defined as:
instance Ord Size where
Small <= _ = True
Medium <= Small = False
Medium <= _ = True
Large <= Large = True
Large <= _ = False
Writing generic code
There’s lots and lots of type classes in standard library:
Num for numeric operations
Integral for integer numbers
Floating for floating numbers
Show for turning data into strings
Read for turning strings to data
Enum for sequentially ordered types (these can be enumerated)
Bounded for things with upper and lower bound
and so on…
Type classes allow you to write really generic code. Following is contrived example using Ord and Show:
check :: (Ord a, Show a) => a -> a -> String
check a b =
case compare a b of
show a ++ " is smaller than " ++ show b
show a ++ " is greater than " ++ show b
show a ++ " and " ++ show b ++ " are equal"
Check takes two parameters that are same type and that type has to have Ord and Show instances. Ord is for ordering and Show is for turning data into string (handy for displaying it). The end result is string telling result of comparison. Below is some examples of usage. Note how our function can handle different types of data: Size, Int and [Int].
> check Medium Small
"Medium is greater than Small"
> check Small Large
"Small is smaller than Large"
> check 7 3
"7 is greater than 3"
> check [1, 2] [1, 1, 1]
"[1, 2] is greater than [1, 1, 1]"
There are many extensions to type classes that add more behaviour. These aren’t part of standard Haskell, but can be enabled with a pragma definition or compiler flag. They can be somewhat more complicated to use, have special cases that need careful consideration, but offer interesting options.
Thank you for listening. Question, comments and feedback welcome. Best way to catch me nowadays is either by email or in fediverse, where I’m email@example.com.
I use pdmenu a lot to help me do work on my main desktop PC. I did an HPR show on pdmenu on 13 December 2017 and the author Joey Hess responded in show 2459.
In the intervening time I have also integrated Zenity into my menus. This is a GUI tool which generates a number of different pop-up windows known as dialogs, which can display information, or into which information can be typed. The capabilities provided by pdmenu are a little too basic to enable me to do what I need to do.
I thought it might be of interest to show some examples of how I use this tool with pdmenu.
I have provided detailed notes as usual for this episode, and these can be viewed here.
Released: 2019-07-24. Duration: 00:15:55. Flag: Clean. Tags:Application development, Application architecture, Security.
A thought experiment in whether reducing runtime dependencies can improve security and how to do it.
Released: 2019-07-23. Duration: 00:13:50. Flag: Clean. Series:Random Elements of Storytelling. Tags:stories, storytelling, art, commerce, lostnbronx.
Lostnbronx examines stories as both art and products.
Released: 2019-06-27. Duration: 00:31:08. Flag: Clean. Tags:Music,Recording,Audio,Tape,Reel-To-Reel,Open-Reel,Recording Devices,Tape Speeds,Bash Scripting.
I talk about my latest thrift-store gadget, a 1969 Sony portable reel-to-reel tape recorder
Released: 2019-06-24. Duration: 00:31:04. Flag: Clean. Series:How I Found Linux. Tags:linux, intro, story, discourse, bsd.
A response to the request for "how i got into linux" and a little of my history with Linux/BSD