Session 09 - Standard Linux Tools - Gathering System Information

Introduction

We will see some basic commands to get to know a running system a little bit better. You should be testing the commands at a terminal as you work through these notes. We will see a few commands to quickly find out with what kind of files we are dealing with, and we'll hae a look at tar.

Before we begin

Remember man?

man is your friend. All commands we discuss have corresponding man pages. The commands discussed here have many more options and possibilities than would fit here.

Saving your work

During this exercise (well, actually throughout all exercises you have done), and in investigations in general, you might want to save your work.

The script command is useful here, as it writes all commands you type and their output in a file.

$ script mycommands.txt

Exit with CTRL-D, history is then found in mycommands.txt

What do you do when you encounter a running machine?

You'll have to investigate. Remember, Linux is a multi-user system. When you think you've secured a room and computer, the suspect's friend is remotely destroying all evidence.

But maybe he's in the middle of a chat session and has open crypto containers. Who knows. Every situation is different, and you'll have to decide what best fits the situation.

To view the date and time on a system, use

$ date

For how long has the system been running?

$ uptime

What kind of system is this anyway?

$ uname -a

$ cat /etc/issue

$ cat /etc/lsb-release

Who is logged in?

$ who

$ last

Who is running what?

$ top (exit with q)

$ ps

Which active network connections are there?

$ netstat (older systems)

$ ss (newer systems; but both netstat and ss may be present)

Maybe there are remotely mounted file systems present?

$ mount

Which (disk) devices are there?

$ df -h

Note: The output of some of these commands might overlap.

What has the user been doing?

$ history

TIP: When you've invented some nice tricks or commands or something you want to keep, use

$ history > myclevercommands.txt

for future reference.

Log rotation

Commands like last retrieve information from log files. System-specific log files are usually stored in the /var/log/ directory. In this case, the last log is stored in /var/log/wtmp.

The system maintenance procedures rotate log files to save space, which means you might see files such as wtmp.1, wtmp.2, wtmp.3, etc. If you want to look further back you'll have to point last to one of these files, by using, for example:

$ last -f /var/log/wtmp.n (where n is the number)

file

We've already seen file in action in lessons and other exercises, but just to refresh your mind and perhaps clarify, here is the run-down again.

file is a command which tells you what type of file a given file is (e.g., picture, video, audio, Word document, compressed archive, etc.).

file looks into the file and makes an educated guess about the file type. It uses a small database which contains file-header specifics ("magic").

Since file looks into the file, it can't be fooled by changing filenames and/or extensions. For example:

$ file *
PKpixExport.jpg: PE executable for MS Windows (GUI) Intel 80386 32-bit

test.txt: JPEG image data, JFIF standard 1.01, comment: " copyright 2000 John. "

Odd, right? The jpg is not a picture, despite the extension suggesting it is, and the txt file is not a text file, despite the extension suggesting it is.

strings

Shows printable characters (i.e., strings) in a file. If you have an image file, run strings on it, and it will print all readable strings in it. If the file is big, redirect the output of strings to another file. This will save you time in subsequent searches:

$ strings filename > filename.txt (the redirect is entirely optional)

Or, you can also, as seen in another exercise, pass its output to another program, such as grep to search for, for example, a word, at the same time as strings is trawling through the file:

$ strings filename | grep word

tar

Tape archiver (tar) is a program to create backups. Don't be misled by the name. TAR files are very common in the Linux world. Often when you download source files, they come in a compressed TAR archive (e.g., file.tar.gz, file.tar.bz2, file.tgz).

tar archives multiple files into one file (the archive). Archives are sometimes referred to as "tarballs". tar doesn't do compression by default. You'll have to specify that with the z flag (for GZip-type compression (.tar.gz), there are others too (e.g., .tar.bz2, .tar.xz), but GZip is very common).

Commonly used flags are:

• f usually the last flag given before the file's name (see example below)
• x extract archive
• c create archive
• t list archive contents
• z compress or decompress (depends on c or x flag)
• v verbose (so you're seeing what's happening; by default tar doesn't give any feedback and will just give you your prompt back when it's successfully finished executing)

To "untar" (i.e., unzip) a file called sources.tgz:

$ tar xzf sources.tgz (note where each flag sits, f being, as mentioned earlier, the last one before the file's name is important!)

Suppose we have a directory /home/steve/forupload/ with many files in it. Now suppose we want to create an archive with all files from the directory /home/steve/forupload/ named myuploads.tgz:

$ tar czf myuploads.tgz /home/steve/forupload

awk

awk is a tool with which you can process files one line at a time. At the same time, every line can be divided up into columns (or fields).

awk uses, by default, a space as a field separator (aka. delimiter):

• -F describes the field separator, by default it's space, but you can also change it to something else with this -F flag, for example, -F ',' changes the field separator to use a comma instead.

• awk -F ':' '{print $1}' /etc/passwd this prints usernames from /etc/passwd (where fields are delimited with a colon :)

Fields in awk are prefixed with a $, thus, $1 is the first field of the current line.

On a side note; with awk it is possible to count the lines in a file:

$ awk '{sum += 1} END {print sum}' filename

The action after END will be executed after all lines are read. At every line, 1 is added to sum (sum is initialised to 0 in the beginning). Of course, if you have wc available as a command, it's arguably easier to use than to count lines.

Summing file sizes

Suppose we have a tarred compressed file called myfiles.tgz. We can list the contents, as follows:

$ tar tzvf myfiles.tgz

The third column of the output of the following command lists the original file sizes of the packed files:

$ tar tzvf myfiles.tgz | awk '{sum += $3} END {print "Total size: " sum " bytes."}'

More actions on one line:

$ tar tzvf foobar.tgz | awk '{sum += $3}{print $6} END {print "Total size: " sum " bytes (" sum/1024^2 " Mb)."}'

Conditional actions

Before an action it is possible to specify a boolean (i.e. TRUE or FALSE) expression. If the condition evaluates to TRUE the specified action is executed.

List all files bigger than 4000 bytes:

$ ls -l | awk '($5 > 4000)'

$ ls -l | awk '($5 > 4000) {print $9 " is bigger than 4000"}'

$ ls -l | awk '($5 > 4000) {sum += $5} END {print "Total of big files: " sum " bytes."}'