Page tree
Skip to end of metadata
Go to start of metadata

Most of the Linux commands given in this wiki space assume basic familiarity with Linux and Bash (Bourne Again SHell). Bash is is typically the default shell on most distros. For ITC personnel who have less experience with Linux, you may trip over some "simple" things.  This page contains a number of tips which may help get your head and fingers used to bash.

For OpenVMS Admins

If you're experienced with OpenVMS, you will hate bash (for a while, or for a long time).   You'll miss things like /LOG and /CONFIRM doing the same thing on every command.  You'll be annoyed that everything is case-sensitive. You'll wonder why your typing 'cat' (or maybe 'dog') to TYPE a file and 'less' to page through a file.   For while, you may try to customize your ~/.bashrc file to have aliases that make it seem like DCL (I know, I tried).

My best advice, learned the hard way, is just let it go.  The world didn't learn the lessons that DCL could have taught it.  However, what bash and the GNU utilities lack in expressiveness and consistency, it more than makes up for in raw power and flexibility. It may take years, but you'll eventually learn to appreciate the bash/GNU combo and get proficient at it.  Fortunately, Google knows the answer to every bash question you're ever going to have. Use it. 

Here are a few links with a mapping from between VMS and Linux:

Basic Commands

CommandDescription
cat [filename]Display file’s contents to standard output
cd /directorypathChange to directory.
chmod mode filenameChange a file’s permissions.
chown filenameChange who owns a file.
cp source destinationCopy files and directories.
dateDisplay or set the system date and time.
dfDisplay used and available disk space.
duShow how much space each file takes up.
file filenameDetermine what type of data is within a file.
find [pathname] [expression]Search for files matching a provided pattern.
grep pattern [filesname]Search files or output for a particular pattern.
kill pidStop a process. If the process refuses to stop, use kill -9 pid.
less [filename]View the contents of a file one page at a time.
ln source [destination]Create a shortcut.
lsList directory contents.
man [command]Display the help information for the specified command.
mkdir directoryCreate a new directory.
mv source destinationRename or move file(s) or directories.
passwd [name [password]]Change the password or allow (for the system administrator) to
change any password.
psDisplay a snapshot of the currently running processes.
pwdDisplay the pathname for the current directory.
rm directoryRemove (delete) file(s) and/or directories.
rmdir directoryDelete empty directories.
ssh user@machineRemotely log in to another Linux machine, over the network.
Leave an ssh session by typing exit.
su [user [arguments]]Switch to another user account.
sudo command
tail [filename]Display the last n lines of a file
tar filenameStore and extract files from a tarfile (.tar) or tarball (.tar.gz or .tgz).
topDisplays the resources being used on your system.
touch filename

Create an empty file with the specified name.

nohup command &execute command in detached job
historyshows the command line history
useradd usernamecreate a user
passwd usernamesets a password and unlocks the user
usermod -a -G groupName userName

add user to a group

  • -a adds the group to whatever existing groups the user has
  • -G takes a comma separated list
visudoModify the /etc/sudoers file.  Members of groups specified in this file can obtain sudo access

Keyboard Commands

Ctrl + AGo to the beginning of the line you are currently typing on
Ctrl + EGo to the end of the line you are currently typing on
Ctrl + UClears the line before the cursor position. If you are at the end of the line, clears the entire line.
Ctrl + HSame as backspace
Ctrl + RLet's you search through previously used commands
Ctrl + CKill whatever you are running
Ctrl + DExit the current shell
Ctrl + ZPuts whatever you are running into a suspended background process. fg restores it.
Ctrl + WDelete the word before the cursor
Ctrl + KClear the line after the cursor
Ctrl + TSwap the last two characters before the cursor
Esc + TSwap the last two words before the cursor
TabAuto-complete files and folder names

SSDT-Utils image

Part of the reason that I created the ssdt-utils image is so that we could distribute utilities to you that would ease your way into linux (and so I didn't have to document so many arcane commands).  See Install and Update SSDT Utils package.

The ssdt-utils container, among other things, creates a utils alias in your process which allow you to executed bunded commands.  When you execute a utils command, a temporary container is created which maps your current working directory to /tmp in the container.   You can then use utilities to operate on your current directory.  For example, to edit a file in your current directory with nano do:

utils nano /tmp/filename

~/.bashrc file

The ~/.bashrc file is like the LOGIN.COM on VMS.  You should already have one.  But it's a "hidden" file (because it starts with a dot).   You can edit the file with nano with:

cd ~
utils nano /tmp/.bashrc

Tuning Tools

Free

The free command displays information about both the physical and swap memory; free also includes buffer information.  This can give a quick view of memory usage on the system.  There are a few options used for the display of memory.  For example, -m displays the memory in megabytes.  The -h option will display it in more "human readable" format.  To show a total line, use the -t option. To display a specific number of times, use the -c option (this requires the -s option). The -s seconds option will continuously display the result seconds apart (this one requires a specific seconds parameter). Note that the -s option does not appear to work in Ubuntu 16.04 (there is a workaround using -c), but does work in earlier versions. This is a reported bug. 

Options can be used together.  Examples:

root@ssdt-docker-dev-09:~# free -m
              total        used        free      shared  buff/cache   available
Mem:           7978        1227        1758          45        4992        6407
Swap:          1019           0        1019

root@ssdt-docker-dev-09:~# free -h
              total        used        free      shared  buff/cache   available
Mem:           7.8G        1.2G        1.7G         45M        4.9G        6.3G
Swap:          1.0G          0B        1.0G

root@ssdt-docker-dev-09:~# free -t
              total        used        free      shared  buff/cache   available
Mem:        8170172     1256704     1800888       46760     5112580     6561876
Swap:       1044476           0     1044476
Total:      9214648     1256704     2845364

root@ssdt-docker-dev-09:~# free -th
              total        used        free      shared  buff/cache   available
Mem:           7.8G        1.2G        1.7G         45M        4.9G        6.3G
Swap:          1.0G          0B        1.0G
Total:         8.8G        1.2G        2.7G

##-s with workound using -c - this will stop after displaying 3 times, 10 seconds apart

root@ssdt-docker-dev-09:~# date
Fri Aug 11 08:00:03 EDT 2017

root@ssdt-docker-dev-09:~# free -hc 3 -s 10
              total        used        free      shared  buff/cache   available
Mem:           7.8G        1.2G        1.7G         45M        4.9G        6.3G
Swap:          1.0G          0B        1.0G

              total        used        free      shared  buff/cache   available
Mem:           7.8G        1.2G        1.7G         45M        4.9G        6.3G
Swap:          1.0G          0B        1.0G

              total        used        free      shared  buff/cache   available
Mem:           7.8G        1.2G        1.7G         45M        4.9G        6.3G
Swap:          1.0G          0B        1.0G

root@ssdt-docker-dev-09:~# date
Fri Aug 11 08:00:28 EDT 2017

Top

The top command in Ubuntu is used to monitor processes.  See this link for more information on the top command.

Some terminal emulators don't provide the proper cursor motion keys.  The following is taken from the link referenced above. With Cygwin, my experience is that the second key combination (for example, alt + k will navigate up) works
              key      equivalent-key-combinations
              Up       alt + \      or  alt + k
              Down     alt + /      or  alt + j
              Left     alt + <      or  alt + h
              Right    alt + >      or  alt + l (lower case L)
              PgUp     alt + Up     or  alt + ctrl + k
              PgDn     alt + Down   or  alt + ctrl + j
              Home     alt + Left   or  alt + ctrl + h
              End      alt + Right  or  alt + ctrl + l



How to add another column to the top command results

In this example, we add Swap to the list.  Original top command


Press "f" to go into Field Management mode

Use alt + j (or down key) to get to SWAP

Select it with d (note you can also deselect using d)


To select it for moving, use alt + l (or right)


Use alt + k to move it up (or the up key)

To commit, press enter (or alt + h or the left key)

Press q to quit the format mode.  Top now looks like this:

If you want top to use this configuration the next time it is used, press W to write (uppercase W).

Apply a filter to top

To apply a filter, enter a lower case letter o to bring up the filter dialog. then filter on COMMAND=java for example.  Note the filter is just for the current session; it does not "stick" for the next time.


Result:

If you have filters already applied and would like to clear the current filters you can simply press the '=' (equals) key from the main screen.

Finding a specific Docker container PID

The top command lists statistics and other per PIDs.  When trying to monitor system issues, it can be beneficial to know specific processes associated with docker containers. Use this command:

docker ps -q | xargs docker inspect --format '{{.State.Pid}}, {{.Name}}'

Example Result

docker ps -q | xargs docker inspect --format '{{.State.Pid}}, {{.Name}}'
5673, /uat1_usasapp_1
5544, /uat1_usasdb_1
4803, /proxy_proxy_1
4567, /proxy_index-gen_1
4706, /proxy_index_1

Major/Minor Page Faults - Monitoring

Without going into a lot of detail it is sufficient to say that major/minor page faults can be a good indicator of how the system is performing and, more specifically, where the system is retrieving data from that are required for processes.  Major page faults occur when a process makes a request to the system for a resource that is not available in shared memory.  Therefore, the disk must be read and the information loaded from storage.  In contrast, the minor page fault is when a process makes a request to the system for a resource that it doesn't yet have a reference to but is already available in shared memory.  Access to this data is much faster because the information is already loaded into memory and simply needs to be made available to the process.  More information on this can be found at the following link:

Understanding page faults and memory swap-in/outs: when should you worry?

One useful command to examine major/minor page faults on a Linux system using the ps command is the following:

ps -eo min_flt,maj_flt,cmd

This command leverages the ps command's support for field selection to display major/minor fault counts for a specific command/process.

Swappiness

Swappiness is how often the swap file is used.  If it is set to zero, there will be no swapping unless the system runs out of memory.  If it's set at 100, programs will be swapped to disk almost immediately.  Swappiess is expressed as a percent of the memory.  When using higher memory systems, it is generally a good idea (depending on the types of programs/processes running on the server) to reduce swappiness and thus reduce swapping to the disk. The default setting for swappiness on Ubuntu is 60.  With a high memory system, you may want to adjust this as low as 10. For more details on swappiness, see here.  Due to the high memory availability, we changed the swappiness to 10.

Check Swappiness:

cat /proc/sys/vm/swappiness

Change Swappiness

Generally, change the setting live, and once that appears to be working as expected, make the permanent change. 

Live:

To change the swappiness on the system and have it take effect immediately

sysctl vm.swappiness=10

Permanent - keeps it on reboot

To ensure the swappiness does not revert to the default value on reboot, edit the /etc/sysctl.conf file and add this line:

vm.swappiness = 10 

Crontab

Crontab is used in Linux distributions to schedule jobs.  It can be scheduled for specific minute(s), hour(s), day(s), and so forth.  More information can be seen here. The Wikipedia article explains it fairly well also. 

Look at existing crontab:

## to look at an existing crontab file
sudo crontab -l

Edit existing crontab:

##to edit/create a crontab file (use sudo to make it for root)
sudo crontab -e

##add any commands

Examples:

In the following example, jobs are scheduled on a daily basis (as the root user)  to update the ssdt utilities at 1:00 a.m., run the /ssdt/scripts/updates-pull.sh to pull any updated images at 1:10 a.m., and apply the updates to the /data/pilot directory tree at 1:30 a.m. Also, it is writing to a logfile for each job. The > syntax overwrites any exiting logfile with the same fully qualified name, and >> appends to any existing logfile with the same fully qualified name (and it will create one if none exists). A date can be added to the log file name.  Note when adding a date, use ` (it is generally on the ~ key) and not '. Note that each crontab is user specific.

##to overwrite exiting logfiles, entries should be:
0 1 * * *   /ssdt/update.sh 2>&1 > /data/pilot/update-utilities.log
10 1 * * *   /ssdt/scripts/updates-pull.sh 2>&1 > /data/pilot/updates-pull.log
30 1 * * *  /ssdt/scripts/updates-apply.sh /data/pilot 2>&1 > /data/pilot/updates-apply.log

##update utilities at 1:00 a.m. and write to /data/pilot/update-utilities-year-month-day-hour:minute.log (adds a date to the logfile name)
##example logfile name with this command:  update-utilities-2017-09-27-01:00.log
0 1 * * *   /ssdt/update.sh 2>&1 > /data/pilot/update-utilities-`date +\%Y-\%m-\%d-\%H:\%M`.log

##to append to exiting logfiles (and create a new one if none exists), entries should be:
0 1 * * *   /ssdt/update.sh 2>&1 >> /data/pilot/update-utilities.log
10 1 * * *   /ssdt/scripts/updates-pull.sh 2>&1 >> /data/pilot/updates-pull.log
30 1 * * *  /ssdt/scripts/updates-apply.sh /data/pilot 2>&1 >> /data/pilot/updates-apply.log

Miscellaneous

Linux is case sensitive, including commands and filename.  Also, most Linux terminals use auto complete.  This can be quite a time savings when you don't remember the exact name of the file.


  • No labels