First, to find out more about any bash command, use
man COMMAND
Now, a primer on the three most useful bash commands: (
IMO)
find:
Find will search through a directory and its subdirectories for objects (files, directories, links, etc) satisfying its parameters.
Parameters are written like a math query, with parenthesis for order of operations (make sure to escape them with a “\”!), -a for boolean “and”, -o for boolean “or”, and ! for “not”. If neither -a or -o is specified, -a is assumed.
For example, to find all files that contain “conf” but do not contain “.bak” as the extension, OR are greater than 5MB:find -type f \( \( -name "*conf*" ! -name "*.bak" \) -o -size +5120k \)
Some useful parameters include:
- -maxdepth & -mindepth: only look through certain levels of subdirectories
- -name: name of the object (-iname for case insensitive)
- -regex: name of object matches regular expression
- -size: size of object
- -type: type of object (block special, character special, directory, named pipe, regular file, symbolic link, socket, etc)
- -user & -group: object is owned by user/group
- -exec: exec a command on found objects
- -print0: output each object separated by a null terminator (great so other programs don’t get confused from white space characters)
- -printf: output specified information on each found object (see man file)
For any number operations, use:| +n | | for greater than n |
| -n | for less than n |
| n | for exactly than n |
For a complete reference, see your find’s man page.
xargs:
xargs passes piped arguments to another command as trailing arguments.
For example, to list information on all files in a directory greater than 1MB: (Note this will not work with paths with spaces in them, use “find -print0” and “xargs -0” to fix this)find -size +1024k | xargs ls -l
Some useful parameters include:
- -0: piped arguments are separated by null terminators
- -n: max arguments passed to each command
- -i: replaces “{}” with the piped argument(s)
So, for example, if you had 2 mirrored directories, and wanted to sync their modification timestamps:cd /ORIGINAL_DIRECTORY
find -print0 | xargs -0 -i touch -m -r="{}" "/MIRROR_DIRECTORY/{}"
For a complete reference, see your xargs’s man page.
grep:
GREP is used to search through data for plain text, regular expression, or other pattern matches. You can use it to search through both pipes and files.
For example, to get your number of CPUs and their speeds:cat /proc/cpuinfo | grep MHz
Some useful parameters include:
- -E: use extended regular expressions
- -P: use perl regular expression
- -l: output files with at least one match (-L for no matches)
- -o: show only the matching part of the line
- -r: recursively search through directories
- -v: invert to only output non-matching lines
- -Z: separates matches with null terminator
So, for example, to list all files under your current directory that contain “foo1”, “foo2”, or “bar”, you would use:grep -rlE "foo(1|2)|bar"
For a complete reference, see your grep’s man page.
And now some useful commands and scripts:List size of subdirectories:
du --max-depth=1
The --max-depth parameter specifies how many sub levels to list.
-h can be added for more human readable sizes.
List number of files in each subdirectory*:
#!/bin/bash
export IFS=$'\n' #Forces only newlines to be considered argument separators
for dir in `find -type d -maxdepth 1`
do
a=`find $dir -type f | wc -l`;
if [ $a != "0" ]
then
echo $dir $a
fi
done
and to sort those results
SCRIPTNAME | sort -n -k2
List number of different file extensions in current directory and subdirectories:
find -type f | grep -Eo "\.[^\.]+$" | sort | uniq -c | sort -nr
Replace text in file(s):
perl -i -pe 's/search1/replace1/g; s/search2/replace2/g' FILENAMES
If you want to make pre-edit backups, include an extension after “-i” like “-i.orig”
Perform operations in directories with too many files to pass as arguments: (in this example, remove all files from a directory 100 at a time instead of using “rm -f *”)
find -type f | xargs -n100 rm -f
Force kill all processes containing a string:
killall -9 STRING
Transfer MySQL databases between servers:
(Works in Windows too)mysqldump -u LOCAL_USER_NAME -p LOCAL_DATABASE | mysql -u REMOTE_USER_NAME -p -D REMOTE_DATABASE -h REMOTE_SERVER_ADDRESS
“-p” specifies a password is needed
Some lesser known commands that are useful:
screen: This opens up a virtual console session that can be disconnected and reconnected from without stopping the session. This is great when connecting to console through SSH so you don’t lose your progress if disconnected.
htop: An updated version of top, which is a process information viewer.
iotop: A process I/O (input/output - hard drive access) information viewer. Requires Python ? 2.5 and I/O accounting support compiled into the Linux kernel.
dig: Domain information retrieval. See
“Diagnosing DNS Problems” Post for more information.
More to come later...
*
Anything staring with “#!/bin/bash” is intended to be put into a script.
