Main Site Discussion > Posts

Useful Bash commands and scripts

(1/2) > >>

Dakusan:
Original post for Useful Bash commands and scripts can be found at https://www.castledragmire.com/Posts/Useful_Bash_commands_and_scripts.
Originally posted on: 05/13/08
First, to find out more about any bash command, useman COMMAND
Now, a primer on the three most useful bash commands: (IMO)
find:
Find will search through a directory and its subdirectories for objects (files, directories, links, etc) satisfying its parameters.
Parameters are written like a math query, with parenthesis for order of operations (make sure to escape them with a “\”!), -a for boolean “and”, -o for boolean “or”, and ! for “not”.  If neither -a or -o is specified, -a is assumed.
For example, to find all files that contain “conf” but do not contain “.bak” as the extension, OR are greater than 5MB:find -type f \( \( -name "*conf*" ! -name "*.bak" \) -o -size +5120k \)Some useful parameters include:

* -maxdepth & -mindepth: only look through certain levels of subdirectories
* -name: name of the object (-iname for case insensitive)
* -regex: name of object matches regular expression
* -size: size of object
* -type: type of object (block special, character special, directory, named pipe, regular file, symbolic link, socket, etc)
* -user & -group: object is owned by user/group
* -exec: exec a command on found objects
* -print0: output each object separated by a null terminator (great so other programs don’t get confused from white space characters)
* -printf: output specified information on each found object (see man file)
For any number operations, use:+nfor greater than n-nfor less than nnfor exactly than n
For a complete reference, see your find’s man page.
--- End quote ---

xargs:
xargs passes piped arguments to another command as trailing arguments.
For example, to list information on all files in a directory greater than 1MB: (Note this will not work with paths with spaces in them, use “find -print0” and “xargs -0” to fix this)find -size +1024k | xargs ls -lSome useful parameters include:

* -0: piped arguments are separated by null terminators
* -n: max arguments passed to each command
* -i: replaces “{}” with the piped argument(s)
So, for example, if you had 2 mirrored directories, and wanted to sync their modification timestamps:cd /ORIGINAL_DIRECTORY
find -print0 | xargs -0 -i touch -m -r="{}" "/MIRROR_DIRECTORY/{}"
For a complete reference, see your xargs’s man page.
--- End quote ---

grep:
GREP is used to search through data for plain text, regular expression, or other pattern matches.  You can use it to search through both pipes and files.
For example, to get your number of CPUs and their speeds:cat /proc/cpuinfo | grep MHzSome useful parameters include:

* -E: use extended regular expressions
* -P: use perl regular expression
* -l: output files with at least one match (-L for no matches)
* -o: show only the matching part of the line
* -r: recursively search through directories
* -v: invert to only output non-matching lines
* -Z: separates matches with null terminator
So, for example, to list all files under your current directory that contain “foo1”, “foo2”, or “bar”, you would use:grep -rlE "foo(1|2)|bar"
For a complete reference, see your grep’s man page.
--- End quote ---

And now some useful commands and scripts:
List size of subdirectories:
du --max-depth=1The --max-depth parameter specifies how many sub levels to list.
-h can be added for more human readable sizes.

List number of files in each subdirectory*:

#!/bin/bash
export IFS=$'\n' #Forces only newlines to be considered argument separators
for dir in `find -type d -maxdepth 1`
do
   a=`find $dir -type f | wc -l`;
   if [ $a != "0" ]
   then
      echo $dir $a
   fi
done
and to sort those resultsSCRIPTNAME | sort -n -k2
List number of different file extensions in current directory and subdirectories:find -type f | grep -Eo "\.[^\.]+$" | sort | uniq -c | sort -nr
Replace text in file(s):perl -i -pe 's/search1/replace1/g; s/search2/replace2/g' FILENAMESIf you want to make pre-edit backups, include an extension after “-i” like “-i.orig”

Perform operations in directories with too many files to pass as arguments: (in this example, remove all files from a directory 100 at a time instead of using “rm -f *”)find -type f | xargs -n100 rm -f
Force kill all processes containing a string:killall -9 STRING
Transfer MySQL databases between servers: (Works in Windows too)mysqldump -u LOCAL_USER_NAME -p LOCAL_DATABASE | mysql -u REMOTE_USER_NAME -p -D REMOTE_DATABASE -h REMOTE_SERVER_ADDRESS“-p” specifies a password is needed

Some lesser known commands that are useful:
screen: This opens up a virtual console session that can be disconnected and reconnected from without stopping the session. This is great when connecting to console through SSH so you don’t lose your progress if disconnected.
htop: An updated version of top, which is a process information viewer.
iotop: A process I/O (input/output - hard drive access) information viewer.  Requires Python ? 2.5 and I/O accounting support compiled into the Linux kernel.
dig: Domain information retrieval. See “Diagnosing DNS Problems” Post for more information.

More to come later...

*Anything staring with  “#!/bin/bash” is intended to be put into a script.

Dakusan:
Since I’ve never really been a fan of for loops in bash... here’s an alternative to the “List number of files in each subdirectory” script from above.
find -maxdepth 1 -mindepth 1 -type d -print0 | xargs -i -0 sh -c "echo -n {} \" \"; find \"{}\" -type f | wc -l"
The sub-shell script (sh) is required because you can’t run sub-pipes naturally within a xargs command.

You can also modify the maxdepth and mindepth arguments (keep them the same) to show deeper directory sizes, but any files above that depth will of course be ignored.

Dakusan:
To get the exported entries from a single dll in cygwin, create a script with the following code. It takes 1 argument as a parameter.

--- Code: ---objdump -p $1 | grep -Pzo '(?is)^\[Ordinal/Name Pointer\] Table.*?\n\n' | grep -oP '(?<=\d\] ).*$'
--- End code ---
If you saved the script as "get_dll_exports", to run it against multiple DLLs at a time, create another script as follows

--- Code: ---for i in $@; do echo -e "--------\n$i\n--------"; get_dll_exports $i; done
--- End code ---
Or to process multiple dlls, but output all of a single file's results on one line with the filename preceding

--- Code: ---for i in $@; do echo -n "$i: "; get_dll_exports $i | perl -pe 's/\n/ /' -; echo; done
--- End code ---

Dakusan:
Clear the buffer of a terminal in bash

--- Code: ---echo -e '\0033\0143'
--- End code ---

Dakusan:
Dumping a plex database. This includes:
* Episode path
* Episode name
* Episode number
* Hints (string containing season and episode numbers, and some other info)
--- Code: ---AppDirectory='/cygdrive/c/Users/Administrator/AppData/Local/Plex Media Server/';
sqlite3 "$AppDirectory/Plug-in Support/Databases/com.plexapp.plugins.library.db" 'SELECT file, title, MDI."index", hints FROM media_parts AS MP INNER JOIN media_items AS MI ON MI.id=MP.media_item_id INNER JOIN metadata_items AS MDI ON MDI.id=MI.metadata_item_id;'[/code
--- End code ---

Navigation

[0] Message Index

[#] Next page

Reply