cheatsheet

redirect

show stdout but redirect all to file

[!NOTE|label:references:]

  • redirect overview

              || visible in terminal ||   visible in file   || existing
      Syntax  ||  StdOut  |  StdErr  ||  StdOut  |  StdErr  ||   file
    ==========++==========+==========++==========+==========++===========
        >     ||    no    |   yes    ||   yes    |    no    || overwrite
        >>    ||    no    |   yes    ||   yes    |    no    ||  append
              ||          |          ||          |          ||
       2>     ||   yes    |    no    ||    no    |   yes    || overwrite
       2>>    ||   yes    |    no    ||    no    |   yes    ||  append
              ||          |          ||          |          ||
       &>     ||    no    |    no    ||   yes    |   yes    || overwrite
       &>>    ||    no    |    no    ||   yes    |   yes    ||  append
              ||          |          ||          |          ||
     | tee    ||   yes    |   yes    ||   yes    |    no    || overwrite
     | tee -a ||   yes    |   yes    ||   yes    |    no    ||  append
              ||          |          ||          |          ||
     n.e. (*) ||   yes    |   yes    ||    no    |   yes    || overwrite
     n.e. (*) ||   yes    |   yes    ||    no    |   yes    ||  append
              ||          |          ||          |          ||
    |& tee    ||   yes    |   yes    ||   yes    |   yes    || overwrite
    |& tee -a ||   yes    |   yes    ||   yes    |   yes    ||  append

tips:

  • echo to stderr

    $ echo 'abcdefg' >/dev/null
    $ echo 'abcdefg' >/dev/null >&2
    abcdefg
$ bash -c "echo a;bahs;echo b;bhas" >>file 2> >( tee -a file >&2 )
bash: line 1: bahs: command not found
bash: line 1: bhas: command not found

$ cat file
a
bash: line 1: bahs: command not found
b
bash: line 1: bhas: command not found
  • stderr output with filter

    $ bash -c "echo a;bahs;echo b;bhas" >>file 2> >( tee -a file 2>&1 | grep -v bahs >&2 )
    bash: line 1: bhas: command not found
    # or
    $ rm -rf file; bash -c "echo a;bahs;echo b;bhas" >>file 2> >( tee -a file | grep -v bahs >&2 )
    bash: line 1: bhas: command not found
    
    $ cat file
    a
    bash: line 1: bahs: command not found
    b
    bash: line 1: bhas: command not found
  • or

    $ bash -c "set -e; echo a;bahs;echo b;bhas" >>cmd.out 2> >( tee -a cmd.out >&2 )
    bash: line 1: bahs: command not found
    
    $ cat cmd.out
    a
    bash: line 1: bahs: command not found

time & date

[!TIP|label:see also:]

# with quotes
$ TZ=':Asia/Shanghai' date

# or without quotes
$ TZ=America/Los_Angeles date

show cal

$ cal -y

# or
$ cal -y |
      tr '\n' '|' |
      sed "s/^/ /;s/$/ /;s/ $(date +%e) / $(date +%e | sed 's/./#/g') /$(date +%m | sed s/^0//)" |
      tr '|' '\n'
                             2014
      January               February               March
Su Mo Tu We Th Fr Sa  Su Mo Tu We Th Fr Sa  Su Mo Tu We Th Fr Sa
          1  2  3  4                     1                     1
 5  6  7  8  9 10 11   2  3  4  5  6  7  8   2  3  4  5  6  7  8
12 13 14 15 16 17 18   9 10 11 12 13 14 15   9 10 11 12 13 14 15
19 20 21 22 23 24 25  16 17 18 19 20 21 22  16 17 ## 19 20 21 22
26 27 28 29 30 31     23 24 25 26 27 28     23 24 25 26 27 28 29
                                            30 31

       April                  May                   June
Su Mo Tu We Th Fr Sa  Su Mo Tu We Th Fr Sa  Su Mo Tu We Th Fr Sa
       1  2  3  4  5               1  2  3   1  2  3  4  5  6  7
 6  7  8  9 10 11 12   4  5  6  7  8  9 10   8  9 10 11 12 13 14
13 14 15 16 17 18 19  11 12 13 14 15 16 17  15 16 17 18 19 20 21
20 21 22 23 24 25 26  18 19 20 21 22 23 24  22 23 24 25 26 27 28
27 28 29 30           25 26 27 28 29 30 31  29 30


        July                 August              September
Su Mo Tu We Th Fr Sa  Su Mo Tu We Th Fr Sa  Su Mo Tu We Th Fr Sa
       1  2  3  4  5                  1  2      1  2  3  4  5  6
 6  7  8  9 10 11 12   3  4  5  6  7  8  9   7  8  9 10 11 12 13
13 14 15 16 17 18 19  10 11 12 13 14 15 16  14 15 16 17 18 19 20
20 21 22 23 24 25 26  17 18 19 20 21 22 23  21 22 23 24 25 26 27
27 28 29 30 31        24 25 26 27 28 29 30  28 29 30
                      31

      October               November              December
Su Mo Tu We Th Fr Sa  Su Mo Tu We Th Fr Sa  Su Mo Tu We Th Fr Sa
          1  2  3  4                     1      1  2  3  4  5  6
 5  6  7  8  9 10 11   2  3  4  5  6  7  8   7  8  9 10 11 12 13
12 13 14 15 16 17 18   9 10 11 12 13 14 15  14 15 16 17 18 19 20
19 20 21 22 23 24 25  16 17 18 19 20 21 22  21 22 23 24 25 26 27
26 27 28 29 30 31     23 24 25 26 27 28 29  28 29 30 31
                      30

synchronize date and time with over ssh

[!NOTE|label:inspired from:]

$ date --set="$(ssh [username]@[sshserver] date)"
  • verify

    $ date;
      ssh username@sshserver date;
      curl -fsSL "http://worldtimeapi.org/api/timezone/America/Los_Angeles" | jq -r .datetime
    Tue Apr  2 20:03:32 PDT 2024
    Tue Apr  2 20:03:33 PDT 2024
    2024-04-02T20:03:33.405227-07:00

download and extract

[!NOTE|label:references:]

  • *.gz

    $ wget -O - http://example.com/a.gz | tar xz
  • *.zip

    $ curl -fsSL https://services.gradle.org/distributions/gradle-4.7-all.zip | bsdtar xzf - -C <EXTRACT_PATH>
    
    # with password
    $ curl -fsSL \
           -u<user>:<passwd> \
           https://path/to/file.zip |
      bsdtar -xzf- --passphrase <PASSWD_OF_ZIP> - -C <EXTRACT_PATH>
  • *.tar.gz

    $ curl -fsSL https://path/to/file.tar.gz | tar xzf - -C <EXTRACT_PATH>
    
    # example
    $ curl -fsSL -j -k \
           -H "Cookie: oraclelicense=accept-securebackup-cookie" \
           http://download.oracle.com/otn-pub/java/jdk/8u181-b13/96a7b8442fe848ef90c96a2fad6ed6d1/jdk-8u181-linux-x64.tar.gz |
      tar xzf - -C '/opt/java'

check file without extract

$ tar -Oxvf myfile.tgz path/to/my.sh | less

extract jar

$ unzip <jar-name>.jar -d <target-folder>
  • Delete files from JAR without unzip

    $ zip -d <jar-name>.jar <path/to/file.txt>

[!TIP] references:

params shortcuts :

SHORT PARAMSLONG PARAMS

-r

--recursive

-m

--mirror

-l

--level

-k

--convert-links

-K

--backup-converted

-P

--directory-prefix

-nv

--no-verbose

-nc

--no-clobber

-nd

--no-directories

-nH

--no-host-directories

-np

--no-parent

-x

--force-directories

-b

--background

-v

--verbose

-p

--page-requisites

$ wget --recursive                       \     # -r
       --user=admin                      \
       --password=admin                  \     # --ask-password
       --auth-no-challenge               \     # optional
       --no-host-directories             \     # -nH
       --no-parent                       \     # -np
       --reject '*.html*'                \
       --directory-prefix=./             \     # -P
       --include-directories=local-dir   \     # -I
  http://example.com/remote-dir

# or
$ wget --recursive                                   \    # -r
       --no-parent                                   \    # -np : will not crawl links in folders above the base of the URL
       --convert-links                               \    # -k  : convert links with the domain name to relative and uncrawled to absolute
       --random-wait --wait 3 --no-http-keep-alive   \    #       do not get banned
       --no-host-directories                         \    # -nH : do not create folders with the domain name
       --execute robots=off --user-agent=Mozilla/5.0 \    #       I AM A HUMAN!!!
       --level=inf --accept '*'                      \    # -l  : do not limit to 5 levels or common file formats
       --reject="index.html*"                        \    #       use this option if you need an exact mirror
       --cut-dirs=0                                  \    #       replace 0 with the number of folders in the path, 0 for the whole domain
  $URL
  • mirror whole website

    $ wget -m https://www.baeldung.com/

download directly

$ wget -r -np -nH --cut-dirs=1 https://www.baeldung.com/linux
# or
$ wget -r --no-parent --no-host-directories --cut-dirs=1 https://www.baeldung.com/linux
  • with level

    $ wget -r -np -l 2 https://www.baeldung.com/linux/
    # or
    $ wget -r --no-parent --level=2 https://www.baeldung.com/linux/
  • with credentials

    • .wgetrc

      $ cat ~/.wgetrc
      user=admin
      password=admin
    • from cmd

      $ wget --user=admin --password=admin
      
      # or
      $ wget --user=admin --ask-password
      Password for user 'admin': admin
    • ignore credentials

      $ wget --no-check-certificate
  • converting links for local viewing

    $ wget -r --no-parent --convert-links https://www.baeldung.com/linux/category/web
  • switching off robot exclusion

    $ wget -r --level=1 --no-parent --convert-links -e robots=off -U="Mozilla"

compress

zip package with dot-file

  • .[^.]*

    $ zip name.zip * .[^.]*'
  • shopt -s dotglob

    [!NOTE|label:references:]

    $ shopt -s dotglob
    $ zip name.zip *
  • .

    $ zip -r name.zip .

remove dot-file without skipping '..' '.' issue

[!NOTE|label:references:]

show 256 colors

[!TIP] see also:

$ for i in {0..255}; do echo -e "\e[38;05;${i}m${i}"; done | column -c 80 -s ' '; echo -e "\e[m"
# or
$ yes "$(seq 1 255)" | while read i; do printf "\x1b[48;5;${i}m\n"; sleep .01; done

# or: https://www.commandlinefu.com/commands/view/5879/show-numerical-values-for-each-of-the-256-colors-in-bash
$ for code in {0..255}; do echo -e "\e[38;05;${code}m $code: Test"; done)

# or: https://www.commandlinefu.com/commands/view/6138/show-numerical-values-for-each-of-the-256-colors-in-bash
$ for i in {0..255}; do echo -e "\e[38;05;${i}m${i}"; done | column -c 80 -s ' '; echo -e "\e[m"

# or: https://www.commandlinefu.com/commands/view/11759/show-numerical-values-for-each-of-the-256-colors-in-bash-for-bold-and-normal-fonts
$ for code in $(seq -w 0 255); do for attr in 0 1; do printf "%s-%03s %bTest%b\n" "${attr}" "${code}" "\e[${attr};38;05;${code}m" "\e[m"; done; done | column -c $((COLUMNS*2))
# better vesion
$ for code in $(seq -w 0 255); do for attr in 0 1; do printf "%b%s-%03s%b\n"  "\e[${attr};38;05;${code}m" "${attr}" "${code}" "\e[m"; done; done | column -c $((COLUMNS*3))

# .. zsh ..
# or: https://www.commandlinefu.com/commands/view/5876/show-numerical-values-for-each-of-the-256-colors-in-zsh
$ for code in {000..255}; do print -P -- "$code: %F{$code}Test%f"; done

# or: https://www.commandlinefu.com/commands/view/12471/show-numerical-values-for-each-of-the-256-colors-in-zsh
for i in {0..255}; do echo -e "\e[38;05;${i}m\\\e[38;05;${i}m"; done | column -c 80 -s ' '; echo -e "\e[m"

commands

ls

[!TIP|label:references]

PWD's secrets

$ l | grep bc
lrwxrwxrwx 1 marslo marslo   37 Mar  4 00:25 bc -> /home/marslo/Tools/Git/BrowserConfig//
$ cd bc/
$ pwd -L
/home/marslo/bc
$ pwd -P
/home/marslo/Tools/Git/BrowserConfig

list the command startsWith

$ compgen -c "system-config-"
system-config-authentication
system-config-authentication
system-config-date
system-config-firewall
system-config-firewall-tui
system-config-kdump
system-config-keyboard
system-config-keyboard
system-config-network
system-config-network
system-config-network-cmd
system-config-network-cmd
system-config-network-tui
system-config-printer
system-config-printer-applet
system-config-services
system-config-services
system-config-users

fuzzy find for commands

$ apropos editor | head
Git::SVN::Editor (3pm) - commit driver for "git svn set-tree" and dcommit
INIFILE (1)          - OpenLink Virtuoso Opensource ini File Editor
atobm (1)            - bitmap editor and converter utilities for the X Window System
bitmap (1)           - bitmap editor and converter utilities for the X Window System
bmtoa (1)            - bitmap editor and converter utilities for the X Window System
ed (1)               - line-oriented text editor
editor (1)           - Nano's ANOther editor, an enhanced free Pico clone
editres (1)          - a dynamic resource editor for X Toolkit applications
ex (1)               - Vi IMproved, a programmers text editor
gedit (1)            - text editor for the GNOME Desktop

batch commands

batch rename

$ l
total 4.0K
-rw-r--r-- 1 marslo marslo 10 Feb 21 00:43 a.b
$ rename -v 's/\./_/g' *
a.b renamed as a_b
$ l
total 4.0K
-rw-r--r-- 1 marslo marslo 10 Feb 21 00:43 a_b
  • delete string with find

    $ for file in sw.ras.*; do mv "$file" "${file/ras./}"; done
  • /usr/local/bin/rename in OSX

    $ /usr/local/bin/rename -v 's/_xyz.com//g' *.txt
    'a_xyz.com.txt' renamed to 'a.txt'
    'b_xyz.com.txt' renamed to 'b.txt'
    'c_xyz.com.txt' renamed to 'c.txt'
    'd_xyz.com.txt' renamed to 'd.txt'
    'e_xyz.com.txt' renamed to 'e.txt'
    'f_xyz.com.txt' renamed to 'f.txt'
    'g_xyz.com.txt' renamed to 'g.txt'
    'h_xyz.com.txt' renamed to 'h.txt'
  • rename from util-linux

    [!NOTE|label:references:]

    $ /usr/local/opt/util-linux/bin/rename -v '_xyz.com' '' *.ttf
    `a_xyz.com.ttf' -> `a.ttf'
    `b_xyz.com.ttf' -> `b.ttf'
    `c_xyz.com.ttf' -> `c.ttf'
    `d_xyz.com.ttf' -> `d.ttf'
    `e_xyz.com.ttf' -> `e.ttf'
    `f_xyz.com.ttf' -> `f.ttf'
    `g_xyz.com.ttf' -> `g.ttf'
    `h_xyz.com.ttf' -> `h.ttf'

xargs rename

$ shopt -s extglob
$ ls git-+([0-9])*.png
git-0.png  git-1.png  git-2.png  git-3.png  git-4.png  git-5.png

$ ls --color=none git-+([0-9])*.png | xargs rename -v 's/git-/git-for-windows-/'
'git-0.png' renamed to 'git-for-windows-0.png'
'git-1.png' renamed to 'git-for-windows-1.png'
'git-2.png' renamed to 'git-for-windows-2.png'
'git-3.png' renamed to 'git-for-windows-3.png'
'git-4.png' renamed to 'git-for-windows-4.png'
'git-5.png' renamed to 'git-for-windows-5.png'

batch move

[!NOTE] -I replace-str

$ mkdir backup-folder && ls | grep -Ze ".*rar" | xargs -d '\n' -I {} mv {} backup-folder

batch copy

reference:

$ ls -1 a/b/* 11 12 | xargs cp -t copy-target-folder/

copy single file to multipule folders

$ echo dir1 dir2 dir3 | xargs -n 1 cp file1

# or
$ echo dir{1..10} | xargs -n 1 cp file1

ldapsearch

[!NOTE] enhaanced script

-LLL                                          # just a particular way to display the results
-H ldap://wspace.mydomain.com                 # the URL where the LDAP server listens
-x                                            # use simple authentication, not SASL
-D 'user1'                                    # the account to use to authenticate to LDAP
-w 'user1password'                            # the password that goes with the account on the previous line
-E pr=1000/noprompt                           # ask the server for all pages, don't stop after one
-b 'ou=mydomain,dc=wspace,dc=mydomain,dc=com' # the base of the search. We don't want results from e.g. 'ou=blah,dc=wspace,dc=mydomain,dc=com'
'(&(objectClass=person)(uidNumber=*))'        # Ask for any entry that has attributes objectClass=person and uidNumber has a value
SAMAccountName uid uidNumber                  # Show only these attributes

search specific user

[!NOTE] info :

  • ldap url : ldaps://ldap.mydomain.com:636

  • base search base : dc=mydomain,dc=com

  • login user : user1 / user1password

  • search : user2

remove #refldaps://..

  • remove #.*

    $ ldapsearch ... | sed -r '/^(#.*)$/d'
  • remove empty lines

    $ ldapsearch ... | sed -r '/^\s*$/d'
  • remove all

    $ ldapsearch ... | sed -r '/^(#.*)$/d;/^\s*$/d'
    
    # or
    $ ldapsearch ... | sed -r '/(^#.*)|(^\s*)$/d'
$ ldapsearch \
    -LLL \
    -x \
    -H 'ldaps://ldap.mydomain.com:636' \
    -b 'dc=mydomain,dc=com' \
    -D 'user1' \
    -w 'user1password' \
    CN='user2'
  • or insert password via interactive mode ( -W )

    $ ldapsearch \
        -LLL \
        -x \
        -H 'ldaps://ldap.mydomain.com:636' \
        -b 'dc=mydomain,dc=com' \
        -W \
        -D 'user1' \
        CN='user2'

filter DN field only

$ ldapsearch \
    [-LLL \]
    -H 'ldaps://ldap.mydomain.com:636' \
    -b 'dc=mydomain,dc=com' \
    -x \
    -D 'user1' \
    -w 'user1password' \
    CN='user2' \
    DN

filter SAMAccountName, uid and uidNumber only

[!TIP] filter base on base DN (OU=Person,DC=mydomain,DC=com)

$ ldapsearch \
    -LLL \
    -x \
    -H 'ldaps://ldap.mydomain.com:636' \
    -b 'ou=Workers,dc=mydomain,dc=com' \
    -D 'user1' \
    -w 'user1password' \
    -E 'pr=1000/noprompt' \
    '(&(objectClass=user)(sAMAccountName=*))' \
    SAMAccountName uid uidNumber DN

filter particular group

$ ldapsearch \
    -x \
    -H 'ldaps://ldap.mydomain.com:636' \
    -b 'OU=DL,OU=Groups,OU=GLOBAL,OU=Sites,dc=mydomain,dc=com' \
    -D 'user1' \
    -w 'user1password' \
    -E 'pr=1000/noprompt' \
    '(&(objectClass=group)(CN=*))'
  • search particular group (cn=DL-name-group)

    $ ldapsearch \
        -x \
        -H 'ldaps://ldap.mydomain.com:636' \
        -b 'OU=DL,OU=Groups,OU=GLOBAL,OU=Sites,dc=mydomain,dc=com' \
        -D 'user1' \
        -w 'user1password' \
        -E 'pr=1000/noprompt' \
        CN='DL-name-group'

get userCertificates

[!NOTE|label:references:]

  • 9.2. Certificate Publishing

    # convert a pem certificate into der
    openssl x509 -outform DER -in incert.pem  -out outcert.der
    
    # created LDIF file
    ldif -b "usercertificate;binary" < outcert.der  > cert.ldif
    
    # creates an usercertificate attribute encoded in base64
    ldapmodify -x -W -D "cn=Manager,dc=yourorg,dc=com" -f cert.ldif
  • get cert info

    $ ldapsearch marslo userCertificate |
                 awk '{print $NF}' |
                 xargs -i bash -c "echo {} | base64 -d -w0 | openssl x509 -noout -dates -subject -issuer"
    
    # or
    $ while read -r crt; do
        echo "${crt}" | base64 -d -w0 | openssl x509 -noout -dates -subject -issuer;
      done < <(ldapsearch marslo userCertificate | awk '{print $NF}')
    notBefore=Apr 28 18:05:13 2023 GMT
    notAfter=Apr 27 18:05:13 2025 GMT
    subject=DC = com, DC = example, OU = Workers, CN = marslo, emailAddress = marslo@example.com
    issuer=DC = com, DC = example, CN = example SC Issuing CA V1
  • save local

    • der

      $ while read -r n c; do
          echo "-- ${n} --";
          echo "${c}" | base64 -d -w0 > cert_${n}.der;
        done < <(ldapsearch marslo userCertificate | awk '{print $NF}' | cat -n)
    • crt

      $ while read -r n c; do
          echo "-- ${n} --";
          echo "${c}" | base64 -d -w0 > cert_${n}.der;
          openssl x509 -in cert_${n}.der -inform DER -out cert_${n}.crt;
        done < <(ldapsearch marslo userCertificate | awk '{print $NF}' | cat -n)

others

directory diff

$ diff --suppress-common-lines -y <(cd path_to_dir1; find .|sort) <(cd path_to_dir2; find .|sort)

show some command periodically

$ watch --interval 1 ls -alt
  • watch with pipe

    $ watch -n 1 'ls -Altrh | grep <keywords>'

clear

$ printf "\ec"

use less as tail -f

$ less +F <filename>

netcat & nmap-ncat

[!NOTE] references:

  • install

    $ sudo yum -y install epel-release [yum-utils]
    # or via url
    $ sudo dnf [re]install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
    $ sudo dnf [re]install https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
    
    $ sudo yum update -y
    $ yum list | grep netcat
    netcat.x86_64      1.219-2.el8       @epel
    $ sudo yum install -y netcat.x86_64
    
    # check
    $ sudo yum repolist
  • switch with nmap & netcat

    $ ls -altrh /usr/local/bin/nc
    lrwxrwxrwx 1 root root 22 Mar 14 03:14 /usr/local/bin/nc -> /etc/alternatives/nmap
    
    $ sudo update-alternatives --config nmap
    
    There are 2 programs which provide 'nmap'.
    
      Selection    Command
    -----------------------------------------------
    *+ 1           /usr/bin/ncat
       2           /usr/bin/netcat
    
    Enter to keep the current selection[+], or type selection number: 2
  • check

    # by using netcat
    $ nc -zv google.com 443
    Connection to google.com (142.251.214.142) 443 port [tcp/https] succeeded!
    
    # by using nact
    $ ncat -zv google.com 443
    Ncat: Version 7.70 ( https://nmap.org/ncat )
    Ncat: Connected to 142.251.214.142:443.
    Ncat: 0 bytes sent, 0 bytes received in 0.07 seconds.
  • check package

    $ rpm -ql netcat.x86_64
    /usr/bin/nc
    /usr/bin/netcat
    /usr/lib/.build-id
    /usr/lib/.build-id/f3
    /usr/lib/.build-id/f3/3de6290429f99a8d8f5fe646a93bcc952dafdd
    /usr/share/man/man1/nc.1.gz
    /usr/share/man/man1/netcat.1.gz
    
    $ rpm -ql nmap-ncat.x86_64
    /usr/bin/nc
    /usr/bin/ncat
    ...

alternatives & update-alternatives

  • install

    $ sudo update-alternatives --install /usr/bin/java java /opt/java/jdk1.8.0_121/bin/java 999
    $ sudo update-alternatives --auto java
    $ sudo update-alternatives --install /usr/bin/javac javac /opt/java/jdk1.8.0_121/bin/javac 999
    $ sudo update-alternatives --auto javac
  • modify

    $ ls -altrh $(which -a nc)
    lrwxrwxrwx 1 root root 22 Jun  1 04:02 /usr/bin/nc -> /etc/alternatives/nmap
    
    $ sudo alternatives --config nmap
    There are 2 programs which provide 'nmap'.
    
      Selection    Command
    -----------------------------------------------
    *+ 1           /usr/bin/netcat
       2           /usr/bin/ncat
    
    Enter to keep the current selection[+], or type selection number: 1

Last updated