ssh

reference:
* How SSH Really Works
SSH port forwarding - Example, command, server config
透过代理连接SSH
SSH ProxyCommand example: Going through one host to reach another server
OpenSSH/Cookbook/Proxies and Jump Hosts
Tunneling SSH over PageKite
Transparent Multi-hop SSH
SSH via HTTP proxy in OSX
ssh key
6 ssh authentication methods to secure connection (sshd_config)
8 ways to prevent brute force SSH attacks in Linux (CentOS/RHEL 7)
* SSH 是如何工作的 | How does SSH work

ssh key

[!TIP|label:resolve ssh permission issue:]
$ sudo su devops -c "restorecon -Rf ~/.ssh"
# or
$ /usr/sbin/restorecon .ssh/ .ssh/authorized_keys

generate ssh key

$ keyname='marslo@china'

# rsa4096
$ ssh-keygen -t rsa -b 4096 -f ~/.ssh/${keyname} -C "${keyname}" -P "" -q

# ed25519
$ ssh-keygen -t ed25519 -C "${keyname}" -f ~/.ssh/${keyname} -P '' -q
$ ssh-keygen -t ed25519 -o -a 100 -C "${keyname}" -f ~/.ssh/${keyname} -P '' -q

get servers public key

$ ssh-keyscan -H www.sample.com

# or
$ ssh-keyscan -p 29418 -t rsa www.sample.com
$ ssh-keyscan -p 29418 -t rsa www.sample.com >> ~/.ssh/known_hosts

upload the local ~/.ssh/know_hosts

[!NOTE|label:references:]
We updated our RSA SSH host key

$ ssh-keygen -R github.com
# Host github.com found: line 63
/home/marslo/.ssh/known_hosts updated.
Original contents retained as /home/marslo/.ssh/known_hosts.old

# or
$ curl -sL https://api.github.com/meta | jq -r '.ssh_keys | .[]'  | sed -e 's/^/github.com /' >> ~/.ssh/know_hosts

# more details
$ curl -sL https://api.github.com/meta | jq -r '.ssh_keys | .[]'
ssh-ed25519 AAAA***9GKJl
ecdsa-sha2-nistp256 AAAA***ockg=
ssh-rsa AAAA***wsjk=
$ curl -sL https://api.github.com/meta | jq -r '.ssh_keys | .[]'  | sed -e 's/^/github.com /'
github.com ssh-ed25519 AAAA***9GKJl
github.com ecdsa-sha2-nistp256 AAAA***ockg=
github.com ssh-rsa AAAA***wsjk=

add ssh key into agent

$ ssh-add ~/.ssh/${keyname}

# e.g.:
$ ssh-add ~/.ssh/id_ed25519
Identity added: /Users/marslo/.ssh/id_ed25519 (marslo@devops)

$ ssh-agent -s
SSH_AUTH_SOCK=/var/folders/s3/mg_f3cv54nn7y758j_t46zt40000gn/T//ssh-MgCrKA3ZS06N/agent.25376; export SSH_AUTH_SOCK;
SSH_AGENT_PID=25377; export SSH_AGENT_PID;
echo Agent pid 25377;

get public key from private key

$ ssh-keygen -y -f ~/.ssh/id_rsa > ~/.ssh/id_rsa.pub

verify public and private key pair

$ diff [-qs] <( ssh-keygen -y -e -f /path/to/private ) \
             <( ssh-keygen -y -e -f /path/to/public.pub )

generate new passphrase

$ ssh-keygen -p -f /path/to/private

get fingerprinter

sha256:

$ ssh-keygen -l -f ~/.ssh/id_rsa

# or
$ ssh-keygen -l -f ~/.ssh/id_rsa.pub

md5:

$ ssh-keygen -E md5 -l -f ~/.ssh/id_rsa

# or
$ ssh-keygen -E md5 -l -f ~/.ssh/id_rsa.pub

with openssl

$ openssl pkey -in ~/.ssh/ec2/primary.pem -pubout -outform DER | openssl md5 -c

keys performance

$ openssl speed rsa1024 rsa2048 dsa1024 dsa2048
Doing 1024 bit private rsa's for 10s: 91211 1024 bit private RSA's in 9.97s
Doing 1024 bit public rsa's for 10s: 1161461 1024 bit public RSA's in 9.93s
Doing 2048 bit private rsa's for 10s: 12305 2048 bit private RSA's in 9.94s
Doing 2048 bit public rsa's for 10s: 403455 2048 bit public RSA's in 9.98s
Doing 1024 bit sign dsa's for 10s: 84873 1024 bit DSA signs in 10.00s
Doing 1024 bit verify dsa's for 10s: 109544 1024 bit DSA verify in 9.99s
Doing 2048 bit sign dsa's for 10s: 30010 2048 bit DSA signs in 9.99s
Doing 2048 bit verify dsa's for 10s: 33202 2048 bit DSA verify in 9.98s
OpenSSL 1.0.2t  10 Sep 2019
built on: reproducible build, date unspecified
options:bn(64,64) rc4(ptr,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: clang -I. -I.. -I../include  -fPIC -fno-common -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -arch x86_64 -O3 -DL_ENDIAN -Wall -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
                  sign    verify    sign/s verify/s
rsa 1024 bits 0.000109s 0.000009s   9148.5 116964.9
rsa 2048 bits 0.000808s 0.000025s   1237.9  40426.4
                  sign    verify    sign/s verify/s
dsa 1024 bits 0.000118s 0.000091s   8487.3  10965.4
dsa 2048 bits 0.000333s 0.000301s   3004.0   3326.9

ssh

force use password

$ ssh -o PreferredAuthentications=password -o PubkeyAuthentication=no user@target.server

allows ssh-rsa

[!TIP|label:references:]
openssh v8.8

$ ssh -o HostkeyAlgorithms=+ssh-rsa -p <port> -l <user> <host>

for git commands

[!NOTE|label:references:]
since git v2.33.1

$ GIT_SSH_COMMAND="ssh -o HostkeyAlgorithms=+ssh-rsa" git clone
# since v2.41.0
$ GIT_SSH_COMMAND="ssh -o HostkeyAlgorithms=+ssh-rsa -o PubkeyAcceptedAlgorithms=+ssh-rsa" git clone

# or
$ git config [--global] core.sshCommand 'ssh -o HostKeyAlgorithms=+ssh-rsa'
# since v2.41.0
$ git config [--global] core.sshCommand 'ssh -o HostKeyAlgorithms=+ssh-rsa -o PubkeyAcceptedAlgorithms=+ssh-rsa'

force bypass `~/.ssh/config`

[!NOTE|label:references:]

How to tell ssh client to ignore ssh config file

$ man ssh

     -F configfile
             Specifies an alternative per-user configuration file.  If a configuration file is given on
             the command line, the system-wide configuration file (/etc/ssh/ssh_config) will be ignored.
             The default for the per-user configuration file is ~/.ssh/config.  If set to “none”, no
             configuration files will be read.

$ ssh -F none user@sample.com

# or
$ ssh -vvT -F /dev/null -i ~/.ssh/id_ed25519 user@sample.com

ssh and tar

[!TIP|label:references:]
* iMarslo: bash sugar - find and tar for JENKINS_HOME

copy multiple files to remote server

$ tar cvzf - -T list_of_filenames | ssh hostname tar xzf -

# or
$ tar cvf - /path/*.jpg | ssh foo@bar.com "tar xvf -"

tar all and extra in remote

# ssh -C
#    no `-z`      `-C`
#       |           |
#       v           v
$ tar cf - . | ssh -C hostname "cd ~/.marslo/test/; tar xvf -"
Warning: Permanently added '10.69.78.40' (ECDSA) to the list of known hosts.
./
./temp/
./a/
./a/a.txt
./c/
./b/

# tar z-flag
#     `-z`       no `-C`
#       |          |
#       v          v
$ tar cfz - . | ssh hostname "cd ~/.marslo/test/; tar xvzf -"

copy local file content into remote

[!NOTE|label:references:]
How to copy a file without using scp inside an ssh session?

$ ssh user@remote 'cat > /path/to/remote/file' < /path/to/local/file

# or
$ cat /path/to/local/file | ssh user@remote 'cat > /path/to/remote/file'

# to retrieve the file
$ ssh user@remote 'cat /path/to/remote/file' > pbcopy

with proxy

using command directly

Linux:

$ ssh -o 'ProxyCommand nc -x proxy.url.com proxy-port %h %p' -vT user@target.server

# or
$ ssh -o 'ProxyCommand corkscrew proxy.url.com proxy-port %h %p' -vT user@target.server

windows:

git for windows

$ ssh -o 'ProxyCommand connect -H http://proxy.url.com:proxy-port %h %p' user@target.server

cygwin

$ apt-cyg install corkscrew
$ ssh -o 'ProxyCommand corkscrew proxy.url.com proxy-port %h %p' user@target.server

# or
$ apt-cyg install nc
$ ssh -o 'ProxyCommand nc -X connect -x proxy.url.com:proxy-port %h %p' -vT git@github.com

start app remotely

$ ssh -f user@remote.ip DISPLAY=:0.0 smplayer movie.avi

launch firefox

$ ssh -fY user@REMOTESERVER firefox -no-remote

create a persistent connection to a machine

$ ssh -MNf <user>@<host>

ssh certificate

[!NOTE|label:references:]
* OpenSSH/Cookbook/Certificate-based Authentication
* Configure ssh certificate based authentication
14.3.3. Creating SSH CA Certificate Signing Keys
Files Associated with SSH Host Certificates
Certificate Authority (ca) - a private key generated for signing other keys
Certificate public key - the public component of the certificate authority
Host Public Key - the actual key that the SSH daemon uses to identify itself to the clients
Host Certificate - the signature made for the Host Public Key using the Certificate Authority
How to Generate and Configure SSH Certificate-Based Authentication
If you’re not using SSH certificates you’re doing SSH wrong
How to configure and setup SSH certificates for SSH authentication
How To Configure SSH Key-Based Authentication on a Linux Server
Tightening SSH access using short-lived SSH certificates
How to setup SSH certificates for SSH authentication

SSH User Certificates

ca

remote $ ssh-keygen -t ed25519 -b 4096 -f devops@ca -C 'devops@ca' -P '' -q
remote $ sudo cp devops@ca* /etc/ssh/
remote $ echo 'TrustedUserCAKeys /etc/ssh/devops@ca.pub' | sudo tee -a /etc/ssh/sshd_config

remote $ sudo grep TrustedUserCAKeys /etc/ssh/sshd_config
TrustedUserCAKeys /etc/ssh/devops@ca.pub

sign and generate cert key

# download devops@ca
#                                 key id  principals
#                                    v        v
local $ ssh-keygen -s devops@ca -I marslo -n marslo -V +52w ~/.ssh/marslo.pub
$ ls ~/.ssh/marslo*
/Users/marslo/.ssh/marslo  /Users/marslo/.ssh/marslo-cert.pub  /Users/marslo/.ssh/marslo.pub

verify

$ ssh-keygen -Lf ~/.ssh/marslo-cert.pub
/Users/marslo/.ssh/marslo-cert.pub:
        Type: ssh-ed25519-cert-v01@openssh.com user certificate
        Public key: ED25519-CERT SHA256:JfJnCDxjWhLwW3BcBX3XycBr3dT3JtHTwD1M4H3828E
        Signing CA: ED25519 SHA256:5dNlpIIjX/pdoNSpmtfcGQijSrl3W87TByA9KeCe2M0 (using ssh-ed25519)
        Key ID: "marslo"
        Serial: 0
        Valid: from 2023-08-17T17:41:00 to 2024-08-15T17:42:52
        Principals:
                marslo
        Critical Options: (none)
        Extensions:
                permit-X11-forwarding
                permit-agent-forwarding
                permit-port-forwarding
                permit-pty
                permit-user-rc

$ ssh -i ~/.ssh/marslo marslo@remote "cat /home/marslo/.ssh/authorized_keys; du -hs /home/marslo/.ssh/authorized_keys; hostname"
0 /home/marslo/.ssh/authorized_keys
remote

update for existing key

#                                       serial
#                                         v
$ ssh-keygen -s devops@ca -I 'edcba' -z '0002' -n marslo marslo.pub
Signed user key marslo-cert.pub: id "edcba" serial 2 for marslo valid forever

# verify
$ ssh-keygen -f marslo-cert.pub -L
marslo-cert.pub:
        Type: ssh-ed25519-cert-v01@openssh.com user certificate
        Public key: ED25519-CERT SHA256:JfJnCDxjWhLwW3BcBX3XycBr3dT3JtHTwD1M4H3828E
        Signing CA: ED25519 SHA256:5dNlpIIjX/pdoNSpmtfcGQijSrl3W87TByA9KeCe2M0 (using ssh-ed25519)
        Key ID: "edcba"
        Serial: 2
        Valid: forever
        Principals:
                marslo
        Critical Options: (none)
        Extensions:
                permit-X11-forwarding
                permit-agent-forwarding
                permit-port-forwarding
                permit-pty
                permit-user-rc

[!NOTE|label:references:]
OpenSSH/Cookbook/Certificate-based Authentication
$ ssh -o CertificateFile=server01.ed25519-cert.pub -i server01.ed25519 \
        fred@server01.example.org

$ ssh marslo@sample.server.com
marslo@sample.server.com's password:

$ ssh -o CertificateFile=marslo-cert.pub marslo@sample.server.com "du -hs ~/.ssh/authorized_keys"
0 /home/marslo/.ssh/authorized_keys

# via config
$ cat ~/.ssh/confg
Host example sample.server.com
     Hostname sample.server.com
     User marslo
     IdentitiesOnly yes
     IdentityFile /home/marslo/.ssh/marslo
     CertificateFile /home/marslo/.ssh/marslo-cert.pub

SSH Host Certificates

ssh tunnel

references:
ssh端口转发(跳板机)实战详解
SSH 通过跳板机直接访问内网机器

[!TIP|label:references:]
key point:
-L : <--
-R : -->
basic command line
$ ssh -Nf -L <localhost/localip>:<local_port>:<target>:<target_port> <to-be-mapping/localhost>
usage:
1 -> [2 ->] 3 : ssh host2:port2:host3:port3 host1:port1
if ignore host2. default using local.host
references:
SSH connection through host in the middle
$ ssh -t reachable_host ssh unreachable_host

two servers

`-L`

in jumper

[!TIP]
purpose:
local:22 --> jumper:6666

# -L : <--
$ ssh user@jumper.server
$ ssh -Nf -L [jumper.server:]6666:my.server:22 root@my.server

# verify
$ sudo netstat -an | grep 6666
tcp 0 0 jumper.ip:6666 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:6666 0.0.0.0:* LISTEN
tcp 0 0 ::1:6666 :::* LISTEN

`-R`

[!TIP]

status:

               current status
                    |
                    v
       mymac:22 <--------- jumper:22

 purpose:  |                  ^
           + ---------------> +

# login to jumper first via GUI
                         destination
                             |
                             v
<jumper> $ ssh -Nf -R 6666:<mymac>:22 marslo@<mymac>
<jumper> $ ps auxfww | grep ssh | grep -v grep
marslo    2281  0.0  0.0  11716  4064 ?        Ss   19:55   0:00      \_ ssh -Nf -R 6666:<my.pc.ip>:22 marslo@<my.pc.ip>
<jumper> $ sudo netstat -anp | grep ssh
tcp        0      0 <jumper>:46176    <mymac>:22        ESTABLISHED 2281/ssh

# verify in <mymac>
<mymac> $ netstat -an | grep 6666
tcp4       0      0  127.0.0.1.6666         *.*                    LISTEN
tcp6       0      0  ::1.6666               *.*                    LISTEN

<mymac> $ ssh -p 7777 localhost
## or
<mymac> $ cat ~/.ssh/config
Host  jumper
      Hostname              localhost
      Port                  7777
<mymac> $ ssh jumper

three serves

scenario 1

in jumper

> - purpose: > ```bash > local:6666 <--- jumper:6666 <--- remote:6666 > ```

# -R : -->
$ ssh user@jumper
$ ssh -Nf -R [jumper:]6666:local:6666 root@remote

# verify
## in remote
$ ssh root@remote
$ scp -P 6666 root@localhost:/remote/path/file /local/path
## in local
$ scp -P 6666 /local/path/file root@localhost:/remote/path/file

$ ssh user@jumper
<jumper> $ ps awwx | grep ssh | grep 6666
17549 ? Ss 0:00 ssh -Nf -L 6666:remote:22 root@remote
18740 ? Ss 3:50 ssh -Nf -R 6666:local:6666 root@remote

scenario 2

[!TIP]

details :

                    current status
                    --------------
                   |              |

          mymac  <----  jumper  ----> destination

            |                             ^
wants:      v                             |
             -------------->--------------
                   visit directly

# login to jumper
<jumper> $ ssh -Nf -R7777:<destination>:22 marslo@<mymac>
<jumper> $ ps auxfww | grep ssh | grep -v grep
marslo      41  0.0  0.0  11716  5832 ?        Ss   21:51   0:00      \_ ssh -Nf -R7777:<destination>:22 marslo@<my.mac.ip>

# verify in mymac
<mymac> $ ssh -p 7777 localhost       # localhost in <mymac> == <destination>
<mymac> $ netstat -an | grep 7777
tcp6       0      0  ::1.7777               ::1.65371              ESTABLISHED
tcp6       0      0  ::1.65371              ::1.7777               ESTABLISHED
tcp4       0      0  127.0.0.1.7777         *.*                    LISTEN
tcp6       0      0  ::1.7777               *.*                    LISTEN

config

ssh config

$ cat ~/.ssh/config
HOST  *
      LogLevel                   ERROR
      HostkeyAlgorithms          +ssh-rsa
      # PubkeyAcceptedAlgorithms +ssh-rsa              # since git v2.41.0
      GSSAPIAuthentication       no
      StrictHostKeyChecking      no
      UserKnownHostsFile         /dev/null
      IdentityFile               ~/.ssh/id_ed25519
      IdentityFile               ~/.ssh/id_rsa         # keep the older key if necessary

Include config.d/*

Host  github.com
      User                  marslo.jiao@gmail.com
      Hostname              ssh.github.com
      IdentityFile          /C/Marslo/my@key
      IdentitiesOnly        yes
      Port                  443
      # ProxyCommand        nc -X connect -x proxy.com:8080 %h %p
      # ProxyCommand        /usr/bin/nc -X 5 -x 127.0.0.1:1087 %h %p
      # ProxyCommand        /usr/local/bin/corkscrew 127.0.0.1 1087 %h %p

sshd_config

references:
Three locks for your SSH door
保护 SSH 的三把锁
7 Default OpenSSH Security Options You Should Change in /etc/ssh/sshd_config
sshd_config (4)

[!TIP]
disable Root Login : PermitRootLogin
allow only specific users or groups : AllowUsers, AllowGroups
deny specific users or groups : DenyUsers, DenyGroups
change sshd port number : Port
change login grace time : LoginGraceTime
Restrict the Interface (IP Address) to Login : ListenAddress
disconnect ssh when no activity : ClientAliveInterval

banner and motd

[!TIP] files:
/etc/pam.d/sshd : session optional pam_motd.so : /usr/lib64/security/pam_motd.so
/etc/motd
/etc/ssh/sshd_config : Banner /path/to/banner

force disable all banner or motd
```
$ ssh user@host
$ touch ~/.hushlogin
```

$ cat /etc/ssh/sshd_config
ChallengeResponseAuthentication no
PasswordAuthentication no
UsePAM no

scripts:

TIMESTAMPE=$(date +"%Y%m%d%H%M%S")
SSHDFILE="/etc/ssh/sshd_config"
sudo cp "${SSHDFILE}{,.org.${TIMESTAMPE}}"

sudo bash -c '/bin/sed -i -e "s:^\(UsePAM.*$\):# \1:" ${SSHDFILE}'
sudo bash -c '/bin/sed -i -e "s:^\(PermitRootLogin.*$\):# \1:" ${SSHDFILE}'
sudo bash -c '/bin/sed -i -e "s:^\(ChallengeResponseAuthentication.*$\):# \1:" ${SSHDFILE}'
sudo bash -c '/bin/sed -i -e "s:^\(PasswordAuthentication.*$\):# \1:" ${SSHDFILE}'

if ! grep 'Add my marslo' ${SSHDFILE} > /dev/null 2>&1; then
  sudo bash -c "cat >> ${SSHDFILE}" << EOF

    # Add my marslo
    PermitRootLogin no
    UsePAM no
    ChallengeResponseAuthentication no
    PasswordAuthentication no
    PrintMotd yes
    Banner /etc/ssh/server.banner
  EOF
fi

disallow group to use password

[!TIP|label:references:]
Directive 'UsePAM' is not allowed within a Match block
Directive 'ChallengeResponseAuthentication' is not allowed within a Match block
Directive 'PrintMotd' is not allowed within a Match block
Directive 'LoginGraceTime' is not allowed within a Match block

if ! grep 'Add my marslo' ${SSHDFILE} > /dev/null 2>&1; then
  sudo bash -c "cat >> ${SSHDFILE}" << EOF

    # Add my marslo
    Match Group devops
                PasswordAuthentication no
                Banner /etc/ssh/server.banner
                MaxAuthTries 3
                ClientAliveInterval 600       # 60*10 secs
  EOF
fi

disallowing user to use tcp forwarding

Match User testuser
           AllowTcpForwarding no

displaying a special banner for users not in the staff group

Match Group *,!staff
  Banner /etc/banner.text

Match Host rootallowed.example.com
  PermitRootLogin yes

allowing anyone to use gatewayports from the local net

Match Address 192.168.0.0/24
  GatewayPorts yes

avoid ssh timeout by sending a keepalive message

$ echo 'ServerAliveInterval 60' >> /etc/ssh/ssh_config

# https://www.commandlinefu.com/commands/view/28/command-to-keep-an-ssh-connection-open
$ watch -n 30 uptime

debug

debug git

GIT_SSH_COMMAND

$ GIT_SSH_COMMAND="ssh -vvT" git clone git@github.com:Marslo/myblog.git
Cloning into 'myblog'...
OpenSSH_7.9p1, LibreSSL 2.7.3
debug1: Reading configuration data /Users/marslo/.ssh/config
debug1: /Users/marslo/.ssh/config line 1: Applying options for *
debug1: /Users/marslo/.ssh/config line 13: Applying options for github.com
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 48: Applying options for *
debug1: Connecting to github.com port 22.
^C

$ GIT_TRACE=1 git st
00:30:44.772137 git.c:703               trace: exec: git-st
00:30:44.772540 run-command.c:663       trace: run_command: git-st
00:30:44.772894 git.c:384               trace: alias expansion: st => status
00:30:44.772903 git.c:764               trace: exec: git status
00:30:44.772907 run-command.c:663       trace: run_command: git status
00:30:44.777379 git.c:440               trace: built-in: git status
On branch master
Your branch is up to date with 'origin/master'.

00:30:44.782714 run-command.c:663       trace: run_command: GIT_INDEX_FILE=.git/index git submodule summary --cached --for-status --summary-limit -1 HEAD
00:30:44.787490 git.c:703               trace: exec: git-submodule summary --cached --for-status --summary-limit -1 HEAD
00:30:44.788038 run-command.c:663       trace: run_command: git-submodule summary --cached --for-status --summary-limit -1 HEAD
00:30:44.838222 git.c:440               trace: built-in: git rev-parse --git-dir
00:30:44.845054 git.c:440               trace: built-in: git rev-parse --git-path objects
00:30:44.852811 git.c:440               trace: built-in: git rev-parse -q --git-dir
00:30:44.870362 git.c:440               trace: built-in: git rev-parse --show-prefix
00:30:44.878755 git.c:440               trace: built-in: git rev-parse --show-toplevel
00:30:44.893984 git.c:440               trace: built-in: git rev-parse -q --verify --default HEAD HEAD
00:30:44.899709 git.c:440               trace: built-in: git rev-parse --show-toplevel
00:30:44.905200 git.c:440               trace: built-in: git rev-parse --sq --prefix  --
00:30:44.911762 git.c:440               trace: built-in: git diff-index --cached --ignore-submodules=dirty --raw 52c94664ffc09cde2308c6bf9824ca0355ff5ff7 --
00:30:44.917374 run-command.c:663       trace: run_command: GIT_INDEX_FILE=.git/index git submodule summary --files --for-status --summary-limit -1
00:30:44.922165 git.c:703               trace: exec: git-submodule summary --files --for-status --summary-limit -1
00:30:44.922568 run-command.c:663       trace: run_command: git-submodule summary --files --for-status --summary-limit -1
00:30:44.965375 git.c:440               trace: built-in: git rev-parse --git-dir
00:30:44.972784 git.c:440               trace: built-in: git rev-parse --git-path objects
00:30:44.979117 git.c:440               trace: built-in: git rev-parse -q --git-dir
00:30:44.991077 git.c:440               trace: built-in: git rev-parse --show-prefix
00:30:44.997718 git.c:440               trace: built-in: git rev-parse --show-toplevel
00:30:45.012365 git.c:440               trace: built-in: git rev-parse -q --verify --default HEAD
00:30:45.018759 git.c:440               trace: built-in: git rev-parse --show-toplevel
00:30:45.024687 git.c:440               trace: built-in: git rev-parse --sq --prefix  --
00:30:45.031664 git.c:440               trace: built-in: git diff-files --ignore-submodules=dirty --raw --
nothing to commit, working tree clean

debug ssh

references:
OpenSSH Test Mode
OpenSSH Tip: Check Syntax Errors before Restarting SSHD Server
How to check SSH server's configuration validity

debug mode
```
$ sudo /usr/sbin/sshd -d [-d] [-d]
```

test mode

# -T : extended test mode
$ sudo /usr/sbin/sshd -T [-f /path/to/sshd_config]

# -t : test mode
$ sudo /usr/sbin/sshd -t [-f /path/to/sshd_config]

check sshd log

[!NOTE|label:references:]
How to check sshd log?

# -- check `/var/log/auth.log` --
$ grep 'sshd' /var/log/auth.log
# or
$ tail -f -n 500 /var/log/auth.log | grep 'sshd'

# -- check with journalctl --
## -t, --identifier
$ journalctl -t sshd

## -u, --unit
$ journalctl -u ssh

## -o json-pretty: with json format
$ journalctl -t sshd -o json-pretty

## -b0: since last boot
$ journalctl -t sshd -b0

## reserve order
$ journalctl -t sshd -b0 -r

tips

play video via ssh

$ ssh USER@HOST cat REMOTE_FILE.mp4 | tee LOCAL_FILE.mp4 | mplayer -

[!NOTE|label:references:]
* iMarslo: check local failed login

$ awk '/sshd/ && /Failed/ {gsub(/invalid user/,""); printf "%-12s %-16s %s-%s-%s\n", $9, $11, $1, $2, $3}' /var/log/auth.log

compare files via ssh

$ ssh user@host cat /path/to/remotefile | diff /path/to/localfile -

disconnect

[!TIP] Enter + ~ + . + Enter
references:
Linux / UNIX kill unresponsive hung SSH session
Kill an unresponsive ssh session without closing the terminal
How to kill or disconnect hung ssh session in Linux

$ <Enter>
$ ~?
Supported escape sequences:
 ~.   - terminate connection (and any multiplexed sessions)
 ~B   - send a BREAK to the remote system
 ~C   - open a command line
 ~R   - request rekey
 ~V/v - decrease/increase verbosity (LogLevel)
 ~^Z  - suspend ssh
 ~#   - list forwarded connections
 ~&   - background ssh (when waiting for connections to terminate)
 ~?   - this message
 ~~   - send the escape character by typing it twice
(Note that escapes are only recognized immediately after newline.)

execute shell commands via ssh

[!NOTE|label:references:]
* iMarslo: knrun
Bash while read loop breaking early
$ ssh -n username@hosts "cmd"
Source a file after ssh login drops to prompt | docker run -it over ssh: the input device is not a TTY
$ ssh -t username@hosts "cmd"

ctrl-c to break while loop

trap exit SIGINT SIGTERM; while read -r _node; do
  ssh -n username@${_node} "cmd"
done <<< "${hostname}"

execute local script via ssh

[!NOTE|label:references:]
How to use SSH to run a local shell script on a remote machine?

trap exit SIGINT SIGTERM; while read -r _node; do
  ssh -q username@${_node} 'bash -s' < '/path/to/scipt.sh'
done <<< "${hostname}"

# with eval
trap exit SIGINT SIGTERM; while read -r _node; do
  declare sshCmd="ssh -q username@${_node} 'bash -s' < \"/path/to/scipt.sh\""
  eval "${sshCmd}"
done <<< "${hostname}"

# or: https://www.commandlinefu.com/commands/view/5772/run-complex-remote-shell-cmds-over-ssh-without-escaping-quotes
$ ssh host -l user $(<cmd.txt)

how ssh works

Previousadmin tools Nextgit

Last updated 10 months ago

Was this helpful?