ssh

reference:
* How SSH Really Works
SSH port forwarding - Example, command, server config
透过代理连接SSH
SSH ProxyCommand example: Going through one host to reach another server
OpenSSH/Cookbook/Proxies and Jump Hosts
Tunneling SSH over PageKite
Transparent Multi-hop SSH
SSH via HTTP proxy in OSX
ssh key
6 ssh authentication methods to secure connection (sshd_config)
8 ways to prevent brute force SSH attacks in Linux (CentOS/RHEL 7)
* SSH 是如何工作的 | How does SSH work

ssh key

[!TIP|label:resolve ssh permission issue:]
$ sudo su devops -c "restorecon -Rf ~/.ssh"
# or
$ /usr/sbin/restorecon .ssh/ .ssh/authorized_keys

generate ssh key

$ keyname='marslo@china'

# rsa4096
$ ssh-keygen -t rsa -b 4096 -f ~/.ssh/${keyname} -C "${keyname}" -P "" -q

# ed25519
$ ssh-keygen -t ed25519 -C "${keyname}" -f ~/.ssh/${keyname} -P '' -q
$ ssh-keygen -t ed25519 -o -a 100 -C "${keyname}" -f ~/.ssh/${keyname} -P '' -q

get servers public key

$ ssh-keyscan -H www.sample.com

# or
$ ssh-keyscan -p 29418 -t rsa www.sample.com
$ ssh-keyscan -p 29418 -t rsa www.sample.com >> ~/.ssh/known_hosts

upload the local ~/.ssh/know_hosts

[!NOTE|label:references:]
We updated our RSA SSH host key

$ ssh-keygen -R github.com
# Host github.com found: line 63
/home/marslo/.ssh/known_hosts updated.
Original contents retained as /home/marslo/.ssh/known_hosts.old

# or
$ curl -sL https://api.github.com/meta | jq -r '.ssh_keys | .[]'  | sed -e 's/^/github.com /' >> ~/.ssh/know_hosts

# more details
$ curl -sL https://api.github.com/meta | jq -r '.ssh_keys | .[]'
ssh-ed25519 AAAA***9GKJl
ecdsa-sha2-nistp256 AAAA***ockg=
ssh-rsa AAAA***wsjk=
$ curl -sL https://api.github.com/meta | jq -r '.ssh_keys | .[]'  | sed -e 's/^/github.com /'
github.com ssh-ed25519 AAAA***9GKJl
github.com ecdsa-sha2-nistp256 AAAA***ockg=
github.com ssh-rsa AAAA***wsjk=

add ssh key into agent

$ ssh-add ~/.ssh/${keyname}

# e.g.:
$ ssh-add ~/.ssh/id_ed25519
Identity added: /Users/marslo/.ssh/id_ed25519 (marslo@devops)

$ ssh-agent -s
SSH_AUTH_SOCK=/var/folders/s3/mg_f3cv54nn7y758j_t46zt40000gn/T//ssh-MgCrKA3ZS06N/agent.25376; export SSH_AUTH_SOCK;
SSH_AGENT_PID=25377; export SSH_AGENT_PID;
echo Agent pid 25377;

get public key from private key

$ ssh-keygen -y -f ~/.ssh/id_rsa > ~/.ssh/id_rsa.pub

verify public and private key pair

$ diff [-qs] <( ssh-keygen -y -e -f /path/to/private ) \
             <( ssh-keygen -y -e -f /path/to/public.pub )

generate new passphrase

$ ssh-keygen -p -f /path/to/private

get fingerprinter

sha256:

$ ssh-keygen -l -f ~/.ssh/id_rsa

# or
$ ssh-keygen -l -f ~/.ssh/id_rsa.pub

md5:

$ ssh-keygen -E md5 -l -f ~/.ssh/id_rsa

# or
$ ssh-keygen -E md5 -l -f ~/.ssh/id_rsa.pub

with openssl

$ openssl pkey -in ~/.ssh/ec2/primary.pem -pubout -outform DER | openssl md5 -c

keys performance

$ openssl speed rsa1024 rsa2048 dsa1024 dsa2048
Doing 1024 bit private rsa's for 10s: 91211 1024 bit private RSA's in 9.97s
Doing 1024 bit public rsa's for 10s: 1161461 1024 bit public RSA's in 9.93s
Doing 2048 bit private rsa's for 10s: 12305 2048 bit private RSA's in 9.94s
Doing 2048 bit public rsa's for 10s: 403455 2048 bit public RSA's in 9.98s
Doing 1024 bit sign dsa's for 10s: 84873 1024 bit DSA signs in 10.00s
Doing 1024 bit verify dsa's for 10s: 109544 1024 bit DSA verify in 9.99s
Doing 2048 bit sign dsa's for 10s: 30010 2048 bit DSA signs in 9.99s
Doing 2048 bit verify dsa's for 10s: 33202 2048 bit DSA verify in 9.98s
OpenSSL 1.0.2t  10 Sep 2019
built on: reproducible build, date unspecified
options:bn(64,64) rc4(ptr,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: clang -I. -I.. -I../include  -fPIC -fno-common -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -arch x86_64 -O3 -DL_ENDIAN -Wall -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
                  sign    verify    sign/s verify/s
rsa 1024 bits 0.000109s 0.000009s   9148.5 116964.9
rsa 2048 bits 0.000808s 0.000025s   1237.9  40426.4
                  sign    verify    sign/s verify/s
dsa 1024 bits 0.000118s 0.000091s   8487.3  10965.4
dsa 2048 bits 0.000333s 0.000301s   3004.0   3326.9

ssh

force use password

$ ssh -o PreferredAuthentications=password -o PubkeyAuthentication=no user@target.server

allows ssh-rsa

[!TIP|label:references:]
openssh v8.8

$ ssh -o HostkeyAlgorithms=+ssh-rsa -p <port> -l <user> <host>

for git commands

[!NOTE|label:references:]
since git v2.33.1

$ GIT_SSH_COMMAND="ssh -o HostkeyAlgorithms=+ssh-rsa" git clone
# since v2.41.0
$ GIT_SSH_COMMAND="ssh -o HostkeyAlgorithms=+ssh-rsa -o PubkeyAcceptedAlgorithms=+ssh-rsa" git clone

# or
$ git config [--global] core.sshCommand 'ssh -o HostKeyAlgorithms=+ssh-rsa'
# since v2.41.0
$ git config [--global] core.sshCommand 'ssh -o HostKeyAlgorithms=+ssh-rsa -o PubkeyAcceptedAlgorithms=+ssh-rsa'

force bypass `~/.ssh/config`

[!NOTE|label:references:]

How to tell ssh client to ignore ssh config file

$ man ssh

     -F configfile
             Specifies an alternative per-user configuration file.  If a configuration file is given on
             the command line, the system-wide configuration file (/etc/ssh/ssh_config) will be ignored.
             The default for the per-user configuration file is ~/.ssh/config.  If set to “none”, no
             configuration files will be read.

$ ssh -F none user@sample.com

# or
$ ssh -vvT -F /dev/null -i ~/.ssh/id_ed25519 user@sample.com

ssh and tar

[!TIP|label:references:]
* iMarslo: bash sugar - find and tar for JENKINS_HOME

copy multiple files to remote server

$ tar cvzf - -T list_of_filenames | ssh hostname tar xzf -

# or
$ tar cvf - /path/*.jpg | ssh foo@bar.com "tar xvf -"

tar all and extra in remote

# ssh -C
#    no `-z`      `-C`
#       |           |
#       v           v
$ tar cf - . | ssh -C hostname "cd ~/.marslo/test/; tar xvf -"
Warning: Permanently added '10.69.78.40' (ECDSA) to the list of known hosts.
./
./temp/
./a/
./a/a.txt
./c/
./b/

# tar z-flag
#     `-z`       no `-C`
#       |          |
#       v          v
$ tar cfz - . | ssh hostname "cd ~/.marslo/test/; tar xvzf -"

copy local file content into remote

[!NOTE|label:references:]
How to copy a file without using scp inside an ssh session?

$ ssh user@remote 'cat > /path/to/remote/file' < /path/to/local/file

# or
$ cat /path/to/local/file | ssh user@remote 'cat > /path/to/remote/file'

# to retrieve the file
$ ssh user@remote 'cat /path/to/remote/file' > pbcopy

with proxy

using command directly

Linux:

$ ssh -o 'ProxyCommand nc -x proxy.url.com proxy-port %h %p' -vT user@target.server

# or
$ ssh -o 'ProxyCommand corkscrew proxy.url.com proxy-port %h %p' -vT user@target.server

windows:

git for windows

$ ssh -o 'ProxyCommand connect -H http://proxy.url.com:proxy-port %h %p' user@target.server

cygwin

$ apt-cyg install corkscrew
$ ssh -o 'ProxyCommand corkscrew proxy.url.com proxy-port %h %p' user@target.server

# or
$ apt-cyg install nc
$ ssh -o 'ProxyCommand nc -X connect -x proxy.url.com:proxy-port %h %p' -vT git@github.com

start app remotely

$ ssh -f user@remote.ip DISPLAY=:0.0 smplayer movie.avi

launch firefox

$ ssh -fY user@REMOTESERVER firefox -no-remote

create a persistent connection to a machine

$ ssh -MNf <user>@<host>

ssh certificate

[!NOTE|label:references:]
* OpenSSH/Cookbook/Certificate-based Authentication
* Configure ssh certificate based authentication
14.3.3. Creating SSH CA Certificate Signing Keys
Files Associated with SSH Host Certificates
Certificate Authority (ca) - a private key generated for signing other keys
Certificate public key - the public component of the certificate authority
Host Public Key - the actual key that the SSH daemon uses to identify itself to the clients
Host Certificate - the signature made for the Host Public Key using the Certificate Authority
How to Generate and Configure SSH Certificate-Based Authentication
If you’re not using SSH certificates you’re doing SSH wrong
How to configure and setup SSH certificates for SSH authentication
How To Configure SSH Key-Based Authentication on a Linux Server
Tightening SSH access using short-lived SSH certificates
How to setup SSH certificates for SSH authentication

SSH User Certificates

ca

remote $ ssh-keygen -t ed25519 -b 4096 -f devops@ca -C 'devops@ca' -P '' -q
remote $ sudo cp devops@ca* /etc/ssh/
remote $ echo 'TrustedUserCAKeys /etc/ssh/devops@ca.pub' | sudo tee -a /etc/ssh/sshd_config

remote $ sudo grep TrustedUserCAKeys /etc/ssh/sshd_config
TrustedUserCAKeys /etc/ssh/devops@ca.pub

sign and generate cert key

# download devops@ca
#                                 key id  principals
#                                    v        v
local $ ssh-keygen -s devops@ca -I marslo -n marslo -V +52w ~/.ssh/marslo.pub
$ ls ~/.ssh/marslo*
/Users/marslo/.ssh/marslo  /Users/marslo/.ssh/marslo-cert.pub  /Users/marslo/.ssh/marslo.pub

verify

$ ssh-keygen -Lf ~/.ssh/marslo-cert.pub
/Users/marslo/.ssh/marslo-cert.pub:
        Type: ssh-ed25519-cert-v01@openssh.com user certificate
        Public key: ED25519-CERT SHA256:JfJnCDxjWhLwW3BcBX3XycBr3dT3JtHTwD1M4H3828E
        Signing CA: ED25519 SHA256:5dNlpIIjX/pdoNSpmtfcGQijSrl3W87TByA9KeCe2M0 (using ssh-ed25519)
        Key ID: "marslo"
        Serial: 0
        Valid: from 2023-08-17T17:41:00 to 2024-08-15T17:42:52
        Principals:
                marslo
        Critical Options: (none)
        Extensions:
                permit-X11-forwarding
                permit-agent-forwarding
                permit-port-forwarding
                permit-pty
                permit-user-rc

$ ssh -i ~/.ssh/marslo marslo@remote "cat /home/marslo/.ssh/authorized_keys; du -hs /home/marslo/.ssh/authorized_keys; hostname"
0 /home/marslo/.ssh/authorized_keys
remote

update for existing key

#                                       serial
#                                         v
$ ssh-keygen -s devops@ca -I 'edcba' -z '0002' -n marslo marslo.pub
Signed user key marslo-cert.pub: id "edcba" serial 2 for marslo valid forever

# verify
$ ssh-keygen -f marslo-cert.pub -L
marslo-cert.pub:
        Type: ssh-ed25519-cert-v01@openssh.com user certificate
        Public key: ED25519-CERT SHA256:JfJnCDxjWhLwW3BcBX3XycBr3dT3JtHTwD1M4H3828E
        Signing CA: ED25519 SHA256:5dNlpIIjX/pdoNSpmtfcGQijSrl3W87TByA9KeCe2M0 (using ssh-ed25519)
        Key ID: "edcba"
        Serial: 2
        Valid: forever
        Principals:
                marslo
        Critical Options: (none)
        Extensions:
                permit-X11-forwarding
                permit-agent-forwarding
                permit-port-forwarding
                permit-pty
                permit-user-rc

[!NOTE|label:references:]
OpenSSH/Cookbook/Certificate-based Authentication
$ ssh -o CertificateFile=server01.ed25519-cert.pub -i server01.ed25519 \
        fred@server01.example.org

$ ssh marslo@sample.server.com
marslo@sample.server.com's password:

$ ssh -o CertificateFile=marslo-cert.pub marslo@sample.server.com "du -hs ~/.ssh/authorized_keys"
0 /home/marslo/.ssh/authorized_keys

# via config
$ cat ~/.ssh/confg
Host example sample.server.com
     Hostname sample.server.com
     User marslo
     IdentitiesOnly yes
     IdentityFile /home/marslo/.ssh/marslo
     CertificateFile /home/marslo/.ssh/marslo-cert.pub

SSH Host Certificates

ssh tunnel

references:
ssh端口转发(跳板机)实战详解
SSH 通过跳板机直接访问内网机器

[!TIP|label:references:]
key point:
-L : <--
-R : -->
basic command line
$ ssh -Nf -L <localhost/localip>:<local_port>:<target>:<target_port> <to-be-mapping/localhost>
usage:
1 -> [2 ->] 3 : ssh host2:port2:host3:port3 host1:port1
if ignore host2. default using local.host
references:
SSH connection through host in the middle
$ ssh -t reachable_host ssh unreachable_host

two servers

`-L`

in jumper

[!TIP]
purpose:
local:22 --> jumper:6666

# -L : <--
$ ssh user@jumper.server
$ ssh -Nf -L [jumper.server:]6666:my.server:22 root@my.server

# verify
$ sudo netstat -an | grep 6666
tcp 0 0 jumper.ip:6666 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:6666 0.0.0.0:* LISTEN
tcp 0 0 ::1:6666 :::* LISTEN

`-R`

[!TIP]

status:

               current status
                    |
                    v
       mymac:22 <--------- jumper:22

 purpose:  |                  ^
           + ---------------> +

# login to jumper first via GUI
                         destination
                             |
                             v
<jumper> $ ssh -Nf -R 6666:<mymac>:22 marslo@<mymac>
<jumper> $ ps auxfww | grep ssh | grep -v grep
marslo    2281  0.0  0.0  11716  4064 ?        Ss   19:55   0:00      \_ ssh -Nf -R 6666:<my.pc.ip>:22 marslo@<my.pc.ip>
<jumper> $ sudo netstat -anp | grep ssh
tcp        0      0 <jumper>:46176    <mymac>:22        ESTABLISHED 2281/ssh

# verify in <mymac>
<mymac> $ netstat -an | grep 6666
tcp4       0      0  127.0.0.1.6666         *.*                    LISTEN
tcp6       0      0  ::1.6666               *.*                    LISTEN

<mymac> $ ssh -p 7777 localhost
## or
<mymac> $ cat ~/.ssh/config
Host  jumper
      Hostname              localhost
      Port                  7777
<mymac> $ ssh jumper

three serves

scenario 1

in jumper

> - purpose: > ```bash > local:6666 <--- jumper:6666 <--- remote:6666 > ```

# -R : -->
$ ssh user@jumper
$ ssh -Nf -R [jumper:]6666:local:6666 root@remote

# verify
## in remote
$ ssh root@remote
$ scp -P 6666 root@localhost:/remote/path/file /local/path
## in local
$ scp -P 6666 /local/path/file root@localhost:/remote/path/file

$ ssh user@jumper
<jumper> $ ps awwx | grep ssh | grep 6666
17549 ? Ss 0:00 ssh -Nf -L 6666:remote:22 root@remote
18740 ? Ss 3:50 ssh -Nf -R 6666:local:6666 root@remote

scenario 2

[!TIP]

details :

                    current status
                    --------------
                   |              |

          mymac  <----  jumper  ----> destination

            |                             ^
wants:      v                             |
             -------------->--------------
                   visit directly

# login to jumper
<jumper> $ ssh -Nf -R7777:<destination>:22 marslo@<mymac>
<jumper> $ ps auxfww | grep ssh | grep -v grep
marslo      41  0.0  0.0  11716  5832 ?        Ss   21:51   0:00      \_ ssh -Nf -R7777:<destination>:22 marslo@<my.mac.ip>

# verify in mymac
<mymac> $ ssh -p 7777 localhost       # localhost in <mymac> == <destination>
<mymac> $ netstat -an | grep 7777
tcp6       0      0  ::1.7777               ::1.65371              ESTABLISHED
tcp6       0      0  ::1.65371              ::1.7777               ESTABLISHED
tcp4       0      0  127.0.0.1.7777         *.*                    LISTEN
tcp6       0      0  ::1.7777               *.*                    LISTEN

config

ssh config

$ cat ~/.ssh/config
HOST  *
      LogLevel                   ERROR
      HostkeyAlgorithms          +ssh-rsa
      # PubkeyAcceptedAlgorithms +ssh-rsa              # since git v2.41.0
      GSSAPIAuthentication       no
      StrictHostKeyChecking      no
      UserKnownHostsFile         /dev/null
      IdentityFile               ~/.ssh/id_ed25519
      IdentityFile               ~/.ssh/id_rsa         # keep the older key if necessary

Include config.d/*

Host  github.com
      User                  marslo.jiao@gmail.com
      Hostname              ssh.github.com
      IdentityFile          /C/Marslo/my@key
      IdentitiesOnly        yes
      Port                  443
      # ProxyCommand        nc -X connect -x proxy.com:8080 %h %p
      # ProxyCommand        /usr/bin/nc -X 5 -x 127.0.0.1:1087 %h %p
      # ProxyCommand        /usr/local/bin/corkscrew 127.0.0.1 1087 %h %p

sshd_config

references:
Three locks for your SSH door
保护 SSH 的三把锁
7 Default OpenSSH Security Options You Should Change in /etc/ssh/sshd_config
sshd_config (4)

[!TIP]
disable Root Login : PermitRootLogin
allow only specific users or groups : AllowUsers, AllowGroups
deny specific users or groups : DenyUsers, DenyGroups
change sshd port number : Port
change login grace time : LoginGraceTime
Restrict the Interface (IP Address) to Login : ListenAddress
disconnect ssh when no activity : ClientAliveInterval

banner and motd

[!TIP] files:
/etc/pam.d/sshd : session optional pam_motd.so : /usr/lib64/security/pam_motd.so
/etc/motd
/etc/ssh/sshd_config : Banner /path/to/banner

force disable all banner or motd
```
$ ssh user@host
$ touch ~/.hushlogin
```

$ cat /etc/ssh/sshd_config
ChallengeResponseAuthentication no
PasswordAuthentication no
UsePAM no

scripts:

TIMESTAMPE=$(date +"%Y%m%d%H%M%S")
SSHDFILE="/etc/ssh/sshd_config"
sudo cp "${SSHDFILE}{,.org.${TIMESTAMPE}}"

sudo bash -c '/bin/sed -i -e "s:^\(UsePAM.*$\):# \1:" ${SSHDFILE}'
sudo bash -c '/bin/sed -i -e "s:^\(PermitRootLogin.*$\):# \1:" ${SSHDFILE}'
sudo bash -c '/bin/sed -i -e "s:^\(ChallengeResponseAuthentication.*$\):# \1:" ${SSHDFILE}'
sudo bash -c '/bin/sed -i -e "s:^\(PasswordAuthentication.*$\):# \1:" ${SSHDFILE}'

if ! grep 'Add my marslo' ${SSHDFILE} > /dev/null 2>&1; then
  sudo bash -c "cat >> ${SSHDFILE}" << EOF

    # Add my marslo
    PermitRootLogin no
    UsePAM no
    ChallengeResponseAuthentication no
    PasswordAuthentication no
    PrintMotd yes
    Banner /etc/ssh/server.banner
  EOF
fi

disallow group to use password

[!TIP|label:references:]
Directive 'UsePAM' is not allowed within a Match block
Directive 'ChallengeResponseAuthentication' is not allowed within a Match block
Directive 'PrintMotd' is not allowed within a Match block
Directive 'LoginGraceTime' is not allowed within a Match block

if ! grep 'Add my marslo' ${SSHDFILE} > /dev/null 2>&1; then
  sudo bash -c "cat >> ${SSHDFILE}" << EOF

    # Add my marslo
    Match Group devops
                PasswordAuthentication no
                Banner /etc/ssh/server.banner
                MaxAuthTries 3
                ClientAliveInterval 600       # 60*10 secs
  EOF
fi

disallowing user to use tcp forwarding

Match User testuser
           AllowTcpForwarding no

displaying a special banner for users not in the staff group

Match Group *,!staff
  Banner /etc/banner.text

Match Host rootallowed.example.com
  PermitRootLogin yes

allowing anyone to use gatewayports from the local net

Match Address 192.168.0.0/24
  GatewayPorts yes

avoid ssh timeout by sending a keepalive message

$ echo 'ServerAliveInterval 60' >> /etc/ssh/ssh_config

# https://www.commandlinefu.com/commands/view/28/command-to-keep-an-ssh-connection-open
$ watch -n 30 uptime

debug

debug git

GIT_SSH_COMMAND

$ GIT_SSH_COMMAND="ssh -vvT" git clone git@github.com:Marslo/myblog.git
Cloning into 'myblog'...
OpenSSH_7.9p1, LibreSSL 2.7.3
debug1: Reading configuration data /Users/marslo/.ssh/config
debug1: /Users/marslo/.ssh/config line 1: Applying options for *
debug1: /Users/marslo/.ssh/config line 13: Applying options for github.com
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 48: Applying options for *
debug1: Connecting to github.com port 22.
^C

$ GIT_TRACE=1 git st
00:30:44.772137 git.c:703               trace: exec: git-st
00:30:44.772540 run-command.c:663       trace: run_command: git-st
00:30:44.772894 git.c:384               trace: alias expansion: st => status
00:30:44.772903 git.c:764               trace: exec: git status
00:30:44.772907 run-command.c:663       trace: run_command: git status
00:30:44.777379 git.c:440               trace: built-in: git status
On branch master
Your branch is up to date with 'origin/master'.

00:30:44.782714 run-command.c:663       trace: run_command: GIT_INDEX_FILE=.git/index git submodule summary --cached --for-status --summary-limit -1 HEAD
00:30:44.787490 git.c:703               trace: exec: git-submodule summary --cached --for-status --summary-limit -1 HEAD
00:30:44.788038 run-command.c:663       trace: run_command: git-submodule summary --cached --for-status --summary-limit -1 HEAD
00:30:44.838222 git.c:440               trace: built-in: git rev-parse --git-dir
00:30:44.845054 git.c:440               trace: built-in: git rev-parse --git-path objects
00:30:44.852811 git.c:440               trace: built-in: git rev-parse -q --git-dir
00:30:44.870362 git.c:440               trace: built-in: git rev-parse --show-prefix
00:30:44.878755 git.c:440               trace: built-in: git rev-parse --show-toplevel
00:30:44.893984 git.c:440               trace: built-in: git rev-parse -q --verify --default HEAD HEAD
00:30:44.899709 git.c:440               trace: built-in: git rev-parse --show-toplevel
00:30:44.905200 git.c:440               trace: built-in: git rev-parse --sq --prefix  --
00:30:44.911762 git.c:440               trace: built-in: git diff-index --cached --ignore-submodules=dirty --raw 52c94664ffc09cde2308c6bf9824ca0355ff5ff7 --
00:30:44.917374 run-command.c:663       trace: run_command: GIT_INDEX_FILE=.git/index git submodule summary --files --for-status --summary-limit -1
00:30:44.922165 git.c:703               trace: exec: git-submodule summary --files --for-status --summary-limit -1
00:30:44.922568 run-command.c:663       trace: run_command: git-submodule summary --files --for-status --summary-limit -1
00:30:44.965375 git.c:440               trace: built-in: git rev-parse --git-dir
00:30:44.972784 git.c:440               trace: built-in: git rev-parse --git-path objects
00:30:44.979117 git.c:440               trace: built-in: git rev-parse -q --git-dir
00:30:44.991077 git.c:440               trace: built-in: git rev-parse --show-prefix
00:30:44.997718 git.c:440               trace: built-in: git rev-parse --show-toplevel
00:30:45.012365 git.c:440               trace: built-in: git rev-parse -q --verify --default HEAD
00:30:45.018759 git.c:440               trace: built-in: git rev-parse --show-toplevel
00:30:45.024687 git.c:440               trace: built-in: git rev-parse --sq --prefix  --
00:30:45.031664 git.c:440               trace: built-in: git diff-files --ignore-submodules=dirty --raw --
nothing to commit, working tree clean

debug ssh

references:
OpenSSH Test Mode
OpenSSH Tip: Check Syntax Errors before Restarting SSHD Server
How to check SSH server's configuration validity

debug mode
```
$ sudo /usr/sbin/sshd -d [-d] [-d]
```

test mode

# -T : extended test mode
$ sudo /usr/sbin/sshd -T [-f /path/to/sshd_config]

# -t : test mode
$ sudo /usr/sbin/sshd -t [-f /path/to/sshd_config]

check sshd log

[!NOTE|label:references:]
How to check sshd log?

# -- check `/var/log/auth.log` --
$ grep 'sshd' /var/log/auth.log
# or
$ tail -f -n 500 /var/log/auth.log | grep 'sshd'

# -- check with journalctl --
## -t, --identifier
$ journalctl -t sshd

## -u, --unit
$ journalctl -u ssh

## -o json-pretty: with json format
$ journalctl -t sshd -o json-pretty

## -b0: since last boot
$ journalctl -t sshd -b0

## reserve order
$ journalctl -t sshd -b0 -r

tips

play video via ssh

$ ssh USER@HOST cat REMOTE_FILE.mp4 | tee LOCAL_FILE.mp4 | mplayer -

[!NOTE|label:references:]
* iMarslo: check local failed login

$ awk '/sshd/ && /Failed/ {gsub(/invalid user/,""); printf "%-12s %-16s %s-%s-%s\n", $9, $11, $1, $2, $3}' /var/log/auth.log

compare files via ssh

$ ssh user@host cat /path/to/remotefile | diff /path/to/localfile -

disconnect

[!TIP] Enter + ~ + . + Enter
references:
Linux / UNIX kill unresponsive hung SSH session
Kill an unresponsive ssh session without closing the terminal
How to kill or disconnect hung ssh session in Linux

$ <Enter>
$ ~?
Supported escape sequences:
 ~.   - terminate connection (and any multiplexed sessions)
 ~B   - send a BREAK to the remote system
 ~C   - open a command line
 ~R   - request rekey
 ~V/v - decrease/increase verbosity (LogLevel)
 ~^Z  - suspend ssh
 ~#   - list forwarded connections
 ~&   - background ssh (when waiting for connections to terminate)
 ~?   - this message
 ~~   - send the escape character by typing it twice
(Note that escapes are only recognized immediately after newline.)

execute shell commands via ssh

[!NOTE|label:references:]
* iMarslo: knrun
Bash while read loop breaking early
$ ssh -n username@hosts "cmd"
Source a file after ssh login drops to prompt | docker run -it over ssh: the input device is not a TTY
$ ssh -t username@hosts "cmd"

ctrl-c to break while loop

trap exit SIGINT SIGTERM; while read -r _node; do
  ssh -n username@${_node} "cmd"
done <<< "${hostname}"

execute local script via ssh

[!NOTE|label:references:]
How to use SSH to run a local shell script on a remote machine?

trap exit SIGINT SIGTERM; while read -r _node; do
  ssh -q username@${_node} 'bash -s' < '/path/to/scipt.sh'
done <<< "${hostname}"

# with eval
trap exit SIGINT SIGTERM; while read -r _node; do
  declare sshCmd="ssh -q username@${_node} 'bash -s' < \"/path/to/scipt.sh\""
  eval "${sshCmd}"
done <<< "${hostname}"

# or: https://www.commandlinefu.com/commands/view/5772/run-complex-remote-shell-cmds-over-ssh-without-escaping-quotes
$ ssh host -l user $(<cmd.txt)

how ssh works

Previousadmin tools Nextgit

Last updated 1 year ago

hashtagssh key

hashtagget servers public key

hashtagadd ssh key into agent

hashtagget public key from private key

hashtaggenerate new passphrase

hashtagget fingerprinter

hashtagwith openssl

hashtagkeys performance

hashtagssh

hashtagforce use password

hashtagallows ssh-rsa

hashtagforce bypass ~/.ssh/config

hashtagtar all and extra in remote

hashtagcopy local file content into remote

hashtagwith proxy

hashtagusing command directly

hashtagssh certificate

hashtagca

hashtagsign and generate cert key

hashtagupdate for existing key

hashtaglogin via specific cert

hashtagssh tunnel

hashtagtwo servers

hashtag-L

hashtag-R

hashtagthree serves

hashtagscenario 1

hashtagscenario 2

hashtagconfig

hashtagssh config

hashtagsshd_config

hashtagbanner and motd

hashtagdisable login password

hashtagdisallow group to use password

hashtagdisallowing user to use tcp forwarding

hashtagdisplaying a special banner for users not in the staff group

hashtagallowing root login from host rootallowed.example.com

hashtagallowing anyone to use gatewayports from the local net

hashtagdebug

hashtagdebug git

hashtagdebug ssh

hashtagcheck sshd log

hashtagtips

hashtagdisconnect

hashtagexecute shell commands via ssh

hashtagctrl-c to break while loop

hashtagexecute local script via ssh