SSH, SCP and Rsync

There are many ways to access a shell remotely. One of the older ways is to use the telnet program, which is available on most network capable operating systems.

Accessing shell account through the telnet method, it is not at all secure because everything that you send and receive over the telnet method is visible in an plain text on your local network. For this reason, you need a more secure program that telnet to connect to remote host.

What is SSH

Secure Shell (SSH) is an open source and most trusted network protocol for operating network services securely over an unsecured network. The SSH protocol is a method for secure remote login from one computer to another. It is also used to transfer files from one computer to another computer over the network using secure copy (SCP) Protocol.

It provides several alternative options for strong authentication, and it protects the communications security and integrity with strong encryption.

The SSH protocol is used in corporate networks for:

  • providing secure access for users and automated processes
  • interactive and automated file transfers
  • issuing remote commands
  • managing network infrastructure and other mission-critical system components.

Configuration and demon

Configuration file : /etc/ssh/sshd_config

SSH demon or service is sshd

# service sshd status
openssh-daemon (pid 2130) is running..

Accessing the remote machine using SSH.

To access the remote machine using ssh, the syntax is

# ssh usename@IP Address/Hostname

# ssh 192.168.150.133
The authenticity of host '192.168.150.133 (192.168.150.133)' can't be established.
RSA key fingerprint is 88:33:32:74:08:aa:1c:2c:54:23:be:e2:ec:52:fb:ff.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.150.133' (RSA) to the list of known hosts.

For the first time, it will ask you if you wish to add the remote host to a list of know_hosts, go ahead and say yes. Then it will ask for password to login.

To leave the session, just type exit or logout or use shortcut CTRL+D command and you will back to your own machine.

Password less login using SSH keys (trusted connection).

If you are dealing with number of Linux remote servers, then SSH Password-less login is one of the best way to automate tasks such as automatic backups with scripts, synchronization files using scp and remote command execution.

By generating SSH keys, a public key and private key, it will work as authorized access and it will not prompt for password.

SSH keys are an implementation of public-key cryptography. They solve the problem of brute-force password attacks by making then computationally impractical.
Public key cryptography uses a public key to encrypt data and private key to decrypt data.

Generating SSH key pair.

To generate a pair of public keys use following command.

# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): [Press enter key]
Enter passphrase (empty for no passphrase): [Press enter key]
Enter same passphrase again: [Press enter key]
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
0b:51:d3:ac:96:74:0a:e9:b1:9c:f7:74:35:23:d1:15 root@myserver.localdomain
The key's randomart image is:
+--[ RSA 2048]----+
|        .oo .o Eo|
|      +...+ . =  |
|       o.* = o o |
|      =.* . .    |
|      .oSo .     |
|      . ..       |
|        .        |
|                 |
|                 |
+-----------------+

It will prompt above to mention the file where these keys should be stored, to keep its default directory just press Enter. The default location will be /root/.ssh/ .

# cd /root/.ssh/
# ls
id_rsa id_rsa.pub

Copying the public key on client machine.

To copy the server’s public key in clinet system, the command is

# ssh-copy-id -i /root/.ssh/id_rsa.pub 192.168.150.133
root@192.168.150.133's password:
Now try logging into the machine, with "ssh '192.168.150.133'", and check in:

.ssh/authorized_keys

to make sure we haven't added extra keys that you weren't expecting.

#

Now check in client machine.

# cat /root/.ssh/authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5zWA3lFZ+ZL4Vnlpw8FdBTCMV7NBLSq/B9URVb5NUIMF8w+2zzqTWh5jC2+/2cjS1cIYtUPn03FyViKEKnRukI7iCuybTOcGWyoJW10sZIkhO61pRQjj2CH8M755Y970LkbjFU4WY7xXcsQo5IA+TymxxhP0MiDJg4IvWU4fIwajdA97JPE054IOARsBgNKLSF+Be8sVzYQqr32LXGzR8ACTQS+YYmAZKPH0Kh/vOCZOKweerW9QDxAH5E0GJbjZAbQsbhfkm790HEFKETuQsx5ElklChsXbPlkCFlHCnIonJN2ZxUb9wH32GmdHur3YKw5TtToyZCyONhxeBh+iNQ== root@myserver.localdomain

Try login to the client machine using SSH, check whether it is asking for password.

[root@myserver ~]# ssh 192.168.150.133
Last login: Thu Sep 14 04:26:31 2016 from 192.168.150.1
[root@localhost ~]#

It will not prompt for password once trusted connection established.

Remote file transfer with SCP and RSYNC

Secure copy or SCP is a means of securely transferring computer files between a local host and a remote host or between two remote hosts. It is based on the Secure Shell (SSH) protocol.

Copying file to host, the sytax is

scp SourceFile user@host:directory/TargetFile

Local Server

# scp file1.txt root@192.168.150.133:/root/file1
file1.txt 100% 0 0.0KB/s 00:00
#

Remote server

# ls file1
file1
#

Copying file from host, the sytax is

scp user@host:directory/SourceFile TargetFile
scp -r user@host:directory/SourceFolder TargetFolder

Remote server

# ls features.txt
features.txt
#

Local server

# scp root@192.168.150.133:/root/features.txt features.txt
features.txt 100% 176 0.2KB/s 00:00
# ls features.txt
features.txt
#

To copy all files under direcory :

Remote server

# ll |wc -l
21
#

Local Server

# ll |wc -l
6
#

# scp -r root@192.168.150.133:/root/ /root/

features.txt    100% 176 0.2KB/s 00:00
Install.log     100% 0 0.0KB/s 00:00
anaconda-ks.cfg 100% 3277 3.2KB/s 00:00
.xauthER2pZD    100% 66 0.1KB/s 00:00
.xauthntfvB2    100% 66 0.1KB/s 00:00
sample.txt      100% 109 0.1KB/s 00:00
file2           100% 23 0.0KB/s 00:00
.cshrc          100% 100 0.1KB/s 00:00
...........................
...........................
...........................
#

# cd root/
# ll |wc -l
21
#

If the remote host uses a port other than the default of 22, it can be specified in the command. For example, copying a file from host:

scp -P 2222 user@host:directory/SourceFile TargetFile

If we don’t have trusted connection then it will prompt for password.

Rsync

Rsync is a utility for efficiently transferring and synchronizing files across computer systems, by checking the timestamp and size of files.

Rsync is typically used for synchronizing files and directories between two different systems.

For example, if the command rsync local-file user@remote-host:remote-file is run, rsync will use SSH to connect as user to remote-host. Once connected, it will invoke the remote host’s rsync and then the two programs will determine what parts of the file need to be transferred over the connection.

Rsync can also operate in a daemon mode, serving files in the native rsync protocol.

Basic syntax of rsync command

rsync options source destination

Some common options used with rsync commands

-v : verbose
-r : copies data recursively (but don’t preserve timestamps and permission while transferring data
-a : archive mode, archive mode allows copying files recursively and it also preserves symbolic links, file permissions, user & group ownerships and timestamps
-z : compress file data
-h : human-readable, output numbers in a human-readable format

Copy/Sync Files and Directory Locally

This following command will sync a single file on a local machine from one location to another location.

# rsync -zvh backup.tar /tmp/backups/

The following command will transfer or sync all the files of from one directory to a different directory in the same machine.

# rsync -avzh /root/rpmpkgs /tmp/backups/

Copy/Sync Files and Directory to or From a Server

This following command will sync a directory from a local machine to a remote machine.

# rsync -avz rpmpkgs/ root@192.168.150.133:/home/

Copy/Sync a Remote Directory to a Local Machine

# rsync -avzh root@192.168.150.133:/root/rpmpkgs /tmp/myrpms

Rsync Over SSH

With rsync, we can use SSH (Secure Shell) for data transfer, using SSH protocol while transferring our data you can be ensured that your data is being transferred in a secured connection with encryption so that nobody can read your data while it is being transferred over the wire on the internet.

To specify a protocol with rsync you need to give “-e” option with protocol name you want to use. Here in this example, We will be using ssh with -e option and perform data transfer.

# rsync -avzhe ssh root@192.168.150.133:/root/install.log /tmp/

Copy a File from a Local Server to a Remote Server with SSH

# rsync -avzhe ssh backup.tar root@192.168.150.133:/backups/

Show Progress While Transferring Data with rsync

# rsync -avzhe ssh --progress backup.tar root@192.168.150.133:/backups/

Use of -include and -exclude Options

These two options allows us to include and exclude files by specifying parameters with these option helps us to specify those files or directories which you want to include in your sync and exclude files and folders with you don’t want to be transferred.

# rsync -avzhe ssh --include '*.xml' --exclude '*.html' root@192.168.150.133:/home/sree/ /tmp/

Use of –delete Option

If a file or directory not exist at the source, but already exists at the destination, you might want to delete that existing file/directory at the target while syncing. We can use ‘–delete’ option to delete files that are not there in source directory.

# rsync -avz --delete root@192.168.150.133:/var/lib/rpm/

Set the Max Size of Files to be Transferred

You can specify the Max file size to be transferred or sync. You can do it with “–max-size” option.

# rsync -avzhe ssh --max-size='200k' root@192.168.150.133:/root/ /tmp/

Set Bandwidth Limit and Transfer File

# rsync --bwlimit=100 -avzhe ssh root@192.168.150.133:/root/ /tmp/

Do a Dry Run with rsync

If you are a newbie and using rsync and don’t know what exactly your command going do. Rsync could really mess up the things in your destination folder and then doing an undo can be a tedious job.

# rsync --dry-run -zvh backup.tar /tmp/backups/

See manual page (man command) for more options.

 

Advertisements