Documentation / Tutorials / Server

Commonly needed Linux commands

/ Server / Commonly needed Linux commands

At WPoets we manage all our servers in house, so here we have listed some of the most common commands that we use.

Disk space

Here is the command that gives us exact disk space.

df -h

This will give you the size of each folder within the folder path you have run the command

du -sh */ .*/ | sort -n

If you  to count the directories in a folder you do so by

ls -lR | grep ^d | wc -l

and, if you need the list of files just do

ls -1R | wc -l

To count files (even files without an extension) recursively from the root of the current directory, use:

ls -lR | grep ^- | wc -l

Here is an alternative

tree share/some/directory/ | tail -1

Dealing with Zip and Tar

Creating a Tar.gz file from a folder

tar -zcf wordpoets_com.tar.gz /_data/htdocs/

where wordpoets_com.tar.gz is the file name and  /_data/htdocs/ is the folder path we want to compress.

If you want to create tar file, with a relative path instead of full absolute url you can do something like this

tar -czf /var/www/backups/full/2019/20190807.tar.gz -C /var/www/backups/db/fullbackup/2019/07 20190807

It will create the tar file relative to 20190807 instead of  /var/www/backups/db/fullbackup/2019/07/20190807 which is what it generally does.

Let's say the above folder you want to extract within /htdocs/ folder, without creating another /htdocs/ within, use the following command

tar --strip-components=1  -xzvf wordpoets_com.tar.gz

the --strip-components allows you to set the depth while extracting the tar file.

If you need to compress a file or uncompress it we can use the following command, we generally use it to compress SQL files

gunzip abc.sql

it will create the compressed version of the file as abc.sql.gz and delete the abc.sql, and if you want to uncompress the abc.sql.gz file just type the following command to get the original file back.

gunzip abc.sql.gz

 

Fail2Ban

On servers managed by us, we use Fail2ban actively to block IPs and threats, but sometimes we end up banning IPs wrongly in those times from command easily allows you to check which IP was blocked by which rule, making it easier to identify

fail2ban-client status | sed -n 's/,//g;s/.*Jail list://p' | xargs -n1 fail2ban-client status

and, if just want to see list of jails active run following

fail2ban-client status

If you know the name of specific jail, you also use the following command to check the list of banned IP addresses

fail2ban-client status wordpress2

where wordpress2 is the name of the jail specified in fail2ban jail.conf file.

Finally, to unban an IP, you can use the following command

fail2ban-client set wordpress2 unbanip 103.68.18.146

just remember to replace the wordpress2 and the IP address with your own.

WGET

Once in a while, we need to run wget in a cron, and one the trouble was ignoring the output here is the setting that does that

wget -q -O - https://www.wpoets.com/crons/modules/add-hub-note >/dev/null 2>&1

The -O - option makes sure that the fetched content is sent to stdout.

another alternative is

wget -O /dev/null -o /dev/null https://www.mangalamjobs.com/site-notifications/candidate-reports

Here -O sends the downloaded file to /dev/null and -o logs to /dev/null instead of stderr. That way redirection is not needed at all.

Rsync & SCP

When moving files between two servers,  we currently prefer either rsync or scp depending on the situation.

rsync -a --partial /var/www/wordpoets.com/ /var/www/dev.wordpoets.com/  #folders are matched…

rsync -a --partial /var/www/dev.wordpoets.com/db root@192.168.0.15:/var/www/wordpoets.com/ #db folder will be created on new server

nohup rsync -a --partial /var/www/sync-vayas root@192.168.0.15:/root/sync/&  #this will create sync-vyas folder in remote sync folder

nohup rsync -av --partial --rsync-path "sudo rsync" /var/www/dev.wordpoets.com/htdocs ubuntu@192.168.0.15:/var/www/ls1.wordpoets.com/&  #to run sudo on remote

rsync -rP --ignore-existing /var/www/delphianlogic.com/htdocs2/wp-content/uploads/ /var/www/delphianlogic.com/htdocs/wp-content/uploads #ignore existing files in destination

RSYNC options:
-r to recurse into directories,
--ignore-existing to ignore existing files in the destination,
-P if you want to watch progress,
--list-only if you want to see what it would copy without actually copying, and a
-t if you want to preserve the timestamps.

scp -vrC /var/www/inc/ root@192.168.0.15:/var/www/inc/ #Copy all the files form inc folder to remote inc folder
scp foobar.txt your_username@remotehost.edu:/some/remote/directory #Copy the file "foobar.txt" from the local host to a remote host
scp your_username@remotehost.edu:foobar.txt /some/local/directory #Copy the file "foobar.txt" from a remote host to the local host
scp -r foo your_username@remotehost.edu:/some/remote/directory/bar #Copy the directory "foo" from the local host to a remote host's directory "bar"

SCP options:

r Recursively copy entire directories. Note that this follows symbolic links encountered in the tree traversal.
-C Compression enable. Passes the -C flag to ssh to enable compression.
-l limit – Limits the used bandwidth, specified in Kbit/s.
-o ssh_option – Can be used to pass options to ssh in the format used in ssh_config.
-P port – Specifies the port to connect to on the remote host. Note that this option is written with a capital ‘P’.
-p Preserves modification times, access times, and modes from the original file.
-q Quiet mode: disables the progress meter as well as warning and diagnostic messages from ssh.
-v Verbose mode. Print debugging messages about progress. This is helpful in debugging connection, authentication, and configuration problems.

Categories
Most Popular

Leave a Reply

Your email address will not be published. Required fields are marked *