Linux Commands Every Data Engineer Needs to Know
The Linux command line is a text interface that runs on the command line. Often referred to as the shell, terminal, console, prompt or various other names.
When dealing with the Linux operating system, commands are required to perform a specific operation. Linux commands will allow you to manipulate, navigate and display files and directories, change permissions, start/stop services, display informations, and more.
Getting Information About a Command
Almost all Linux commands are distributed with a manual page. A manual page also known as man page is documentation that explains what the command does, how you run the command and what arguments it accepts.
man command_nameMost commands also have a --help option, which prints a short message about how to use the command.
command_name --helppwd, cd, whoami
pwd (print working directory) command to find out what directory you are currently working on.
cd (change directory) command is used to change current working directory.
cd .. command moves one directory up.
whoami displays the username of the current user.
ls
ls command displays the files and directories within a directory.
ls -l
To get the additional information and a cleaner view, use the flag “-l”.
ls -lh
This command will show you the file sizes in human readable format.
ls -lr
To sort the output in reverse order, use the flag “-r”.
ls -lt
To sort the files and directories by time and date of creation/modification, use the flag “-t” instead.
ls -ltr
To sort the file names in the last modification time in reverse order.
ls -la
To show the hidden files, use the flag “-a”.
mkdir, rmdir
mkdir command is used for creating a directory.
To remove an empty directory, we use rmdir command.
touch, rm
touch command is used to create empty files. If the file already exists, touch will change the file’s last access and modification times to the current time.
rm command uses to remove files and directories. This command doesn’t remove directories by default. If you want to delete an empty directory, you can use the recursive (-r) flag and to remove a directory with content inside of it, you need to use the force (-rf) and recursive flags.
cp, mv
cp command allows you to copy files and directories.
cp file file_backup
To copy a directory, including all its files and subdirectories, use the -R or -r option.
mv command is used to move files and directories. You can also use this command for renaming files and directories.
which
which command is used to locate the executable files or location of a program from the file system. It displays the path where the specified file or command is stored.
echo
echo command writes text to standard output (stdout).
Input a line of text and display it on standard output:
Declare a variable and echo its value:
echo command can be used with a redirect operator to output to a file and not standard output.
cat, tac, more, less, head, tail
cat command displays a file’s content as output. It also helps us to create, view, concatenate files.
It creates a file withcat > file.txt Press Enter key, type the text and when you are done pressCTRL+D to save the file.
To view a single file cat file.txt:
To view with line numbers cat -n file.txt
To copy the content of a file to another file:
To append the content of a file to another file:
The taccommand is the reverse of thecat command. It will display the file content in reverse order.
more command is similar to cat command. The only difference between both commands is that, in case of larger files, the more command displays screenful output at a time.
In more command, the following keys are used to scroll the page:
ENTER key: To scroll down page by line.
Space bar: To move to the next page.
b key: To move to the previous page.
/ key: To search the string.
less command is a command line utility that displays the contents of a file or a command output, one page at a time.
head command is used to display the content of a file. It displays the first 10 lines of a file by default. head -n option displays specified number of lines.
tail command is used to display the last ten lines by default. You can also use-n option to display more lines.
alias
You can save yourself some time by creating aliases for your most used commands.
$ alias wr=”cd /var/www/html”
histroy
history command shows all the basic commands in Linux that you have used in the past for the current terminal session.
locate, find
locate command is used to search a file by file name. It searches the file in the database. It is faster than the find command. To find the file with the locatecommand, keep your database updated. Using the -i argument along with this command will make it case-insensitive. To search for a file that contains two or more words, use an asterisk (*).
Similar to the locatecommand, using findalso searches for files and directories. The difference is, you use the findcommand to locate files within a given directory.
grep
grep command is useful for searching the content from a file. Generally, it is used with the pipe.
su, sudo
The su command is used to run a function as a different user. It is the easiest way to switch or change to the administrative account in the current logged in session.
The sudo command allows you to run programs with the security privileges of another user
The main difference between suand sudois that the sucommand can interchange between superuser and root user if executed without prior additional options while the sudocommand provides single root privileges. sudemands the password of the root account while sudodemands the password of the current user account.
diff
diff stands for difference. This command is used to display the differences in the files by comparing the files line by line.
clear
clear command to clean out the terminal. You can also use CTRL+L.
Copying/Pasting Commands into the Terminal
For copying a text from a source, we would use basically CTRL+C keyboard shortcut. But in the terminal, we use it for terminate the command. So, you need to use CTRL+SHIFT+C for copying and CTRL+SHIFT+V for pasting. You can also use drop-down menu of the terminal.
