Description

As part of this course, you will learn most of the important Linux commands that can come in handy while working in Linux-based environments.

Here are the high-level details of the Linux Commands that are covered as part of this hands-on course.

    You will get started by having an appropriate environment to practice. Windows users can set up Ubuntu 20.04 using wsl.
    Once the environment is ready, you will understand how to get started with basic Linux Commands.
    The most commonly used command in Linux is ls. It is used to list the files. We will see different variations of ls by going through all the relevant arguments.
    One of the powerful Linux commands and also my favorite is “find“. It helps us to filter the files and directories based on name, type, modified time, etc. You will learn how to use the find command effectively from the perspective of finding the files based on the criteria.
    As we understand how to get details related to files and folders in Linux, we will go through the details related to previewing the data in files using important Linux commands such as cat, tail, more, etc. Also, we will review Linux commands such as cut, sort, uniq to process the data. All these commands are typically used against text files.
    After going through details related to ls, find commands as well as commands related to processing data in files, we will walk you through the details related to file and folder properties in Linux. You will get an overview of commands such as useradd, userdel, chmod, etc which are typically associated with file permissions.
    One of the key aspects related to files and folders is to manage them effectively. We will go through the details about managing files and folders using Linux commands such as cp for copy, mv for move or rename, rm for delete or remove, tar to build uncompressed as well as compressed tar balls. Also, you will get a taste of the awk command to automate some of the common tasks such as reorganizing the files based on patterns.
    The core components of a computer are nothing but CPU, Memory, and Storage. In this section or module, you will be learning Linux commands such as lscpu to get CPU details, free to get memory details, and then df and du to get storage details.
    At times, we need to make sure the data is in files as well as codebase to troubleshoot the production issues or bugs in QA. In this section, you will learn how to use the Linux command called diff to compare files and folders and review the differences.
    Environment Variables are the ones that can impact the sessions logged in by users in Linux. As part of this section, you will be learning details related to Variables, Environment Variables. Also, you will understand how to list all the Environment Variables using Linux commands such env, and how to get a value of a Variable or Environment Variable using the Linux command echo. On top of these things, you will also learn all important aspects related to one of the most important Environment Variables called PATH.
    As we gain a decent amount of knowledge related to Linux commands using a single server, it is time for us to explore other important commands such as ssh, scp, pssh, pscp, etc which require more than one Linux System or Server. Also, we need to go through basic networking concepts. To learn commands to interact between the servers as well as to understand basic networking concepts, we will get started with AWS to provision Ubuntu-based virtual machines. These Virtual Machines will be used in the subsequent sections to learn the concepts mentioned earlier as part of this bullet point.
    Once we go through the details about getting started with AWS, we will focus on learning all key concepts related to SSH such as sshd daemon, updating ssh config files, and restarting sshd using systemctl or service command. Also, we will go through the details related to passwordless login and also ability to run commands remotely via ssh. We will also understand how to set up parallel ssh and then run commands on multiple Linux systems in parallel.
    To be comfortable with learning in a Linux environment, it is very important for one to understand key concepts such as different types of IP Addresses, Port Numbers, relationships between applications and databases with ports, etc. In this section, we will go through all important networking concepts using apache2, MongoDB, etc.
    Once we are comfortable with SSH, it is time for us to use tools such as scp which uses ssh under the hood to copy files and folders between the Linux Systems. We will also explore parallel scp to copy files in parallel to multiple target systems from one source system. Also, we will go through the details related to speeding up the copy using different techniques which include submitting multiple scp commands using nohup.
    One of the important aspects for any IT Engineer is to troubleshoot the application issues deployed in Linux by reviewing logs generated by applications. After going through scp, you will learn how to troubleshoot issues by using grep and find against log files of the applications deployed on the Linux Server.
    To ensure that we can monitor and manage Linux Servers, we not only need to understand how to manage applications and databases in Linux but also need to understand the performance of the applications. For managing applications and databases on a Linux server, one should be familiar with commands such as apt to install applications and systemctl or service to manage the life cycle of applications and databases. To monitor, one should be familiar with commands such as ps, lsof, top, etc. You will learn how to monitor and manage applications as well as databases using examples such as ssh, apache2, mongodb, jupyterlab, etc.
    As we learned quite a few commands and tools on Linux, now it is time for us to work on increasing productivity by using appropriate tools, especially for Windows users. You will see the relevance and also how to setup tools such as notepad++, winscp, visual studio code with remote development extension, git tools for Windows, etc.
    Organizations will end up having hundreds of servers to support their applications. However, these hundreds of servers need to follow a handful of templates. Using Server Automation tools like Ansible, we should be able to automate the process of configuring a large number of servers using a handful of templates. You will get an overview of Ansible covering some of the core concepts of Ansible such as running commands, inventory, playbooks, variables, roles, etc.
    Also, you will understand how to use programming languages like Python to take care of automation by using modules such as os, glob, subprocess, etc.

Basic Linux Commands

    Running the first command on Linux
    Getting usage of Linux commands using the help
    Overview of Files and Folders
    Fully Qualified or Absolute Paths and Relative Paths of Files and Folders

Listing files and folders in Linux File System using ls

    Overview of ls command to list files and folders in Linux
    Setup Files and Folders from GitHub Account to learn Shell Commands
    Setup Jupyter Lab to go through help on linux commands
    Getting the properties of files and folders using the ls command in Linux
    Sorting ls output while listing files in Linux
    List files and folders in Linux sorted by time
    List files and folders in Linux sorted by size
    Overview of Hidden Folders and Files
    Listing Hidden Files and Folders in Linux
    Listing the files and folders recursively in Linux
    Exploring Control Arguments while listing files in Linux
    Overview of representation of file sizes
    Overview of alias command and ll to list files and folders

Filtering files and folders in Linux File System

    Introduction to the filtering files and folders on Linux File System
    Accessing Windows Files and Folders from Linux setup using wsl
    Overview of grep command
    Few basic examples using grep and piping
    Overview of Piping while running shell commands
    Overview of Basic Pattern Matching
    Deep Dive into the wc command to get word count or line count
    Overview of the find command to filter files and folders
    Understand the behavior of the simplest form of find command in Linux
    Using Linux find command to find directories or folders
    Using Linux find command to find files by type and pattern
    Overview of Standard Directories in Linux
    Get the list of files based on the modified time using the Linux find command
    Get the list of files based on size using the Linux find command

Processing Data in Files using Linux Commands

    Introduction to commands used for processing data in files using Linux
    Overview of Different File Formats
    Get file format or type using file command in Linux
    Preview the data in the text file using head command in Linux
    Preview the data in the text file using tail command
    Install apache web server on Ubuntu Linux
    Generate Apache Logs every second using shell script and nohup
    Monitoring Logs using tail command in Linux
    Preview the data in the text file using more command
    Extract data from files in Linux using cut command
    Sort data in files in Linux using sort command
    Sort data in files in Linux using sort command based on key and delimiter
    Extract Information and Redirect to a file in Linux
    Getting Unique Records from File using sort in Linux
    Using uniq to get unique values in Linux

Understanding File and Folder Properties

    Introduction to Properties of Files and Folders
    Understand File and Folder Properties provided by ls command
    Understanding Owner of the Files in Linux
    Overview of groups associated with files and folders in Linux
    Creating Groups in Linux using groupadd
    Create users in Linux using useradd command
    Specifying bash as the default shell while creating users in Linux
    Create Users with Group in Linux
    Add users to supplementary or secondary groups in Linux
    Overview of File Permissions in Linux
    Overview of umask for default permissions on Files and Folders
    Permissions on Home Folders for different Linux Users
    File and Folder Permissions for Groups and Others
    Conclusion for File and Folder Properties using Linux

Managing Files and Folders in Linux

    Introduction to Managing Files and Folders in Linux
    Generate empty files in Linux using touch command
    Creating Directories in Linux using mkdir command
    Overview of awk command in Linux
    Overview of cp command to copy files in Linux
    Using cp command to copy files in Linux
    Create multiple folders at once using awk in Linux
    Copy files using Patterns in Linux
    Automate file copy using awk and cp in linux
    Copying folders recursively using cp command in Linux
    Copying files using cp while preserving properties in Linux
    Moving Files in linux using mv command
    Automate file move using awk and mv in linux
    Overview of tar command to take the backups
    Create backup of files and folders using tar command in linux
    Restoring Files and Folders using tar command in Linux
    Moving Folders in Linux using mv command
    Overview of rm command to remove files from Linux File System
    Delete files from Linux File System using rm command
    Use alias to delete files interactively from Linux File System
    Delete folders from Linux File System using rm command
    Restore Data Folder from the tar ball
    Delete files using find and rm from Linux file system based on name
    Delete files using find and awk from Linux file system based on name
    Generate log files for multiple years using touch with patterns
    Update timestamp of files based on names in Linux using touch
    Delete files using find and rm based on modified time in Linux
    Renaming the files and folders in Linux

Basic System Commands in Linux

    Introduction to Basic System Commands in Linux
    Overview of core components of a computer
    Get CPU Details using lscpu command in Linux
    Get Memory Details using free on Linux
    Get Storage Details using df in Linux
    Get Disk Usage Details using du in Linux
    Get largest folders and files using du and sort on Linux
    Understand Storage Details of Directories using du on Windows
    Get storage use of folders and files
    Get Storage Details of larger files using find and du

Comparing Files and Folders in Linux using diff

    Introduction to comparing files and folders in Linux using diff
    Overview of the diff command
    Prepare Dataset to explore diff command in Linux
    Understand the output of the diff command in Linux
    Compare Files Ignoring Blank Lines and White Spaces using diff in Linux
    Compare Files Ignoring case using diff in Linux
    Unified and Side By Side Comparison using diff in Linux
    Compare Folders in Linux using the diff command

Variables and Environment Variables in Linux

    Creating and Printing Variables using echo in Linux
    Overview of Environment Variables in Linux and env command
    Overview of Environment Variables in Windows
    Review some of the standard environment variables in Linux
    Get the location of commands in linux using which
    Review PATH and unset PATH in Linux
    Relevance of PATH Environment Variable while running Linux Commands
    Overview of Profile and rc in Linux
    Create a simple shell script and grant execute permissions in Linux
    Update PATH with parent folders of custom applications
    Update PATH in profile with parent folders of custom applications

Getting Started with AWS

    Introduction to Getting Started with AWS
    Signing up for AWS
    Overview of AWS Web Console
    Overview of AWS EC2 Console or Dashboard
    Create AWS EC2 Key Pair using AWS EC2 Console
    Create our first AWS EC2 Instance using Ubuntu 20
    Login into AWS EC2 Instance using Key Pair
    Overview of AWS Free Tier and Pricing
    Overview of AWS Web Console Cloud Shell
    Overview of AWS EC2 Instance Concepts
    Managing AWS EC2 Instances

Overview of SSH to interact with remote servers

    Introduction to Overview of SSH to interact with Remote Servers
    Create Multiple AWS EC2 Instances
    Different Modes of SSH Login to Remote Servers
    Understanding Password less login to Remote Server
    Overview of SSH Daemons on Remote Servers
    Understand SSH Properties Files and Properties
    Enabling Password login on Remote Server
    Run Commands on Remote Servers using SSH
    Connect to Remote Servers from Windows using Powershell as well as wsl
    Setup pssh or parallel-ssh on Mac as well as Ubuntu Linux
    Run Commands on multiple servers using parallel-ssh or pssh
    Run Commands on multiple servers using pssh using the hosts file
    Enable Password less login using ssh keys and the ssh-copy-id command
    Terminate AWS EC2 Instances used for demo

Basic Networking Concepts using Linux

    Introduction to Basic Networking Concepts using Linux
    Provision EC2 Instance from AWS
    Overview of AWS Security Groups
    Overview of Different Types of IP Addresses
    Install Apache Webserver on Ubuntu
    Update Security Group to access Apache Web Application
    Overview of Daemon or Background Processes and Ports
    Understanding Structure of IP Address
    Overview of DNS Aliases for IP Addresses
    Install Jupyter Lab on Ubuntu
    Run Jupyter Lab using Localhost
    Update Security Group to access Jupyter Lab
    Setup Mongo DB on AWS EC2 Instance
    Run MongoDB on Universal IP Address
    Update Security Group to access Mongo DB
    Cleanup AWS EC2 Instance used to demo networking concepts

Copy Files and Folders between Multiple Systems using scp

    Introduction to Copying Files and Folders between multiple Linux Systems using scp
    Provision of multiple EC2 Instances from AWS
    Connect to EC2 Instances using Windows Powershell
    Connect to EC2 Instances from Linux on Windows
    Prepare Data Sets to explore Copying Files
    Overview of scp for secure copy between Multiple Systems
    Copy Files using scp between Multiple Systems
    Copy Folders using scp between Multiple Systems
    Copy Tar Ball or Zip File using scp between Linux Servers
    Setup parallel-scp or pscp to copy files to multiple systems
    Prepare Target Systems by recreating folders using pssh
    Copy Files onto remote machines in parallel using pscp
    Copy Folders onto remote machines in parallel using pscp
    Overview of Copying Files and Folders between two systems in parallel
    Submit Long-Running Applications or Commands in the Background using nohup
    Run multiple scp commands in parallel using nohup in Linux
    Stop or Terminate AWS EC2 Instances used for the tasks

Troubleshooting Application issues using find and grep in Linux

    Introduction to Troubleshooting Applicatin Issues using find and grep in Linux
    Review Standard Location for Application Logs on Linux
    Setup Log Files to explore grep and find to troubleshoot the issues
    Overview of Standard Log Message Format
    Quick Review of find command to get log file names based on criteria
    Getting help or official documentation or man page on Linux grep command
    Overview of Standard Input, output and error
    Redirect Linux Command or Program Results to a file
    Overview of Null Device in Linux
    Redirect stanard output and error to files and null device in Linux
    Overview of Case Sensitivity in Linux
    Search for error lines from files in Linux using find and grep
    Get File Names with errors using grep and find in Linux
    Perform Whole Word Search using Grep Command
    Search for multiple patterns in the files using find and grep in Linux
    Get number of lines with pattern using find and grep in linux
    Get line number and output line using grep in Linux
    Get additional lines along with error or exception lines using grep in Linux

Monitoring and Managing Linux Servers using System Commands

    Introduction to Monitoring and Managing Linux Servers using System Commands
    Setup EC2 Instance on AWS using Ubuntu 2
    Overview of Managing Softwares on Ubuntu using apt
    Install Apache Web Server and Mongodb on Ubuntu
    Manage Applications and Databases using systemctl or service command
    Start Apache Web Server and Mongodb using systemctl
    Overview of Service and its components
    Recap of commands to get Linux Server Details
    Get CPU or Load Average of Linux Server using uptime command
    Overview of ps command to get details about processes on Linux Server
    Get details of all the processes running on Linux Server using ps
    Get list of processes owned by user on Linux Server using ps
    Customize the output of ps command for specific details of processes
    Number of processes on Linux Server owned by each user using ps
    Get Process Details for a service on Linux Server using ps command
    Overview of lsof command to get list of open files on Linux
    Get open file details using lsof for a given process id
    Get open file details using lsof for a given port number
    Get Process Details using lsof command for a given file
    Troubleshoot Port Usage on Linux using lsof command
    Overview of top command in Linux to get system performance details
    Getting help or man page of top command in Linux
    Using Command Line Options of top command in LInux
    Customize top interactive interface in Linux
    Switching between groups in top interactive interface in Linux
    Kill Processes using top interactive interface in Linux
    Terminate AWS EC2 Instance setup to practice Linux system management commands

Overview of Developer Tools for Linux using Windows

    Introduction to Developer Tools for Linux on Windows
    Install Notepad++ on Windows
    Install WinScp on Windows to access files on remote Linux Servers or Sites
    Add Site to WinScp for Password Login into Remote Linux Server
    Add Site to WinScp for Passwordless Login into Remote Linux Server
    Copying Files between Windows and Remote Linux Systems using WinScp
    Opening Files on Remote Machines using WinScp and Notepad++ Locally on Windows
    Overview of Advanced Editing Capabilities of Notepad++
    Overview of Visual Studio Code on Windows for Remote Development
    Install Visual Studio Code on Windows
    Install Remote Development Extension Kit for Visual Studio Code
    Setup Project on remote machine for remote development using Visual Studio Code
    Open Remote Project via SSH using Visual Studio Code
    Open Project via WSL using Visual Studio Code
    Access Configuration Files on Remote Machines using Visual Studio Code
    Install Git and Git Bash on Windows
    Install Beyond Compare on Windows
    Setup Repository using Git to explore Beyond Compare
    Quick Demo of Beyond Compare to compare code base

Overview of Ansible for Server Automation

    Introduction to Overview of Ansible for Server Automation
    Setup Ansible on Ubuntu 20
    Provision EC2 Instances to learn Ansible
    Run first command using Ansible on Multiple Instances using inline inventory
    Run first command using Ansible on Multiple Instances using inventory file
    Update Ansible Hosts Files on AWS EC2 Reboots
    Overview of Ansible help, man page and official documentation
    Add User and Group Manually in Linux
    Add Group on remote instances using ansible
    Adding User using Ansible user module on remote servers
    Add Folder on remote systems using Ansible file module
    Copy authorized keys using Ansible copy module
    Setup Project Structure for Ansible Playbooks using Visual Studio Code
    Create and Run first ansible playbook
    Define Variables for Credentials to run Ansible Playbooks
    Develop Ansible Playbook to Mange Users
    Getting Started with Developing Ansible Roles
    Develop and Validate Ansible Playbook with Role
    Adding more tasks to the Ansible Role and Troubleshooting the issues
    Create Ansible Playbook with Variables
    Using Lists and Loops in Ansible Playbooks
    Using Tags to run specific tasks from Playbooks
    Update Ansible Role and Playbook with variables and tags
    Add Variables under vars folder of Ansible Role
    Terminate AWS EC2 Instances used for Ansible Tasks for Server Automation
    Conclusion of Overview of Server Automation using Ansible

Overview of Automation using Python on Linux

    Introduction to Overview of Automation using Python on Linux
    Validate Python and Python Modules such as pip on Ubuntu
    Setup and Validate Jupyter Lab on Ubuntu
    Create Folder for Notebooks using Jupyter Lab
    Overview of getpass module to get username
    Running Linux Shell Commands from Jupyter Notebooks
    Overview of Python os module
    Setup Data Sets for Python and Linux Integration
    Overview of Python glob module
    Overview of running Linux Commands using subprocess
    Sort all files in Linux File System by size using Python
    Uncompress all Compressed Files using Python on Linux
    Overview of Python Multiprocessing Module
    Uncompress Files in Parallel using Python multiprocessing
    Add Extension to Files using Python
    Overview of Copying Files using Python and scp
    Run Commands on Remote Machines using Python and ssh
    Copy Files to multiple systems in parallel using Python and scp

Desired Audience

Here is the desired audience for this advanced course.

    Experienced application developers to gain expertise related to Linux shell commands
    Experienced Data Engineers to gain enough skills related to Linux shell commands
    Testers to improve their testing capabilities by using Linux shell commands effectively

Prerequisites

    Logistics
        Computer with decent configuration (At least 4 GB RAM, however 8 GB is highly desired)
        Dual Core is required and Quad-Core is highly desired
        Chrome Browser
        High-Speed Internet

Who this course is for:

    Students who want to learn Linux commands by hands-on practice
    Freshers and entry level IT professionals who want to gain the knowledge of Linux commands
    Any Software developers such as Python, Java, etc who would like to gain proficiency of Linux Commands
    Any Software Testers who would like to gain proficiency related to Linux Commands
    This course is designed for all those IT Professionals who frequently work on Linux based Environment
    System Administrators, Network Administrators, DevOps Engineers, etc

Requirements

    PC or Mac with decent configuration
    At least 8 GB RAM, i5 or equivalent processor
    CPU with dual core or with higher configuration

Last Updated 6/2022