Download web presentation (multiple files) using wget

In my case, the page started with 1, 2, 3, ..... to 350.  I can download a page one at a time. A lazy person like me, could not do that. So, I just use a for loop to download these files. Presentation was not available to download and all pages were images. 

Login to your Unix system and use for loop at your prompt.

[sam@master RI]$ for((i=1;i<=350;i++)); do wget https://meropreseation.com/ebooks/travel-E-Book/content/pages/page$i.jpg; done

Downloading files to windows system from Unix system.

PS C:\Users\Sam\RI> scp sam@192.168.100.100:/home/sam/RI/* .

RHCSA - Linux partition

 

Disk Management With Parted Command In Linux:
=============================================

MBR partition scheme and GPT partition scheme both use different method to store partition information in disk. We cannot use both scheme in a single disk.
Since in this tutorial we will create both types of partitions, we need two separate disks. We have already added two virtual disks in our system.

# lsblk

OR

# fdisk –l

Creating MBR Partition with parted command:

The parted command needs the name of disk where we want to create the partition as its argument.

# parted

Always use parted command with argument. Without argument it will start with first available disk.

# parted /dev/sdb

print

The print command provides important information about the hard disk. This information includes manufacturer information, disk model, disk size, Partition Table and disk flag. If this is a new disk, we will get unrecognised disk label error. Disk label is the information which is used to determine the type of disk. The parted command needs a valid disk label before it can do anything with disk.
To see it practically run mkpart command. The mkpart command is used to create new partition.

mkpart

As above output confirmations that we are not allowed to create new partition unless it has a valid disk label. The msdos and gpt are two valid disk labels which we can use here. The msdos label is used to define the disk as MBR disk and the gpt label is used to define the disk as GPT disk.

To assign the label mklabel command is used. Let’s assign the msdos label to this disk.

Type mklabel and press Enter key.
Type msdos and press Enter key.
Verify the action with print command.

# parted /dev/sdb

mklabel

msdos

print

Do not use this command if disk contains data. This command is very risky. This command will convert disk in un-partitioned blank disk silently. 

Now disk has label. We can create partition in it. To create partition following steps are used.

Type mkpart and press Enter key.

As we know in MBR partition scheme we are allowed to create maximum four primary partitions. If more partitions are required, we have to use one primary partition as extended partition. Further Extended partition can be used to create 15 logical partitions.

Type the partition type and press Enter key.

Type the starting point for partition and press Enter key.

The parted utility supports very limited file system types. To add worse, most of them are out dated nowadays. We will make file system individually later. By now just keep the default file system type.

Type the ending point for partition. We can use KB, MB or GB units to define the size of partition.

We can verify the creation of partition with print command

Following figure illustrates above steps

mkpart

primary

ext2

1M

600M

print

Always start first partition with 1MB even only first 512 bytes are used to store the MBR data. This way we can avoid several entry point errors.

Following same process create two more partitions

mkpart

primary

ext2

601

800

mkpart

primary

801

1000

print

We have created three primary partitions from maximum (four) allowed. Now we can create one more primary partition or can create one extended partition.

Let’s create one allowed extended partition. Since extended partition is used to create all allowed logical partitions, we should allocate the maximum disk space in this partition. We will assign all remaining disk space in this partition.

mkpart

extended

1001

2100

print

All logical partitions will be created in extended partition. The sum (of size) of all logical partitions cannot go beyond the extended partition. In this example we created an extended partition of 1145MB, so we can create logical partitions from 1145MB. We are allowed to create up to 15 logical partitions in extended partition.
We can create any number of partitions from range of one to fifteen. The only limit is that the sum of all partitions must be below the size of extended partition.

Let’s create a single logical partition of all available space.

mkpart

logical

ext2

1002

2045

print

We have used all space of extended partition in one logical partition. So even we are allowed to create up to 15 logical partitions and free space is available outside the extended partition, we cannot create the another logical partition. To create another logical partition we need free space inside the extended partition not outside the partition.

So far we have created all possible partitions including primary, extended and logical partitions.

To exit from parted use quit command.

quit

The parted cannot update the in-memory kernel partition table. Kernel reads partition table when system boots. So the kernel will be updated about this change at next reboot automatically. Meanwhile if we want to update the kernel immediately, we can use partprobe command which force kernel to reread the partition table.

# partprobe /dev/sdb

# lsblk

# fdisk -l /dev/sdb

Creating GPT partition with parted command:
GPT is the new partition scheme. It allows us to create up to 128 partitions. It does not divide partitions in primary, extended and logical. First part of this article explains basic concepts of GPT in detail with examples.

We will create GPT partition in /dev/sdb disk. Let’s initiate parted again for /dev/sdb disk

# parted /dev/sdc

print

mkpart

mklabel

gpt

Now disk has a valid label. We can create partition in it. Creating partition is a six steps process.

Type mkpart and press Enter key
Set desired descriptive name for partition
Accept default file system type
Enter starting point of partition
Enter ending point of partition
Verify the partition with print command

mkpart

linux

ext2

1M

600M

print

mkpart

documents

601

1000

print

quit

# partprobe /dev/sdc

lsblk

So far we have created MBR and GPT partitions in /dev/sdb and /dev/sdc respectively.

These partitions are useless unless we put a file system in them. File system is a logical container which allows us to store files and directories.

How to create file system
To create a file system in partition we format it with supported file system. To format a partition following command is used.

# mkfs.ext3 /dev/sdb1

# mkfs.ext4 /dev/sdb2

# mkdir /srv/test1 ; mkdir /srv/test2

# mount /dev/sdb1 /srv/test1

# mount /dev/sdc1 /srv/test2

# df -hT

Make entries in /etc/fstab for permanent mounting.

# cd /srv/test1

# touch {1..10}

# ls -l

For LVM:

# pvcreate /dev/sdc1

# pvs

# vgcreate LinuxVG /dev/sdc1

# vgs

# lvcreate -L 500M -n LinuxLV LinuxVG

# lvs

# mkfs.xfs /dev/LinuxLV/LinuxVG

# mkdir /srv/lvm

# mount /dev/LinuxLV/LinuxVG /srv/lvm

# df -hT

# cd /srv/lvm

# touch {a..m}

# ls -l

How to delete MBR partitions
To delete a partition rm sub-command is used. Before we delete a partition it must be un-mounted. We cannot delete a mounted partition. Let’s see it practically.

# umount /srv/test*

# df -hT

# parted /dev/sdb

print

rm 1

rm 2

rm 3

rm 5

print

rm 4

print

quit

# partprobe

Delete GPT partitions
Deleting a GPT partition is as same as deleting a MBR partition. Initiate parted command with GPT disk (/dev/sdb) and use print command to view the number of partition and use rm sub command to delete that partition.

Day2 - Docker - container, volume, start container

Day 2 - Introduction to Docker
Day2 - Docker - container, volume, start container
Can I start/stop the container? How about image?
- You can start/stop container and remove container but you can not stop/start image. You can pull image and remove the image.
How to run a container?
$ docker run -dt centos bash # Runs on deatach mode
here, you got a template called centos and created one container.
The container has started.
Check the running containers
$ docker ps
Now, how do you login to container?
$ docker ps
$ docker exec -it <container name/ID> bash
here bash is the working shell
$ docker exec -it centos bash
Now, create a new session of host machine and stop the container
$ docker container stop <container_ID> ## run docker ps to find the container id
Create a new container
$ docker run --rm -it contos bash
Note:
docker ps - shows you container list that are running
docker ps -a -> shows you list of all container, in fact history

How to remove a container
$ docker ps # get the contiainer id
$ docket container rm element_1
$ docker ps -a $ list all available container including stop ones
$ docket run --rm -it --name <name of container>
$ docker run --rm -it --name first_cont ubuntu bash
Now, you are at the container prompt. run hostname, you will see some id.
if you type exit or quit at the prompt, you will be out of container and container will stop as well.
on the host machine, type 
$ docker ps # you will see the containers running
$ docker ps -a # list all containers including stopped one.

Lets login to continer
$ docket run --rm -it --name first_cont --hostname ubuntu ubuntu bash
and type hostname' cat /etc/os-release at the prompt.
Note: Downloaded images are read only images. 
container runs on a concept called copy on write (COW)
copy on write mean, you are adding content on the container.
at the container prompt, type 
# apt-get update   -> it will go and update the container
so, you can write to your container but you can't update the image. (readonly)
At the prompt, run
$ apt-get install apache2 # we are installing apache
Note: if you are using interactive mode and you exit out of the container prompt, container will be removed.
$ docker images ls # list container images
$ docker image rm ubuntu
$ docker pull ubuntu
go to the URL hub.docker.com
at the search box, type ubuntu and click on ubuntu official
- click on docker file.
- read everything on the file and try to understand it.
- This file is the foundation of the image.
also search for nginx and read the dockerfile for the image.
so, images are build based on dockerfile.
[ Remember: Copy on write filessytem]
so what happens when you run docker pull command?
- pull command first goes to the hub.docker.com and checks the dockerfile and grabs all the things stated on the file. Once pull is complated, you will have particular image. These images are stored in local repository. The default location is /var/lib/docker
Things to remember,
- Images are read only but container that are based on images can be modified.
- Docker containers are volatile
  - if you store data in a container, and you remove the container, all the data is lost. so containers are usually immutable (ephenreal - temporary or disposible)
Once you remove the container, data on the containers are gone.
$ docker run -it --name first-cont --hostname ubuntu ubuntu bash
# echo "Hello" > hello.txt
# cat hello.txt
# exit
$ docker ps # container is gone/ stopped
$ docker ps -a
Start the stopped container
$ docker ps -a 
$ docker ps
$ docker container start first_cont  # first_cont is started
Now, you have to login
$ docker exec -it first_cont bash
# cat hello.txt # you may still be able to see it.
# exit
$ docker ps
$ docker ps -a
$ docker container rm -f first_cont
$ docker ps ; docker ps -a # now, its gone. all data on the container is gone.
so, containers are temporary, and data is also temporary. So, you can say containers are volatile.
To resolve this issue what we can do is,
1. create a persistence volume
2. Bind mounts
What is the benefit of working with volumes,
1. To keep the data around when container is removed
2. To share data between host and container
why should we use container?
- It is a light weight OS (about 200mb) compare to virtual machine(appr 4GB).
- Only one app is install and it only contains some library and binaries.
- volume helps you to keep data even you remove the container
what happens if you wan to upgrade?
you can still upgrade without any issue.
So, what is the benefit of using volume?
we can say that
1. A data volume is a specially designed directory in the container
2. It is initialized when the container is created. By default, it is not deleted when the container is removed
3. Data volume can be shared across the containers too.
4. Volumes are stored on host machines.
5. You can name particular volume with meaningful name
How to create a volume
$ docker volume ls # lists the volumes
$ docker container run -it ---name secon_con -v /data ubuntu bash
-v -> name of the volume
/ -> mount at root of container
-it -> interactive.
Open new session of the host machine
$ docker volume ls
$ docker inspect <container_ID> second
inspect - what to inspect
the above command gives detail about container such as ip, gateway, id, when it is started, 
image name, volume name, volume id and other info
review the source and the path that start with
/var/lib/docker/volumes/<id>/_data
$ docker volume ls # verify the id, you will find these two same.
$ ls /var/lib/docker/volumes/...../_data
permission denied
$ sudo ls /var/lib/docker/volumes/.../_data
Login to docker and create a file
# cd /data; echo "Hello" >hello.txt
# exit
$ docker ps
$ docker container rm second
$ docker ps -a # everything is done about second container
$ docker volume ls # you will still see the volume
Where the volume is pointing to?
$ docker volume inspect <volume_ID>
$ sudo ls -l /var/lib/docker/volume/<id>/_data
you will still see the data.
if you want to use this volume, you have to specify that this is the volume you want to use when you start a container.

so the question is:
How to start a container with the same volume?