Docker

Docker Container is a single bundle/ unit where we have packed our Application and Dependencies into a single unit. And this is shared with other developer systems and deployed to server.

so replicating code from one system to other becomes simple.
We can share our bundle from machine A to machine B without installing dependencies.

We can replicate our one project on one system to other. It is for local environment.

Docker container: 
It includes [app + dependencies] that share it with other machines.
It is portable - Container as unit can be shared across teams.

Lightweight in nature - it is simple to create, update and delete.
Less size like VM.

We can create 2 apps with different versions of same language in same system and share it with others.

Docker image: 

It is an executable file where instruction is defined to build the containers. So, Docker Image Builds Container. Their relation is same as Class and Objects. So docker image can create multiple container like class can create multiple objects.
So, Docker Image is blueprint that helps to create multiple containers.

So we share image with team members not the container. Class don't share memory but object takes memory. So image is smaller in size and container has running environment. So resources are used by container.
Docker Container is an actual running instance whereas a Docker Image is a static screenshot/ snapshot of what a local development environment should look like.

Creating a Docker environment in MacOS

Consider we have a macOS machine as host and want to run Ubuntu  environment in MacOS and want this Ubuntu environment to be completely isolated from MacOS.
so to install this we run the command : docker run -it ubuntu 
means we will run ubuntu container in Docker or in our machine in interactive mode.

It is available in our current system then it takes from it else it takes it from docker/hub .

After installing we get a container that can be seen in docker desktop and this can be checked in terminal.

This container is different from macOS and container can be deleted that will not affect other macOS.

The Linux specific things can be built in ubuntu container.

Docker is similar to Virtual machine and is not the same but it is lightweight.

Setup Docker in our local system
We can download the docker desktop and install it in pc.

to check docker installation on our local system write docker in cmd/ powershell or terminal.

Docker hub
it is where we can deploy our docker image files.

with every image there is different instruction.
to get image from docker hub we use docker commands.

docker pull IMAGE_NAME
fetches the latest image.
If this image not exist on local then it will pull it from docker hub.

Each docker image in docker hub has a tagname - version, imageID, created data, size.

tag is like a version or variant of same docker image.
After that we create container from that image.

docker run IMAGE_NAME

-it is interactive mode that will help us to run some commands in the container environment e.g. we have pulled the image of ubuntu and running the commands in it. It helps us to access container terminal which helps us in input output anything. In ubuntu it gives us command line tool for ubuntu.
docker run -it ubuntu

exit command in the above terminal exits the container and stops at the docker hub.

The running containers can be seen in docker hub.

List all Local containers (running & stopped)
docker ps -a 

List the running containers
docker ps

Start or Stop an existing container 

docker start|stop <container_name>   (or <container_id) 

List of docker images
docker images

Deleting a image

docker rmi image_name  (not continer_id)

Deleting a container

docker rm <container_name>   (or <container_id) 

Note before deleting an image delete the container first else it will give conflict.

Versions in docker images

docker pull mySql  =>  pulls the latest image

docker pull mySql:8.0 => pulls specific version

During the pull command it pulls layers of image, common layers are not pulled again where it was pulled in previous versions.

docker run -d Image_name
means a container can be run in the background i.e. detach mode.
All the containers by default run in attach mode.

There are some environment variables that we can declare which containers require using -e

docker run -d -e mysql_root_password=secret mysql

docker run -d -e mysql_root_password=secret mysql-older mysql:8.0 (it is with new name)

Give custom name to container name

Run container with custom name 

docker run --name <container_name> -d <image_name>

Docker Image Layers

Docker image layer is made up of different layers. Imagine it is as a collection of different layers.

BaseLayer - is generally a small size Linux base image. e.g. Debian, alpine etc. 
Above this we can have multiple layers and at top we have container layer.

only contaioner lauyer is editable and rest layers are read only.

consider if we are working on 2 images from differtnet versions e.g. img1, img2 then when installing we get already exists layer info. Rest layers will be downloaded.

Port Binding

When a containter is created so by default all docker container have a port which is bind to them. All docker container have a separate virtual file system separated from host machiine and component ports are separated from host machine.

Every container has a port that can be seen in command: docker ps 
We can also bind the host machine port and container port so that request forwarded from port host machine to container.

docker run -p8080:3306 IMAGE_NAME

docker run -p<hostport>:<containerport> image_name

Once a host machine port is bind to container then it cannot be bind to another container. giving error port already allocated.

So to bind another container with the host machine we need another port of the host machine.

So different container needs different ports of host machine.

If we get an error container name already in use by container 'dfdfsdfds'.

So use code stop and rm to remove the container.

then again bind the container.

Troubleshoot commands

docker logs container_name = this is to log the container

docker exec -it CONT_ID bin/bash

it allows us to run additional commands on an already running container.

If bash is not available then use 'sh'

docker exec -it CONT_ID bin/sh

Above -it command opens the interactive terminal.

If we exit from last command then it don't stop docker container. Container is still running but we got out of interactive bash.

Docker vs Virtual Machine

A system has 3 parts:

1. At the base: Hardware
2. In the middle: Host OS Kernel

3. At top: Application Layer

In case of Docker

Whichever machine Docker is installed, it uses that OS host kernel and only virtualize the Application layer.

In case of VM

It virtualizes both host OS kernel and application layer.

So docker is lightweight due to less overhead and makes docker faster and smaller in size.
And VM size can go up to GBs.

Disadvantage of Docker:

VM is compatible with all OS and because it uses host Kernel.

Docker was initially built for Linux OS and docker images exists for Linux OS. So to run on mac and windows we've docker desktop. Because it adds a lightweight hypervisor layer to our system, which internally uses a lightweight Linux distribution, with which we can run our containers on Non Linux systems.

Developing with Docker

Docker network

If in our host system when we've containers where we've containers and it is true that containers have their own ports. e.g. Mongo and mongo-express have their own ports to interact with each other.
So to interact with each other the need other port for communication.

The containers can interact with each other without any port or limitations using docker network.

Docker has the ability to create these isolated networks where if we have to setup 2 containers so these containers will interact without any port or localhost or anything else.

docker network ls   =  lists all the docker networks
docker network create network_name  =  creating a network with a name 

First we create a single network.

Then we setup both our container inside this common network. So that they interact with each other without any port.
Let's setup mongo and mongo-express. We will do setup in detach mode. Since we don't have any container on our system so we don't need to pull image but instead we will run it, it will auto pull a container.

With mongo we need additional params like -d, bind port 27017 of container  i.e. mongo with our host port. Default port for mongo is 27017 (-p) and this mongo container will run inside our isolated created network (--network) specifying particular network and with defining mongo network and environment variables is required for mongo.

Another important for mongo ios username and password by using (-e) i.e.
root_username: admin and root_password: qwerty

In windows:

use ^ instead of \ in CMD while running commands in command prompt

in Linux and macOS \ is working. example code below

C:\Windows\System32>docker run -d ^

More? -p27017:27017 ^

More? --name mongo ^

More? --network mongo-network ^

More? -e MONGO_INITDB_ROOT_USERNAME=admin ^

More? -e MONGO_INITDB_ROOT_PASSWORD=qwerty ^

More? mongo

--name mongo means we are giving container our name and could be anything.

Sometimes network gets lost or deleted by the system in that case again run the command to create network and run mongo.

C:\Windows\System32>docker network create mongo-network

Error response from daemon: network with name mongo-network already exists

C:\Windows\System32>docker start mongo

mongo

Then  import mongo express

C:\Windows\System32>docker run -d ^

More? -p8081:8081 ^

More? --name mongo-express ^

More? --network mongo-network ^

More? -e ME_CONFIG_MONGODB_ADMINUSERNAME=admin ^

More? -e ME_CONFIG_MONGODB_ADMINPASSWORD=qwerty ^

More? -e ME_CONFIG_MONGODB_URL="mongodb://admin:qwerty@mongo:27017" ^

More? mongo-express

Note both the container must have network name defined to it.

And to check if mongo-express is running then we have to open the localhost:8081 that must open with no error and it's id - password will be different and is standard i.e. admin and pass .

In the webpage we will be able to add and edit the database etc.

Then we will open the web app by running node server.js that will automatically connect to mongo db that was there in docker without actually installing the monger db. this type of setup is applicable to all the languages in the world that has image on docker.

Docker Compose

Docker compose is a tool for defining and running multi-container applications.

This eliminates the need for writing multiple line of codes.

Like we create a container with the help of  multiple commands which contains setup and environment variables, here we can create a single file which includes docker commands which is YAML file. it is another markup language.

To use docker compose we will create .yaml file then using this file to run containers we will use Compose.

We create a compose yaml file as 
Step1: anyname.yaml

Step2: All containers will be defined in single tag services:

We have 2 commands in compose i.e. up and down
up - means we want to create container 

down - to delete the containers permanently

docker compose -f filename.yaml up -d

docker compose -f filename.yaml down

While writing docker compose we don't define docker-network to put our containers in the same network instead docker creates a single default network and put the containers in that network by default.

When we stop the container then old data will be deleted and there is no data persistence. For this we have Docker Volumes.

Dockerizing our App

The most common use of Docker is dockerising our own app.
that will convert our App to Docker Image then convert to Container. And this is done so that we can share this image with other team members and it can also be used for deployment. Here we can convert our app into Container.

But generally in production environment, a tool called Jenkins in CI/CD does that work for us. So Jenkins convert that app into a docker image, then this image can be published on private or public repo.

To dockize our app we use a special file called docker file. It is a blueprint, how our docker image or docker container will be built. So docker file has instructions how our app will be dockerize.
Below are some important keywords.

FROM

Every docker image that we want to create is based on Base Image

syntax: FROM Base_Image

e.g. if we have a nodejs app that will only be run on system when node is installed

So to build the container we must have this node image because our app runs on node and on above this we will be able to build our app. this is how layers are created step by step.

WORKDIR

where we define our working directory name. And this is the directory where the path in the image where files will be copied and commands will be executed.

So commands like COPY, RUN, CMD, EXPOSE, ENV will be declared here.

COPY

It is used to copy data from host to image where we define our host and image path.

RUN

It used to run the instructions e.g. installing

It can have multiple commands

CMD

We can have only one run command

It will be responsible to run the container when our container is completely built from image.

EXPOSE

we expose the port of the image.

ENV

To define environment variables.

Like our app is built on node that will be included in dockerfile and it will be a layer there and this node is already build on some debian based image means images are created on other image. This is layering when one image is dependent on other layer or created from other image.

run mkdir -p testapp = means we will create a new directory testapp into the container that was created from the image.

now for the container we are in the root folder where we created a new folder testapp and within this testapp we will COPY all the files and folder into here. 

testapp is copied from the testapp.

COPY . /testapp

CMD will have path of server.js inside testapp. We write command in the form of array of strings.

CMD ["node", "server.js"]

If we are in the testapp folder of the project then we can run node server.js 
but
in case of docker container we would be inside root folder. root does not contain any server.js instead testapp folder copied in the container.

CMD ["node", "/testapp/server.js"]

to build docker image from this file we use docker build -t testapp:1.0

testapp is the image name we want to define.

To run the app: 

docker build -t testapp:1.0 .  (note . with space before)

-t is tag. 

1.0 . is the version defined.

To run and build a container use the commad

docker run testapp:1.0

FROM node

WORKDIR /testapp

# WORKDIR is must to deploy on localhost

# EXPOSE 5050 is not that much required

ENV MONGO_DB_USERNAME=admin \

    MONGO_DB_PWD=qwerty

RUN mkdir -p testapp

COPY . /testapp

CMD ["node", "/testapp/server.js"]

Logging in and logout from docker

In the CMD use the below commands

docker logout

and 

docker login
login will give a link and code. follow the link and paste the code to login.

We will push our code to Docker hub

docker tag testapp:1.0 sunilk5/testapp:1.0

docker push sunilk5/testapp:1.0

push command has changed over time

Whenever we need to pull the image we will use this container using the command

docker pull sunilk5/testapp:1.0

Docker Volumes

Volumes are persistent data stores for containers. We've an extra space which we can reserve on our host system which is either managed by host system or mostly by docker.
And we mount this Volume space on our container. Here we created a connection of our container. Here we created a connection of our container with volume. So whatever data that is created/ store in container will be replicated in this volume. If we stop this mongo db container and if we want to restart so the data is fetched from the volume and when it restarts it can access the old data. 

Even if we completely delete the container we've the access of volumes. If we've multiple containers we can also connect this volume with it.

-v defines which volume gets attached to which container directory.

We must sign in with the same docker account that we sign up on the docker hub.

The command that we use is 

-v host_path:container_path

host_path is current system i.e. user pc

container_path is the path of folder within the container.

We are creating here an ubuntu container.

if we exit from this ubuntu container then it's data will delete but data will persist inside the volume and when exited and comes back then the data that used to be deleted will come back this is when we don't have volumes. 
We exit the container using exit command.

Let's again start the container

check container with : docker ps

D:\docker>docker start f02c36f553c979208d59811b28545de61f1facc9f442b82bbef92a6a999c9d37

f0... is the id of the container

then call the below codes

in the ls above command it is seen that we have access to index.html and server.js that should be deleted when container exited but it refetched when we have volumes on our hot system.

Usage of docker volumes when we are trying to work with docker compose

We can define a location to create a volume in our mongo.

services:

  mongo:

    image: mongo

    ports:

    - 27017:27017

    environment:

      MONGO_INITDB_ROOT_USERNAME: admin

      MONGO_INITDB_ROOT_PASSWORD: qwerty

    volumes:

    - D:\docker\DbVolume:/data/db

# /data/db is the default mongo db provided as on mongo image

This is done using docker compose yaml compose commands.

And if we open 8081 port then it shows our database existing there.

Docker Volumes commands

docker volume ls = lists all volumes

docker volume create named_volumes = create named volumes

docker volume rm volume_name = remove volume

The named volumes are present on the location in OS below

these volume are not attached with anyone.

then how can we connect with some container. 
it is using below codes.

--volume flag or -v is used to connect the volume with running container and

--mount flag is used for the same almost similar feature for both of them.

Named volumes is the preferred way to do this.

MOUNT_PATH  is the container directory. this is used for temporary storage.

Named and anonymous volumes are created and managed by docker.

docker volume prune => this command is used to remove the anonymous unused volumes.

More Docker Network Concepts

Whenever we create a network we are provided with networkid, name, driver and scope.

Driver are basically most important helpers that defines over a network any system will send or receive messages.

Drivers are of 3 types: BRIDGE, HOST and NULL

By default whatever network we create is a BRIDGE type of driver.

So when a container is created it connects to a driver called bridge driver and many containers can connect to it and it is helpful in intercommunication of containers and to the ourside world.

There are 2 types of BRIDGE networks 

1. Default

2. CUSTOM (created by us)

HOST N/W

The container we created has no ip address and is acts like a host machine.

NULL N/W

created an isolated network.

It is isolated from host and other containers.