In this we will do monitoring using Grafana and Prometheus of our 2-Tier-Flask-App-and-MYSQL Project that we did earlier. If you're interested in exploring my 2-Tier Flask App with MySQL project, to check out my project : Click here.

Prerequisites

Before starting the project you should have these things in your system :-

Account on AWS

The application should be running

Part 1 : Install Prometheus

Prometheus we use for metrics & alerting. Now make sure docker.io & docker-compose is installed. Now to configure Prometheus & cAdvisor use command as given below [cAdvisor (short for container Advisor) analyzes and exposes resource usage and performance data from running containers. cAdvisor exposes Prometheus metrics out of the box] :-

sudo apt-get update
wget https://raw.githubusercontent.com/prometheus/prometheus/main/documentation/examples/prometheus.yml

Now to see prometheus.yml use command :-

ls

Now we will make a docker-container.yml to run prometheus, redis & cAdvisor for this write a docker-compose file and run command docker-compose up -d.
Now if we will do docker ps we will see our prometheus, cadvisor & redis container is running.

Now we saw prometheus container is running. So copy and paste Public IPv4 address:9090 in a new tab and you will see :-

Now we saw cadvisor container is running. So copy and paste Public IPv4 address:8080 in a new tab and you will see :-

Now we want docker's log and our app container is already added in cAdvisor. So add cAdvisor code in prometheus.yml as given below :-

  - job_name: "docker"
    static_configs:
      - targets: ["cAdvisor:8080"]

Now once we have to restart prometheus container so that prometheus.yml will also update inside our prometheous container. To restart use command docker restart container_id.
Now if we do refresh prometheus page, we will see docker up.

Now if we want see cpu load average in graph, so Goto Graph → Write → rate(container_cpu_load_average_10s{name="node-app"}[5m]).

Part 2 : Install Grafana and set it up to Work with Prometheus

Grafana we use for visualization. Now to install grafana on Ubuntu, use command :-

sudo apt-get update
sudo apt-get install -y apt-transport-https
sudo apt-get install -y software-properties-common wget
sudo wget -q -O /usr/share/keyrings/grafana.key https://apt.grafana.com/gpg.key
echo "deb [signed-by=/usr/share/keyrings/grafana.key] https://apt.grafana.com stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list

Update the package list and install Grafana, use command :-

sudo apt-get update
sudo apt-get install grafana

To enable and Start Grafana Service, use command :-

sudo systemctl enable grafana-server

sudo systemctl start grafana-server

To check Grafana Status, use command :-

sudo systemctl status grafana-server

Now copy and paste Public IPv4 address:3000 in a new tab and you will see grafana interface :-

Now to login grafana we don't know username & password, so initial username and password for Grafana are username=admin & password=admin.

Now to visualize Prometheus metrics in Grafana, we have to add prometheus(data source) in grafana. So Goto Connection → Search & Click on "Prometheus" → Click "Add new data source" → In Connection Paste Prometheus URL → Save & Test.

Prometheus Dashboard

Now we will make a dashboard to make it easier to view metrics, you can follow these steps :-
Click "Dashboard" → Click "Add visualization" → Select "Prometheus" → Select Metric "container_memory_usage_bytes" → Select Name of container "ubuntu_flaskapp_1" → Click "Run queries" → We will a visualization dashboard of our container memory.

Same we can see other metrics also like errors of app container.

We've successfully installed and set up Grafana to work with Prometheus for monitoring and visualization.

Node Exporter

Node Exporter we use for to scrape detailed servers/systems metrics. Now we want node exporter in the network of prometheus,so add given code in the prometheus's docker-compose file :-

node-exporter:
    image: prom/node-exporter:latest
    container_name: node-exporter
    restart: unless-stopped
    volumes:
      - /proc:/host/proc:ro
      - /sys:/host/sys:ro
      - /:/rootfs:ro
    command:
      - '--path.procfs=/host/proc'
      - '--path.rootfs=/rootfs'
      - '--path.sysfs=/host/sys'
      - '--collector.filesystem.mount-points-exclude=^/(sys|proc|dev|host|etc)($$|/)'
    expose:
      - 9100

Now run command docker-compose up -d and if we will do docker ps we will see our node-exporter container is running.

Now to configure Prometheus to scrape metrics from Node Exporter, we need to modify the prometheus.yml file so :-

vim prometheus.yml

And in the prometheus.yml file add as given below :-

  - job_name: "NodeExporter"
    static_configs:
      - targets: ["node-exporter:9100"]

Now once restart prometheus container.
Now if we do refresh prometheus page, we will see node-exporter up.

Now Prometheus is already added in our connection in Grafana. So to view node metrics, we can import a pre-configured dashboard follow these steps :-
Click on Dashboard → Click "Import" → Paste id (e.g., code 1860) → Click "Load" → Select data source "Prometheus" → Click "Import".
Now we will see a Grafana dashboard set up to visualize metrics from Prometheus.

Our Monitoring Project is completed 😄.

🔗 GitHub Repository :- https://github.com/sudhajobs0107/Monitoring-2-Tier.git obs0107/DevSecOps_ToDo_App

𝑴𝒐𝒏𝒊𝒕𝒐𝒓𝒊𝒏𝒈 𝒂 2-𝑻𝒊𝒆𝒓 𝑭𝒍𝒂𝒔𝒌 𝑨𝒑𝒑 𝒘𝒊𝒕𝒉 𝑴𝒚𝑺𝑸𝑳 :- 𝑨 𝑭𝒖𝒍𝒍 𝑮𝒖𝒊𝒅𝒆 😄

In this we will do monitoring using Grafana and Prometheus of our 2-Tier-Flask-App-and-MYSQL Project that we did earlier. If you're interested in exploring my 2-Tier Flask App with MySQL project, to check out my project : Click here.

Prerequisites

Before starting the project you should have these things in your system :-

Account on AWS

The application should be running

Part 1 : Install Prometheus

Now to see prometheus.yml use command :-

Now we will make a docker-container.yml to run prometheus, redis & cAdvisor for this write a docker-compose file and run command docker-compose up -d.

Now if we will do docker ps we will see our prometheus, cadvisor & redis container is running.

Now we saw prometheus container is running. So copy and paste Public IPv4 address:9090 in a new tab and you will see :-

Now we saw cadvisor container is running. So copy and paste Public IPv4 address:8080 in a new tab and you will see :-

Now we want docker's log and our app container is already added in cAdvisor. So add cAdvisor code in prometheus.yml as given below :-

Now once we have to restart prometheus container so that prometheus.yml will also update inside our prometheous container. To restart use command docker restart container_id.

Now if we do refresh prometheus page, we will see docker up.

Now if we want see cpu load average in graph, so Goto Graph → Write → rate(container_cpu_load_average_10s{name="node-app"}[5m]).

Part 2 : Install Grafana and set it up to Work with Prometheus

Grafana we use for visualization. Now to install grafana on Ubuntu, use command :-

Update the package list and install Grafana, use command :-

To enable and Start Grafana Service, use command :-

To check Grafana Status, use command :-

Now copy and paste Public IPv4 address:3000 in a new tab and you will see grafana interface :-

Now to login grafana we don't know username & password, so initial username and password for Grafana are username=admin & password=admin.

Now to visualize Prometheus metrics in Grafana, we have to add prometheus(data source) in grafana. So Goto Connection → Search & Click on "Prometheus" → Click "Add new data source" → In Connection Paste Prometheus URL → Save & Test.

Prometheus Dashboard

Now we will make a dashboard to make it easier to view metrics, you can follow these steps :-

Click "Dashboard" → Click "Add visualization" → Select "Prometheus" → Select Metric "container_memory_usage_bytes" → Select Name of container "ubuntu_flaskapp_1" → Click "Run queries" → We will a visualization dashboard of our container memory.

Same we can see other metrics also like errors of app container.

We've successfully installed and set up Grafana to work with Prometheus for monitoring and visualization.

Node Exporter

Node Exporter we use for to scrape detailed servers/systems metrics. Now we want node exporter in the network of prometheus,so add given code in the prometheus's docker-compose file :-

Now run command docker-compose up -d and if we will do docker ps we will see our node-exporter container is running.

Now to configure Prometheus to scrape metrics from Node Exporter, we need to modify the prometheus.yml file so :-

And in the prometheus.yml file add as given below :-

Now once restart prometheus container.

Now if we do refresh prometheus page, we will see node-exporter up.

Now Prometheus is already added in our connection in Grafana. So to view node metrics, we can import a pre-configured dashboard follow these steps :-

Click on Dashboard → Click "Import" → Paste id (e.g., code 1860) → Click "Load" → Select data source "Prometheus" → Click "Import".

Now we will see a Grafana dashboard set up to visualize metrics from Prometheus.

Our Monitoring Project is completed 😄.

🌟I've put a lot of effort please take a look at the project on GitHub and let me know what you think! Your feedback means a lot. Do Repost on Linkedin to share this valuable Project to your Connections.🌟

Let's connect and grow on Linkedin :Click Here

Let's connect and grow on Hashnode :Click Here

Let's connect and grow on Twitter :Click Here

HappyMonitoring!!!!!

HappyReading!!!!!

Sudha Yadav