Loki와 Promtail 설치

Overview

Loki와 Promtail 설치방법에 대해서 알아본다.

 

출처 : https://hackernoon.com/grafana-loki-architecture-summary-and-running-in-kubernetes

 

참고

Loki-Stack 을 사용해서 간단하게 Loki와 Promtail을 배포할 수 있지만 배포해보니, Loki-Stack에서 제공해주는 버전이 너무 낮아, 최신 Grafana에서 사용하는 Query와 호환되지 않는다. 따라서 Loki와 Promtail 모두 단일 Helm Chart를 사용해서 배포한다.

​

---

Loki 설치

해당 설치과정은 간단하게 SingleBinary로 설치하는 내용이다.

• https://grafana.com/docs/loki/latest/setup/install/helm/install-monolithic/

 

 

Helm Chart 등록

helm repo add grafana https://grafana.github.io/helm-charts
helm repo update

 

 

git clone 및 values 파일 복사

# git clone
git clone https://github.com/grafana/loki.git
cd loki/production/helm/loki/

# helm 의존성 업데이트
helm dependency update

# values 파일 복사
mkdir values
cp values.yaml values/somaz.yaml

 

 

`values/somaz.yaml` 수정

global:
  clusterDomain: "somaz-cluster.local"
  dnsService: "coredns"
  dnsNamespace: "kube-system"
deploymentMode: SingleBinary
loki:
  commonConfig:
    replication_factor: 1
  storage:
    type: filesystem
  schemaConfig:
    configs:
    - from: "2024-01-01"
      store: tsdb
      index:
        prefix: loki_index_
        period: 24h
      object_store: filesystem # we're storing on filesystem so there's no real persistence here.
      schema: v13
singleBinary:
  # -- Number of replicas for the single binary
  replicas: 1
  persistence:
    enabled: true
    storageClass: "default"
    accessModes:
      - ReadWriteOnce
    size: 10Gi
gateway:
  # -- Specifies whether the gateway should be enabled
  enabled: true
  service:
    # -- Type of the gateway service
    type: NodePort
    nodePort: 31400
chunksCache:
  enabled: false
resultsCache:
  enabled: false
tableManager:
  enabled: false
backend:
  replicas: 0
read:
  replicas: 0
write:
  replicas: 0
ingester:
  replicas: 0
querier:
  replicas: 0
queryFrontend:
  replicas: 0
queryScheduler:
  replicas: 0
distributor:
  replicas: 0
compactor:
  replicas: 0
indexGateway:
  replicas: 0
bloomCompactor:
  replicas: 0
bloomGateway:
  replicas: 0
test:
  enabled: false
lokiCanary:
  enabled: false

 

 

설치한다.

# lint 확인
helm lint --values ./values/somaz.yaml  

# 설치 전 확인
helm install loki . -n monitoring -f ./values/somaz.yaml --create-namespace --dry-run >> output.yaml

# 설치
helm install loki . -n monitoring -f ./values/somaz.yaml --create-namespace

# 업그레이드 방법
helm upgrade loki . -n monitoring -f ./values/somaz.yaml

 

 

확인한다. NodePort로 배포했기 때문에 `Kubernetes Node IP:Port` 로 loki의 상태를 조회할 수 있다. Promtail 까지 배포한뒤에는 Query를 사용해 데이터를 확인할수도 있다.

# 상태 확인
curl 192.168.100.11:31400
OK

curl 192.168.100.11:31400/config

curl -G -s "http://192.168.100.11:31400/loki/api/v1/query_range" \
    --data-urlencode 'query={app="node-feature-discovery"}'
...
{"status":"success","data":{"resultType":"streams","result":[{"stream":{"filename":"/var/log/pods/monitoring_node-feature-discovery-worker-5v2pp_3bd016b2-4a69-4a6c-8fc5-17f7f9749dd1/worker/0.log","instance":"node-feature-discovery","job":"monitoring/node-feature-discovery","namespace":"monitoring","app":"node-feature-discovery","container":"worker","stream":"stderr","node_name":"mgmt-w-01","pod":"node-feature-discovery-worker-5v2pp"},"values":[["1725606670299872220","I0906 07:11:10.299790       1 nfd-worker.go:660] \"feature discovery completed\""],["1725606670299519154","I0906 07:11:10.299254       1 nfd-worker.go:647] \"starting feature discovery...\""],["1725606610285963555","I0906 07:10:10.285522       1 nfd-worker.go:660] \"feature discovery completed\""],["1725606610285873819","I0906 07:10:10.285262       1 nfd-worker.go:647] \"starting feature discovery...\""],["1725606550312098625","I0906 07:09:10.312010       1 nfd-worker.go:660] \"feature discovery completed\""],["1725606550311763894","I0906 07:09:10.311531       1 nfd-worker.go:647] \"starting feature discovery...\""],["1725606490300176632","I0906 07:08:10.300070       1 nfd-worker.go:660] \"feature discov

curl -G -s "http://192.168.100.11:31400/loki/api/v1/query_range" \
    --data-urlencode 'query={app="prometheus-node-exporter"}'
{"status":"success","data":{"resultType":"streams","result":[],"stats":{"summary":{"bytesProcessedPerSecond":0,"linesProcessedPerSecond":0,"totalBytesProcessed":0,"totalLinesProcessed":0,"execTime":0.001597489,"queueTime":0.00005817,"subqueries":1,"totalEntriesReturned":0},"querier":{"store":{"totalChunksRef":0,"totalChunksDownloaded":0,"chunksDownloadTime":0,"chunk":{"headChunkBytes":0,"headChunkLines":0,"decompressedBytes":0,"decompressedLines":0,"compressedBytes":0,"totalDuplicates":0}}},"ingester":{"totalReached":1,"totalChunksMatched":0,"totalBatches":0,"totalLinesSent":0,"store":{"totalChunksRef":0,"totalChunksDownloaded":0,"chunksDownloadTime":0,"chunk":{"headChunkBytes":0,"headChunkLines":0,"decompressedBytes":0,"decompressedLines":0,"compressedBytes":0,"totalDuplicates":0}}}}

 

 

---

 

promtail 설치

 

promtail 설치는 훨씬 간단하다.

git clone https://github.com/grafana/helm-charts.git

cd helm-charts/charts/promtail
helm dependency update

mkdir values
cp values.yaml values/somaz.yaml

 

 

`values/somaz.yaml` 수정

daemonset:
  enabled: true
serviceMonitor:
  enabled: true
config:
  enabled: true
  logLevel: info
  logFormat: logfmt
  serverPort: 3101
  clients:
    - url: http://loki-gateway/loki/api/v1/push

 

 

설치한다.

# lint 확인
helm lint --values ./values/somaz.yaml  

# 설치 전 확인
helm install promtail . -n monitoring -f ./values/somaz.yaml --create-namespace --dry-run >> output.yaml

# 설치
helm install promtail . -n monitoring -f ./values/somaz.yaml --create-namespace

# 업그레이드 방법
helm upgrade promtail . -n monitoring -f ./values/somaz.yaml

 

 

간단하게 Loki와 Promtail을 구성해보았다.

다음 포스팅에는 Grafana 까지 구축하여 Prometheus + Thanos 를 활용해 수집한 Metric과

Loki + Promtail 을 활용해 수집한 로그 데이터를 Grafana를 사용해서 시각화 해보려고 한다.

​

 

 

---

Reference

https://hackernoon.com/grafana-loki-architecture-summary-and-running-in-kubernetes

https://github.com/grafana/loki/tree/main/production/helm/loki

https://github.com/grafana/helm-charts/tree/main/charts/promtail

https://github.com/grafana/helm-charts/tree/main/charts/loki-stack

https://grafana.com/docs/loki/latest/setup/install/helm/install-monolithic/