Author: studyk8s

studyk8s

Docker-compose

About installation
Query the latest version installed to the following address

https://github.com/docker/compose/releases

Then you can the following script

sudo curl -L "https://github.com/docker/compose/releases/download/1.28.5/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
docker-compose --version
By studyk8s 0 Comments

Docker container shared network (localhost)

Because the network namespace between different containers is isolated, the localhost loopback address of each container can only point to itself
However, in k8s, containers in the same pod can actually use localhost to access each other. Why?
K8s’s pod will run a container named pause by default, and other containers will be hung on this pause, that is, the docker run –net container is used

docker run -i -t --name pause busybox sleep infinity
docker run -i -t --name nginx --net container:pause nginx
docker run -i -t --name watcher --net container:pause ubuntu sleep infinity
docker exec -i -t watcher /bin/bash


in the container
apt update
apt install curl
curl localhost:80
By studyk8s 0 Comments

Node Allocatable

Capacity:
cpu: 4
ephemeral-storage: 50824704Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
memory: 8123340Ki
pods: 110
Allocatable:
cpu: 4
ephemeral-storage: 49442272013
hugepages-1Gi: 0
hugepages-2Mi: 0
memory: 8123340Ki
pods: 110

  • Kubelet Node Allocatable 用来为 Kube 组件和 System 进程预留资源，从而保证当节点出现满负荷时也能保证 Kube 和 System 进程有足够的资源。
  • 目前支持 cpu, memory, ephemeral-storage 三种资源预留。
  • Node Capacity 是节点的所有硬件资源，kube-reserved 是给 kube 组件预留的资源，system-reserved 是给系统进程预留的资源，eviction-threshold 是 kubelet 驱逐的阈值设定，allocatable 才是真正调度器调度 Pod 时的参考值（保证节点上所有 Pods 的 request 资源不超过 Allocatable）。
Node Allocatable Resource = Node Capacity - Kube-reserved - system-reserved - eviction-threshold

使用 CLI参数

--enforce-node-allocatable=pods
--kube-reserved=memory=...
--system-reserved=memory=...
--eviction-hard=...

使用配置文件

/var/lib/kubelet/config.yaml

apiVersion: kubelet.config.k8s.io/v1beta1
......
enforceNodeAllocatable:
- pods
kubeReserved:  # 配置 kube 资源预留
  cpu: 500m
  memory: 1Gi
  ephemeral-storage: 1Gi
systemReserved:  # 配置系统资源预留
  memory: 1Gi
evictionHard:  # 配置硬驱逐阈值
  memory.available: "300Mi"
  nodefs.available: "10%"

rke1的情况

nodes:
  - address: 10.10.10.1
    user: rke
    role: [controlplane,etcd]
  - address: 10.10.10.2
    user: rke
    role: [controlplane,etcd,worker]
  - address: 10.10.10.3
    user: rke
    role: [worker]
system_images:
    kubernetes: rancher/hyperkube:v1.13.5-rancher1
services:
  etcd:
    backup_config:
      enabled: true 
      interval_hours: 6
      retention: 12
    extra_args:
      quota-backend-bytes: '4294967296'
  kubelet:
    cluster_domain: cluster.local
    cluster_dns_server: 10.43.0.10
    fail_swap_on: true
    extra_args:
      max-pods: 20
      enforce-node-allocatable: 'pods'
      system-reserved: 'cpu=1,memory=2000Mi'
      kube-reserved: 'cpu=1.5,memory=3000Mi'
  kube-controller:
    cluster_cidr: 10.42.0.0/16
    service_cluster_ip_range: 10.43.0.0/16
    extra_args:
      node-cidr-mask-size: '24'
      node-monitor-period: '5s'
      node-monitor-grace-period: '10s'
      node-startup-grace-period: '20s'
      pod-eviction-timeout: '10s'
  kube-api:
    service_cluster_ip_range: 10.43.0.0/16
    service_node_port_range: 30000-32767
    pod_security_policy: false
    always_pull_images: false
dns:
  provider: coredns

rke2的情况

microk8s的情况

它的kubelet 启动程序是

/snap/microk8s/2407/kubelite

它的参数文件

/var/snap/microk8s/2407/args/kubelet

修改/var/snap/microk8s/2407/args/kubelet,追加以下内容

--enforce-node-allocatable=pods
--kube-reserved=cpu=500m
--kube-reserved=memory=1Gi
--system-reserved=cpu=500m
--system-reserved=memory=1Gi
--eviction-hard=10%

By studyk8s 0 Comments

Git 常用命令

获取commitid
git rev-parse HEAD

通过commit获取 邮箱
git log --no-merges --pretty=format:\"%ae\" -1 " + commitid

通过commit获取 注释
git log --pretty=format:\"%s\" -1 " + commitid

通过commit获取 name
git log --pretty=format:\"%cn\" -1 " + commitid

通过commit获取 时间
git log --pretty=format:\"%ad\" -1 " + commitid
By studyk8s 0 Comments

Argocd Notifications annotations

slack

notifications.argoproj.io/subscribe.on-sync-succeeded.slack:<my-channel>

webhook

notifications.argoproj.io/subscribe.on-sync-succeeded.<webhook-name>: ""

gmail

notifications.argoproj.io/subscribe.on-sync-succeeded.gmail: $EMAIL_USER
By studyk8s 0 Comments

Kube-bench

kube-bench is tool that checks whether Kubernetes is deployed securely by running the checks documented in the CIS Kubernetes Benchmark.

使用docker安装

docker run --rm -v `pwd`:/host aquasec/kube-bench install

调用

./kube-bench
By studyk8s 0 Comments

Kustomize patchesStrategicMerge

在使用补丁机制的时候，要尽量保证每一个补丁只做一件事

原始 nginx.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    workload.user.cattle.io/workloadselector: deployment-lizhe-nginx
  name: nginx
  namespace: lizhe
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      workload.user.cattle.io/workloadselector: deployment-lizhe-nginx
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
    type: RollingUpdate
  template:
    metadata:
      labels:
        workload.user.cattle.io/workloadselector: deployment-lizhe-nginx
    spec:
      containers:
      - image: nginx
        imagePullPolicy: Always
        name: nginx
        resources: {}
        stdin: true
        tty: true
      restartPolicy: Always
      terminationGracePeriodSeconds: 30

kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- nginx.yaml
namePrefix: prod-
patchesStrategicMerge:
- increase_replicas.yaml
- set_memory.yaml

increase_replicas.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  namespace: lizhe
spec:
  replicas: 10

set_memory.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  namespace: lizhe
spec:
  template:
    spec:
      containers:
      - name: my-nginx
        resources:
          limits:
            memory: 512Mi
By studyk8s 0 Comments

Use Kubectl in a Pod

参考 https://kubernetes.io/zh/docs/tasks/run-application/access-api-from-pod/

某些情况下，我们会希望在一个 pod 中能够使用 kubectl 对本集群进行操作，例如

argo workflow 中，需要一个 task 可以调用 kubectl 来修改某些资源

方法一、

最笨的方式是，mount kubeconfig yaml 文件，这种情况下你使用的是 config 文件中的 user ，这里就不讨论了，因为这个方式实在是太笨了

方法二、

使用 kubectl 镜像，然后赋给 pod 对应的 RBAC 权限，这样 pod 就可以通过 RBAC 绑定的 service account 对集群进行操作

使用的镜像是 bitnami/kubectl
使用的 entrypoint 是 /bin/bash

通过 kubectl get nodes 尝试获取集群信息

直接给 Namespace 的 default sa添加一个 cluster级的 admin role

kubectl create clusterrolebinding permissive-binding --clusterrole=cluster-admin --group=system:serviceaccounts:lizhe
clusterrolebinding.rbac.authorization.k8s.io/permissive-binding created

然后我们来尝试一下

By studyk8s 0 Comments

Kustomize install

参考 https://kubectl.docs.kubernetes.io/installation/kustomize/

最容易的应该是直接使用 二进制 文件

https://github.com/kubernetes-sigs/kustomize/releases

sudo chmod a+x kustomize /usr/local/sbin
sudo cp kustomize /usr/local/sbin

我们来尝试一个helloworld

先编写一个 nginx 的 deploy

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    workload.user.cattle.io/workloadselector: deployment-lizhe-nginx
  name: nginx
  namespace: lizhe
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      workload.user.cattle.io/workloadselector: deployment-lizhe-nginx
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
    type: RollingUpdate
  template:
    metadata:
      labels:
        workload.user.cattle.io/workloadselector: deployment-lizhe-nginx
    spec:
      containers:
      - image: nginx
        imagePullPolicy: Always
        name: nginx
        resources: {}
        stdin: true
        tty: true
      restartPolicy: Always
      terminationGracePeriodSeconds: 30

创建一个空文件

kustomization.yaml

然后使用 kustomize cli 来对其添加内容

kustomize edit add resource nginx.yaml

下面我们尝试使用 kustomize edit 给它加一个前缀

需要先修改 kustomization.yaml

kustomize edit set nameprefix 'prod-'

执行

kustomize build ./

你也可以直接应用到集群

kustomize build ./  | kubectl apply -f -
By studyk8s 0 Comments

Kube-shell

安装

$ pip install kube-shell

这里我先把我的 kube config 文件拷贝到默认路径下，然后再启动kube-shell

cp /etc/rancher/rke2/rke2.yaml ~/.kube/config
kube-shell
By studyk8s 0 Comments
Send a Message