Kubernetes

Argo Workflows workflow gcttl

Argo 的 workflow 运行单位是 container，每个container都会在kubernetes上创建一个 pod

我的问题来自于 “如果pod创建太多了怎么办”

这个 2018年的帖子上，好多人给出了各种解决方案

https://github.com/kubeflow/testing/issues/53

很幸运的是官方已经给出另一个solution

使用 gc-tll

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: gc-ttl-
spec:
  ttlStrategy:
    secondsAfterCompletion: 60 # Time to live after workflow is completed, replaces ttlSecondsAfterFinished
    secondsAfterSuccess: 60     # Time to live after workflow is successful
    secondsAfterFailure: 60     # Time to live after workflow fails
  entrypoint: whalesay
  templates:
  - name: whalesay
    container:
      image: docker/whalesay:latest
      command: [cowsay]
      args: ["hello world"]

可以看到 workflow启动时是这样的

60秒之后，自动清理了

Argo Workflows workflow steps

官方文档的例子中，有一个 steps

https://argoproj.github.io/argo-workflows/examples/#hello-world

以下脚本中主要创建了
3个steps + 1个template

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: steps-
spec:
  entrypoint: hello-hello-hello

  # This spec contains two templates: hello-hello-hello and whalesay
  templates:
  - name: hello-hello-hello
    # Instead of just running a container
    # This template has a sequence of steps
    steps:
    - - name: hello1            # hello1 is run before the following steps
        template: whalesay
        arguments:
          parameters:
          - name: message
            value: "hello1"
    - - name: hello2a           # double dash => run after previous step
        template: whalesay
        arguments:
          parameters:
          - name: message
            value: "hello2a"
      - name: hello2b           # single dash => run in parallel with previous step
        template: whalesay
        arguments:
          parameters:
          - name: message
            value: "hello2b"

  # This is the same template as from the previous example
  - name: whalesay
    inputs:
      parameters:
      - name: message
    container:
      image: docker/whalesay
      command: [cowsay]
      args: ["{{inputs.parameters.message}}"]

这里的结构实际上是一个 二维数组 [[hello1],[hello2a,hello2b]]

    steps:
    - - name: hello1
    - - name: hello2a
      - name: hello2b

这里还有一个 DAG 模型的 执行器

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: dag-diamond-
spec:
  entrypoint: diamond
  templates:
  - name: echo
    inputs:
      parameters:
      - name: message
    container:
      image: alpine:3.7
      command: [echo, "{{inputs.parameters.message}}"]
  - name: diamond
    dag:
      tasks:
      - name: A
        template: echo
        arguments:
          parameters: [{name: message, value: A}]
      - name: B
        dependencies: [A]
        template: echo
        arguments:
          parameters: [{name: message, value: B}]
      - name: C
        dependencies: [A]
        template: echo
        arguments:
          parameters: [{name: message, value: C}]
      - name: D
        dependencies: [B, C]
        template: echo
        arguments:
          parameters: [{name: message, value: D}]

Git 获取commitid和commit

            script {
                sh "pwd"
                sh "pwd"
                sh "ls /var/jenkins_home/workspace/${env.JOB_NAME}"
                env.GIT_COMMIT_MSG = sh (script: 'git log -1 --pretty=%B ${GIT_COMMIT}', returnStdout: true).trim()
                echo "${env.GIT_COMMIT_MSG}"
                echo "${env.GIT_COMMIT_MSG}"
                echo "${env.GIT_COMMIT_MSG}"
                env.GIT_COMMIT = sh (script: 'git rev-parse HEAD', returnStdout: true).trim()
                echo "${env.GIT_COMMIT}"
                echo "${env.GIT_COMMIT}"
                echo "${env.GIT_COMMIT}"
            }

React build

先使用脚手架创建了2个 react 项目

sudo npm install -g create-react-app
create-react-app projecta
create-react-app projectb

在projectb中创建一个自己的 hello 模块

import React from 'react'

const HelloModule = (props) => {
    return (
        <div>
            <div>helloworld</div>
            <button>Helloworld Button</button>
        </div>
    );
}

export default HelloModule;

然后我们尝试使用 babel 来编译这个模块

先安装 babel cli 工具集

npm install --save-dev @babel/cli
npm install --save-dev @babel/core
npm install --save-dev @babel/preset-env
npm install --save-dev @babel/preset-react

package.json 中添加一个 scripts babel

babel 脚本会使用 “./node_modules/.bin/babel 把 src/components/Hello.js 编译成 hello.js

  "scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject",
    "babel": "./node_modules/.bin/babel src/components/Hello.js --out-file hello.js"
  },

package.json 中添加 main，指向 hello.js

{
  "name": "projectb",
  "main": "hello.js",
  "version": "0.1.0",
  "private": true,
  ...
}

然后调用link

npm link

在projecta中

import Hello from 'projectb'

<Hello />

Argo Workflows workflow helloworld

这里使用官方文档中的 docker/whalesay 来创建一个 helloworld 的 workflow

docker run docker/whalesay cowsay "hello world"

网络问题我手动下载镜像 (这里已经通过tag重命名了）

microk8s ctr image pull docker.io/docker/whalesay:latest
microk8s ctr image pull docker.io/libaibai/whalesay:latest

microk8s ctr image pull docker.io/argoproj/argoexec:v3.0.3

apiVersion: argoproj.io/v1alpha1
kind: Workflow                  # new type of k8s spec
metadata:
  generateName: hello-world-    # name of the workflow spec
  namespace: argo
spec:
  entrypoint: whalesay          # invoke the whalesay template
  templates:
  - name: whalesay              # name of the template
    container:
      image: docker/whalesay
      command: [cowsay]
      args: ["hello world"]
      resources:                # limit the resources
        limits:
          memory: 32Mi
          cpu: 100m

出现了错误

Unable to attach or mount volumes: unmounted volumes=[docker-sock], unattached volumes=[kube-api-access-btrhw podmetadata docker-sock my-minio-cred]: timed out waiting for the condition

参考

https://argoproj.github.io/argo-workflows/workflow-executors/
https://argoproj.github.io/argo-workflows/workflow-controller-configmap.yaml

这里使用的是 microk8s ，没有docker.sock，只有 containerd.sock

/var/snap/microk8s/common/run/containerd.sock

这里我直接去按了一个docker

修改configmap

还是不好用，最后谢谢下面的大哥

https://discuss.kubernetes.io/t/issues-with-running-argo-against-microk8s-on-ubunte/13553

PNS 是好用的 Process Namespace Sharing

使用pns模式之后，正常工作

Argo Workflows workflow template

我们使用 Argo Workflows workflow helloworld 中的 yaml 可以直接创建一个 workflow 的 template

可以通过这个新创建的 template 直接启动需要的 workflow

创建一个带参数的 workflow

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  name: hello-world-parameters
spec:
  # invoke the whalesay template with
  # "hello world" as the argument
  # to the message parameter
  entrypoint: whalesay
  arguments:
    parameters:
    - name: message
      value: hello world

  templates:
  - name: whalesay
    inputs:
      parameters:
      - name: message       # parameter declaration
    container:
      # run cowsay with that message input parameter as args
      image: docker/whalesay
      command: [cowsay]
      args: ["{{inputs.parameters.message}}"]

使用这个template启动新的workflow时 可以输入 新参数

执行结果

Argo Workflows 安装

kubectl create ns argo
kubectl apply -n argo -f https://raw.githubusercontent.com/argoproj/argo-workflows/stable/manifests/quick-start-postgres.yaml

kubectl -n argo port-forward deployment/argo-server 2746:2746

ArgoCD Auto Sync

修改yaml文件，替换nginx的image version 从 1.7.9 到 latest

可以看到 kubernetes 上 nginx 版本已经被替换

ArgoCD HelloWorld

这里使用的是测试仓库(private)

https://github.com/zl86790/argocdtest.git

注意这个仓库不能为空，否则一会儿会被提示一个 empty repository错误

使用argocd连接这个repo

选择Repository

选择 CONNECT REPO USING SSH

ssh-keygen -t rsa -b 4096
id_rsa -> argocd
id_rsa.pub -> github

会得到如下连接

创建一个 application

配置如下

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: argocdtest
spec:
  destination:
    name: ''
    namespace: ''
    server: 'https://kubernetes.default.svc'
  source:
    path: ./
    repoURL: 'git@github.com:zl86790/argocdtest.git'
    targetRevision: HEAD
  project: default
  syncPolicy:
    automated: null

在repo中追加一个新文件

apiVersion: apps/v1
kind: Deployment
metadata:
 name: nginx
 namespace: argocdtest
spec:
 selector:
  matchLabels:
   app: web_server
 replicas: 2
 template:
  metadata:
   labels:
    app: web_server
  spec:
   containers:
   - name: nginx
     image: nginx:1.7.9
     ports:
     - containerPort: 80

点击 REFRESH 会看到

点击 同步 SYNC

可以看到Nginx已经正确部署了

ArgoCD安装

kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

kubectl port-forward --address 0.0.0.0 svc/argocd-server -n argocd 8000:443

用户名 admin

密码是 Pod 名

kubectl get pods -n argocd -l app.kubernetes.io/name=argocd-server -o name | cut -d'/' -f 2
argocd-server-86f7f94488-pv2zk

不过我这里不好用，查看了一下 secrets 发现密码是 HclJUPjSiSwmWZpC