Month: May 2021

Ingress sticky session

测试用的 deployment

测试用的 ingress

使用的镜像是 containous/whoami ， 显示出来的效果是这样的

把 pod 数改成 3 之后，很容易就可以得到下面的情况，请求会在3个pod之间轮训

添加annotation就可以了

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/affinity: "cookie"
    nginx.ingress.kubernetes.io/session-cookie-name: "ingresscoookie"
    nginx.ingress.kubernetes.io/session-cookie-hash: "sha1"
    nginx.ingress.kubernetes.io/session-cookie-expires: "172800"
    nginx.ingress.kubernetes.io/session-cookie-max-age: "172800"

auth服务dns

在 ForwardAuth Middleware 中我用来做 auth 鉴权服务的 golang 应用是启动在 vscode 上的，是一个单机服务

在使用过程中有一个小问题，本地dns无法正确生效

在 /etc/hosts中配置的dns无法让 Kubernetes 中的 traefik forward middleware 连接到 golang auth 服务，替换成IP之后故障消除

192.168.194.181	authgo.lizhe.com

然后修改 middleware

apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: lizhe-auth
  namespace: lizhe
spec:
  forwardAuth:
    address: http://authgo.lizhe.com:3080/auth

curl http://authgo.lizhe.com:3080/auth 可以工作，可惜nginx无法工作

添加一个外部的公网 dns
auth.diynocap.com

apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: lizhe-auth
  namespace: lizhe
spec:
  forwardAuth:
    address: http://authgo.diynocap.com:3080/auth

可惜这样还是会得到一个 500

auth服务水平伸缩

本节中我们的目的是把 golang auth 应用安装进 Kubernetes，以提供水平伸缩和编排

之前已经打过docker image

libaibai/lizhe-auth:v0.1

先测试一下clusterip

然后我们来替换成集群内部 dns

curl -I authservice.auth.svc:3080/auth

也可以正常工作

ForwardAuth Middleware

https://doc.traefik.io/traefik/middlewares/forwardauth/

创建中间件

apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: lizhe-auth
spec:
  forwardAuth:
    address: http://192.168.194.181:3080/auth

我会使用 下面的nginx作为测试用的 真实服务

分别创建 traefik service 和 ingressroute
这里为了测试我们没有添加任何中间件

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: authingressroute
  namespace: lizhe
spec:
  entryPoints:
    - web
  routes:
  - match: Host(`auth.lizhe.com`)
    kind: Rule
    services:
    - name: authts
      kind: TraefikService

kind: TraefikService
metadata:
  name: authts
  namespace: lizhe
spec:
  weighted:
    services:
      - name: nginxsvc
        port: 80
        kind: Service

然后我们加入中间件

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: authingressroute
  namespace: lizhe
spec:
  entryPoints:
    - web
  routes:
  - match: Host(`auth.lizhe.com`)
    kind: Rule
    services:
    - name: authts
      kind: TraefikService
    middlewares:
      - name: lizhe-auth

再次访问 测试用 nginx

这里可以看到header中存在很多信息

我们用curl命令测试在header 中添加一些 token

直接关闭 auth 服务器，会得到 500 错误

这里为了观察被访问真实服务（刚才的nginx）是否能正确获得 request header 的token，我把nginx替换成了我们用来显示 request header 的 golang 程序

可以看到 request header 中的token被正确的在整个调用链上传递，至此，POC结束

Golang Auth服务

以下golang程序会输出接收到的rquest header

package main

import (
	"log"
	"net/http"

	handler "lizhe-auth/internal/handler"
)

func main() {
	http.HandleFunc("/auth", handler.WithArgHandler)
	log.Println("server started on 3080")
	http.ListenAndServe(":"+"3080", nil)
}

package handler

import (
	"log"
	"net/http"
)

func WithArgHandler(writer http.ResponseWriter, request *http.Request) {
	log.Println(request.Header)
}

构建Dockerfile

FROM golang:alpine as golang   

RUN mkdir -p /root/auth
RUN mkdir -p /root/auth/output
COPY ./ /root/auth

WORKDIR /root/auth/output
RUN go build /root/auth/cmd/main.go


FROM alpine as alpine

RUN mkdir -p /root/auth
COPY --from=golang --chown=root:root /root/auth/output /root/auth

CMD /root/auth/main 

然后build镜像

docker build -t libaibai/lizhe-auth:v0.1 .
docker run -i -t --name test -p 3080:3080 libaibai/lizhe-auth:v0.1

Traefik ForwardAuth

ForwardAuth 可以用来外挂一些 验证、鉴权 服务，之所以使用它主要是我需要一个 Gateway

这个Gateway需要调用一些外部接口，然后执行一些业务逻辑，构成一个AOP切面

大约分成几个部分

1. 能够接收请求的Golang rest服务，作为auth restapi ，通过header来获得token
2. 将上面的Golang应用容器化，制作docker镜像
3. 构建服务使用 ForwardAuth中间件调用上面的服务

Traefik Middleware

安装一个测试用的nginx

给这个 deploy 安装一个 svc

创建对应的 Traefik service 和 Ingress route

 apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: ngingressroute
  namespace: lizhe
spec:
  entryPoints:
    - web
  routes:
  - match: Host(`ngts.lizhe.com`)
    kind: Rule
    services:
    - name: ngts
      kind: TraefikService

apiVersion: traefik.containo.us/v1alpha1
kind: TraefikService
metadata:
  name: ngts
  namespace: lizhe
spec:
  weighted:
    services:
      - name: nginxsvc
        port: 80
        kind: Service

创建一个 middleware

apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: test-ipwhitelist
  namespace: lizhe
spec:
  ipWhiteList:
    sourceRange:
      - 127.1.1.1/32
      - 192.1.1.1/32

然后需要修改一下 route ，让route使用这个新创建的 middleware

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: ngingressroute
  namespace: lizhe
spec:
  entryPoints:
    - web
  routes:
  routes:
  - middlewares:
      - name: test-ipwhitelist
    match: Host(`ngts.lizhe.com`)
    kind: Rule
    services:
    - name: ngts
      kind: TraefikService

更新一下这个白名单，打开 本机 ip

这里不需要重启，就可以看见已经生效了

Traefik TCP服务

需要添加 endpoint 和 hostport

    spec:
      containers:
      - args:
        - --global.checknewversion
        - --global.sendanonymoususage
        - --entryPoints.traefik.address=:9000/tcp
        - --entryPoints.web.address=:8000/tcp
        - --entryPoints.websecure.address=:8443/tcp
        - --entryPoints.mysql.address=:3306/tcp
        - --api.dashboard=true
        - --ping=true
        - --providers.kubernetescrd
        - --providers.kubernetesingress

        image: traefik:2.4.8
        imagePullPolicy: IfNotPresent
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /ping
            port: 9000
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 2
        name: traefik
        ports:
        - containerPort: 9000
          name: traefik
          protocol: TCP
        - containerPort: 8000
          name: web
          protocol: TCP
        - containerPort: 8443
          name: websecure
          protocol: TCP
        - containerPort: 8000
          hostPort: 8000
          name: http
          protocol: TCP
        - containerPort: 9000
          hostPort: 9000
          name: db
          protocol: TCP
        - containerPort: 3306
          hostPort: 3306
          name: mysql
          protocol: TCP

apiVersion: v1
kind: Service
metadata:
  name: mysql-traefik
  namespace: database
spec:
  selector:
    app: mysql
  ports:
  - port: 3306
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
  name: traefik-tcp
  namespace: database
spec:
  entryPoints:
    - mysql
  routes:
  - match: HostSNI(`*`)
    services:
    - name: mysql-traefik
      port: 3306

Traefik 流量复制

流量复制可以将 request 发给主 endpoint 的同时，将同样的request发给另一个endpoint，并且忽略第二个endpoint的response

apiVersion: traefik.containo.us/v1alpha1
kind: TraefikService
metadata:
  name: app-mirror
  namespace: version
spec:
  mirroring:
    name: ver1 # 发送 100% 的请求到 K8S 的 Service "v1"
    port: 80
    mirrors:
    - name: ver2 # 然后复制 50% 的请求到 v2
      percent: 50
      port: 80

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: mirror-ingress-route
  namespace: version
spec:
  entryPoints:
  - web
  routes:   
  - match: Host(`mi.lizhe.com`)
    kind: Rule
    services:
    - name: app-mirror
      kind: TraefikService # 使用声明的 TraefikService 服务，而不是 K8S 的 Service

for i in `seq 20`; do curl http://mi.lizhe.com:8000/helloworld.html;echo "" ;done

Traefik 权重分流

不同于 Istio，Traefik需要每个dep对应各自的 service

这里我们分别创建3个不同的 service 使用clusterip

创建 traefik service

apiVersion: traefik.containo.us/v1alpha1
kind: TraefikService
metadata:
  name: lizhets
  namespace: version
spec:
  weighted:
    services:
      - name: ver1
        weight: 3  # 定义权重
        port: 80
        kind: Service  # 可选，默认就是 Service
      - name: ver2
        weight: 1
        port: 80

创建 IngressRoute

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: lizheingressroute
  namespace: version
spec:
  entryPoints:
    - web
  routes:
  - match: Host(`ts.lizhe.com`)
    kind: Rule
    services:
    - name: lizhets
      kind: TraefikService

for i in `seq 20`; do curl http://ts.lizhe.com:8000/helloworld.html;echo "" ;done