lovewell

@yibo2018 那是得要对整个体系（知识面广），基础牢固的人才行，要不然还是得 google 。

@amrom 大佬，我在 3 个节点单节中关闭 2 个，然后用 kubectl 的 admin 证书替换掉 kube-controller-manager 的 kubeconfig,auth/z-kubeconfig 也是失败的。下面是我的生成 kube-controller-manager-kubeconfig.yaml 的过程。
===
这个是 csr:
{
"CN": "system:kube-controller-manager",
"hosts": [
"192.168.62.131",
"192.168.62.132",
"192.168.62.133",
"127.0.0.1",
"localhost"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "Guangdong",
"L": "Dongguan",
"O": "system:kube-controller-manager",
"OU": "Kubernetes"
}
]
}


ansible 对应的脚本：

- name: kube-controller-manager-csr.json
ansible.builtin.shell:
cmd: >-
cfssl gencert -ca {{ control_node_temp_dirs.temp_certs_dir }}/kubernetes-ca.pem
-ca-key {{ control_node_temp_dirs.temp_certs_dir }}/kubernetes-ca-key.pem
-config {{ role_path }}/ca-config.json
-profile peer {{ control_node_temp_dirs.temp_csrs_dir }}/kube-controller-manager-csr.json
| cfssljson -bare {{ control_node_temp_dirs.temp_certs_dir }}/kube-controller-manager
- name: set-cluster kube-controller-manager-kubeconfig.yaml
ansible.builtin.shell:
cmd: >-
kubectl config set-cluster {{ k8s.cluster_name }}
--certificate-authority={{ control_node_temp_dirs.temp_certs_dir }}/kubernetes-ca.pem
--embed-certs=true
--server=https://127.0.0.1:6443
--kubeconfig={{ control_node_temp_dirs.temp_kubeconfigs_dir }}/kube-controller-manager-kubeconfig.yaml
- name: set-credentials kube-controller-manager-kubeconfig.yaml
ansible.builtin.shell:
cmd: >-
kubectl config set-credentials system:kube-controller-manager
--client-certificate={{ control_node_temp_dirs.temp_certs_dir }}/kube-controller-manager.pem
--client-key={{ control_node_temp_dirs.temp_certs_dir }}/kube-controller-manager-key.pem
--embed-certs=true
--kubeconfig={{ control_node_temp_dirs.temp_kubeconfigs_dir }}/kube-controller-manager-kubeconfig.yaml
- name: set-context kube-controller-manager-kubeconfig.yaml
ansible.builtin.shell:
cmd: >-
kubectl config set-context default
--cluster={{ k8s.cluster_name }}
--user=system:kube-controller-manager
--kubeconfig={{ control_node_temp_dirs.temp_kubeconfigs_dir }}/kube-controller-manager-kubeconfig.yaml
- name: kubeconfig use-context kube-controller-manager
ansible.builtin.shell:
cmd: kubectl config use-context default --kubeconfig={{ control_node_temp_dirs.temp_kubeconfigs_dir }}/kube-controller-manager-kubeconfig.yaml

@hwdef hmm, 前面的各种困难都过来了，现在只差成功一几步，很想知道是啥导致的。

@hwdef 我是想一步到位，顺便了解下底层交互。这个是要搭在公司测试的，minikube 更不行了。。

过了认证，但是没被授权。我试过吧 kubeconfig 换成 CN：admin O：system:masters 也不行。。所以我就想是他请求 ip 份配的时候到底是用那个角色。。

这是我的 apiserver 启动配置：
KUBE_APISERVER_ARGS='
--api-audiences=https://kubernetes.default.svc.cluster.local
--runtime-config=api/all=true
--apiserver-count=3
--allow-privileged=true
--advertise-address=192.168.62.131
--bind-address=0.0.0.0
--secure-port=6443
--storage-backend=etcd3
--etcd-cafile=/etc/etcd/pki/etcd-ca.pem
--etcd-certfile=/etc/kubernetes/kube-apiserver/pki/kube-apiserver-client-etcd.pem
--etcd-keyfile=/etc/kubernetes/kube-apiserver/pki/kube-apiserver-client-etcd-key.pem
--etcd-servers=https://192.168.62.131:2379,https://192.168.62.132:2379,https://192.168.62.133:2379
--kubelet-certificate-authority=/etc/kubernetes/pki/kubernetes-ca.pem
--kubelet-client-certificate=/etc/kubernetes/kube-apiserver/pki/kube-apiserver-client-kubelet.pem
--kubelet-client-key=/etc/kubernetes/kube-apiserver/pki/kube-apiserver-client-kubelet-key.pem
--kubelet-preferred-address-types=InternalIP,InternalDNS,ExternalIP,ExternalDNS,Hostname
--kubelet-timeout=10s
--service-cluster-ip-range=10.22.88.0/22
--service-node-port-range=30000-32767
--service-account-key-file=/etc/kubernetes/kube-apiserver/pki/sa-pub.pem
--service-account-issuer=https://kubernetes.default.svc.cluster.local
--service-account-signing-key-file=/etc/kubernetes/kube-apiserver/pki/sa-signing-key.pem
--enable-bootstrap-token-auth=true
--anonymous-auth=false
--authorization-mode=RBAC,Node
--client-ca-file=/etc/kubernetes/pki/kubernetes-ca.pem
--cert-dir=/etc/kubernetes/kube-apiserver/pki
--tls-cert-file=/etc/kubernetes/kube-apiserver/pki/kube-apiserver.pem
--tls-private-key-file=/etc/kubernetes/kube-apiserver/pki/kube-apiserver-key.pem
--event-ttl=168h
--audit-log-maxage=15
--audit-log-maxbackup=3
--audit-log-maxsize=100
--audit-log-truncate-enabled
--audit-log-path=/var/log/kubernetes/kube-apiserver/audit.log
--audit-policy-file=/etc/kubernetes/kube-apiserver/audit-policy.yaml
--requestheader-allowed-names=aggregator-proxy-client
--requestheader-extra-headers-prefix=X-Remote-Extra-
--requestheader-group-headers=X-Remote-Group
--requestheader-username-headers=X-Remote-User
--requestheader-client-ca-file=/etc/kubernetes/pki/aggregation-ca.pem
--proxy-client-cert-file=/etc/kubernetes/kube-apiserver/pki/aggregator-proxy-client.pem
--proxy-client-key-file=/etc/kubernetes/kube-apiserver/pki/aggregator-proxy-client-key.pem
--enable-aggregator-routing=true
--profiling
--default-not-ready-toleration-seconds=360
--default-unreachable-toleration-seconds=360
--max-mutating-requests-inflight=2000
--max-requests-inflight=4000
--default-watch-cache-size=200
--delete-collection-workers=2
--logtostderr=false
--logging-format=text
--log-dir=/var/log/kubernetes/kube-apiserver
--v=2
'


这是我的 kube-controller-manager 配置：

KUBE_CONTROLLER_MANAGER_ARGS='
--cluster-name=kubernetes
--profiling
--kubeconfig=/etc/kubernetes/kube-controller-manager/kube-controller-manager-kubeconfig.yaml
--authentication-kubeconfig=/etc/kubernetes/kube-controller-manager/kube-controller-manager-kubeconfig.yaml
--authorization-kubeconfig=/etc/kubernetes/kube-controller-manager/kube-controller-manager-kubeconfig.yaml
--controllers=*,bootstrapsigner,tokencleaner
--bind-address=0.0.0.0
--service-cluster-ip-range=10.22.88.0/22
--kube-api-qps=1000
--kube-api-burst=2000
--use-service-account-credentials=true
--concurrent-service-syncs=2
--root-ca-file=/etc/kubernetes/pki/kubernetes-ca.pem
--service-account-private-key-file=/etc/kubernetes/kube-controller-manager/pki/sa-key.pem
--allocate-node-cidrs=true
--cluster-cidr=172.16.0.0/12
--node-cidr-mask-size=24
--cert-dir=/etc/kubernetes/kube-controller-manager/pki
--tls-cert-file=/etc/kubernetes/kube-controller-manager/pki/kube-controller-manager.pem
--tls-private-key-file=/etc/kubernetes/kube-controller-manager/pki/kube-controller-manager-key.pem
--client-ca-file=/etc/kubernetes/pki/kubernetes-ca.pem
--cluster-signing-cert-file=/etc/kubernetes/pki/kubernetes-ca.pem
--cluster-signing-key-file=/etc/kubernetes/pki/kubernetes-ca-key.pem
--requestheader-allowed-names=aggregator-proxy-client
--requestheader-extra-headers-prefix=X-Remote-Extra-
--requestheader-group-headers=X-Remote-Group
--requestheader-username-headers=X-Remote-User
--requestheader-client-ca-file=/etc/kubernetes/pki/aggregation-ca.pem
--logtostderr=false
--logging-format=text
--log-dir=/var/log/kubernetes/kube-controller-manager
--v=2
'

感觉没问题，实际不行。

@idblife 直接 github 下载 tar.gz ，用 ansible 安装进 3 台虚拟机。