K8s实战指南：构建高可用Redis Cluster（三主三从）与Proxy的自动化运维体系

1. Redis Cluster与Proxy架构解析Redis Cluster是Redis官方提供的分布式解决方案通过数据分片Sharding和主从复制Replication机制实现水平扩展和高可用性。一个典型的三主三从架构包含3个主节点每个主节点负责5461个哈希槽共16384个槽3个从节点每个从节点自动复制对应主节点数据智能路由客户端通过MOVED/ASK重定向自动找到正确节点这种架构的优势在于自动故障转移当主节点不可用时从节点会自动升级为新主节点数据分片数据均匀分布在多个节点突破单机内存限制线性扩展只需增加节点集群会自动重新分配哈希槽但原生Redis Cluster存在两个显著痛点客户端需要实现集群协议处理重定向逻辑集群拓扑变更时需要客户端动态更新路由表这正是Redis Cluster Proxy的价值所在。它作为中间层提供统一接入点应用像访问单节点Redis一样使用集群自动处理槽位映射和请求路由对客户端透明实时感知集群拓扑变化自动更新路由策略在K8s环境中这套组合方案能完美适配有状态服务的部署需求StatefulSet保证Pod身份和存储的稳定性Headless Service实现集群节点发现ConfigMap统一管理配置Job完成集群初始化等一次性任务2. K8s环境准备与资源配置2.1 创建专用命名空间首先为Redis集群创建隔离环境# redis-cluster-namespace.yaml apiVersion: v1 kind: Namespace metadata: name: redis-cluster执行命令kubectl apply -f redis-cluster-namespace.yaml2.2 配置管理ConfigMap这是集群的核心配置需要特别注意以下参数# redis-cluster-config.yaml apiVersion: v1 kind: ConfigMap metadata: name: redis-config namespace: redis-cluster data: redis.conf: | port 6379 requirepass YourStrongPassword # 集群认证密码 masterauth YourStrongPassword # 主从认证密码 cluster-enabled yes # 启用集群模式 cluster-config-file nodes.conf # 自动生成的集群配置 cluster-node-timeout 5000 # 节点超时时间(ms) appendonly yes # 开启持久化 appendfsync everysec # 每秒同步关键配置说明cluster-node-timeout影响故障判定速度生产环境建议5-15秒requirepass和masterauth必须相同否则主从同步会失败appendfsync平衡性能与可靠性金融场景可设为always2.3 服务暴露方案设计需要两种Service配合工作Headless Service无头服务apiVersion: v1 kind: Service metadata: name: redis-headless namespace: redis-cluster spec: clusterIP: None # 关键配置 selector: app: redis ports: - port: 6379特点直接暴露Pod DNSredis-0.redis-headless.redis-cluster.svc用于集群内部节点发现和通信NodePort Service访问服务apiVersion: v1 kind: Service metadata: name: redis-access namespace: redis-cluster spec: type: NodePort selector: app: redis ports: - port: 6379 targetPort: 6379 nodePort: 30079 # 30000-32767范围注意生产环境建议使用Ingress或LoadBalancer多节点部署时需要配置外部负载均衡3. StatefulSet部署集群节点3.1 有状态副本配置# redis-cluster-statefulset.yaml apiVersion: apps/v1 kind: StatefulSet metadata: name: redis namespace: redis-cluster spec: serviceName: redis-headless replicas: 6 # 3主3从 selector: matchLabels: app: redis template: metadata: labels: app: redis spec: affinity: podAntiAffinity: # 反亲和性确保节点分散 requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: app operator: In values: [redis] topologyKey: kubernetes.io/hostname containers: - name: redis image: redis:7.2-alpine command: [redis-server, /usr/local/etc/redis/redis.conf] ports: - containerPort: 6379 volumeMounts: - name: data mountPath: /data - name: config mountPath: /usr/local/etc/redis/redis.conf subPath: redis.conf volumes: - name: config configMap: name: redis-config volumeClaimTemplates: # 持久化存储声明 - metadata: name: data spec: accessModes: [ ReadWriteOnce ] storageClassName: standard resources: requests: storage: 1Gi关键设计要点反亲和性确保Pod分散在不同物理节点持久化存储PVC模板为每个Pod自动创建独立存储配置注入通过ConfigMap统一管理配置固定网络标识StatefulSet保证Pod名称有序且稳定部署命令kubectl apply -f redis-cluster-statefulset.yaml3.2 集群初始化Job当所有Pod就绪后需要执行集群初始化# init-cluster-job.yaml apiVersion: batch/v1 kind: Job metadata: name: redis-init-cluster namespace: redis-cluster spec: template: spec: containers: - name: redis-init image: redis:7.2-alpine command: - sh - -c - | sleep 10 # 等待所有Pod就绪 redis-cli --cluster create \ redis-0.redis-headless.redis-cluster.svc:6379 \ redis-1.redis-headless.redis-cluster.svc:6379 \ redis-2.redis-headless.redis-cluster.svc:6379 \ redis-3.redis-headless.redis-cluster.svc:6379 \ redis-4.redis-headless.redis-cluster.svc:6379 \ redis-5.redis-headless.redis-cluster.svc:6379 \ --cluster-replicas 1 \ -a YourStrongPassword \ --cluster-yes restartPolicy: OnFailure关键参数说明--cluster-replicas 1每个主节点配1个从节点-a指定配置的认证密码--cluster-yes自动确认集群配置查看初始化日志kubectl logs -n redis-cluster -l job-nameredis-init-cluster4. Redis Proxy部署实践4.1 Proxy部署配置推荐使用官方redis-cluster-proxy# redis-cluster-proxy-deploy.yaml apiVersion: apps/v1 kind: Deployment metadata: name: redis-proxy namespace: redis-cluster spec: replicas: 2 # 建议至少2个实例 selector: matchLabels: app: redis-proxy template: metadata: labels: app: redis-proxy spec: containers: - name: proxy image: redis/redis-cluster-proxy:1.0 args: - --auth # 代理认证密码 - ProxyPassword123 - redis-0.redis-headless.redis-cluster.svc:6379 - redis-1.redis-headless.redis-cluster.svc:6379 - redis-2.redis-headless.redis-cluster.svc:6379 ports: - containerPort: 7777 readinessProbe: tcpSocket: port: 7777 initialDelaySeconds: 5 periodSeconds: 10 --- apiVersion: v1 kind: Service metadata: name: redis-proxy namespace: redis-cluster spec: type: LoadBalancer selector: app: redis-proxy ports: - port: 7777 targetPort: 77774.2 功能验证测试通过Proxy写入数据kubectl exec -it -n redis-cluster redis-proxy-xxx -- redis-cli -p 7777 -a ProxyPassword123 127.0.0.1:7777 SET user:1001 Alice OK直接查询集群节点验证数据分布kubectl exec -it -n redis-cluster redis-0 -- redis-cli -a YourStrongPassword -c 127.0.0.1:6379 GET user:1001 - Redirected to slot [14982] located at 10.244.2.15:6379 Alice5. 运维监控与自动恢复5.1 健康检查配置在StatefulSet中添加探针livenessProbe: exec: command: - redis-cli - -a - YourStrongPassword - ping initialDelaySeconds: 30 periodSeconds: 10 readinessProbe: exec: command: - redis-cli - -a - YourStrongPassword - ping initialDelaySeconds: 5 periodSeconds: 55.2 监控方案建议Prometheus监控annotations: prometheus.io/scrape: true prometheus.io/port: 9121 # redis_exporter端口关键监控指标集群状态redis_cluster_state内存使用redis_memory_used_bytes延迟统计redis_latency_percentiles_usec5.3 自动故障恢复当节点故障时K8s会自动重启Pod但需要注意主节点故障后需要等待集群完成故障转移约15秒从节点升级为新主节点后需要确保新副本同步完成网络分区时需要人工介入处理脑裂情况建议配置HPA实现Proxy层自动扩缩容apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: redis-proxy-hpa namespace: redis-cluster spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: redis-proxy minReplicas: 2 maxReplicas: 5 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 60