第2章：Jenkins与GitLab CI
Jenkins完整实战
安装Jenkins
方式1：Docker安装（推荐）：
# 1. 拉取Jenkins镜像
docker pull jenkins/jenkins:lts
# 2. 创建数据目录
mkdir -p /data/jenkins_home
chmod 777 /data/jenkins_home
# 3. 运行Jenkins
docker run -d \
--name jenkins \
-p 8080:8080 \
-p 50000:50000 \
-v /data/jenkins_home:/var/jenkins_home \
-v /var/run/docker.sock:/var/run/docker.sock \
jenkins/jenkins:lts
# 4. 获取初始密码
docker exec jenkins cat /var/jenkins_home/secrets/initialAdminPassword
方式2：Kubernetes安装：
# jenkins-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: jenkins
namespace: devops
spec:
replicas: 1
selector:
matchLabels:
app: jenkins
template:
metadata:
labels:
app: jenkins
spec:
containers:
- name: jenkins
image: jenkins/jenkins:lts
ports:
- containerPort: 8080
- containerPort: 50000
volumeMounts:
- name: jenkins-home
mountPath: /var/jenkins_home
volumes:
- name: jenkins-home
persistentVolumeClaim:
claimName: jenkins-pvc
---
apiVersion: v1
kind: Service
metadata:
name: jenkins
namespace: devops
spec:
type: NodePort
ports:
- port: 8080
targetPort: 8080
nodePort: 30080
selector:
app: jenkins
# 部署
kubectl apply -f jenkins-deployment.yaml
# 访问
# http://<node-ip>:30080
初始化配置
1. 安装插件：
推荐插件：
Git plugin
Pipeline plugin
Docker plugin
Kubernetes plugin
Blue Ocean（现代化UI）
GitHub/GitLab plugin
2. 配置凭据：
Jenkins → 系统管理 → 凭据管理 → 添加凭据
类型：
1. Username with password（Git账号）
2. SSH Username with private key（SSH密钥）
3. Secret text（API Token）
4. Secret file（配置文件）
3. 配置工具：
Jenkins → 系统管理 → 全局工具配置
配置：
- JDK
- Git
- Maven
- Gradle
- Docker
创建第一个Pipeline
步骤：
1. 新建Item → Pipeline
2. 名称：myapp-pipeline
3. Pipeline script:
// Jenkinsfile（声明式）
pipeline {
agent any
stages {
stage('Checkout') {
steps {
git branch: 'main',
url: 'https://github.com/example/myapp.git'
}
}
stage('Build') {
steps {
sh 'go build -o bin/myapp'
}
}
stage('Test') {
steps {
sh 'go test -v ./...'
}
}
stage('Docker Build') {
steps {
sh 'docker build -t myapp:${BUILD_NUMBER} .'
}
}
stage('Deploy') {
steps {
sh '''
docker stop myapp || true
docker rm myapp || true
docker run -d --name myapp -p 8080:8080 myapp:${BUILD_NUMBER}
'''
}
}
}
post {
success {
echo 'Pipeline succeeded!'
}
failure {
echo 'Pipeline failed!'
}
}
}
Jenkinsfile Pipeline
1. 声明式 vs 脚本式
声明式Pipeline（推荐）：
pipeline {
agent any
environment {
APP_NAME = 'myapp'
DOCKER_REGISTRY = 'harbor.example.com'
}
stages {
stage('Build') {
steps {
sh 'go build'
}
}
}
}
脚本式Pipeline：
node {
def appName = 'myapp'
stage('Build') {
sh 'go build'
}
}
2. 完整的生产级Pipeline
Jenkinsfile：
pipeline {
agent {
kubernetes {
yaml '''
apiVersion: v1
kind: Pod
spec:
containers:
- name: golang
image: golang:1.21
command: ['cat']
tty: true
- name: docker
image: docker:latest
command: ['cat']
tty: true
volumeMounts:
- name: docker-sock
mountPath: /var/run/docker.sock
volumes:
- name: docker-sock
hostPath:
path: /var/run/docker.sock
'''
}
}
environment {
APP_NAME = 'myapp'
DOCKER_REGISTRY = credentials('docker-registry')
GIT_COMMIT_SHORT = sh(
script: "git rev-parse --short HEAD",
returnStdout: true
).trim()
VERSION = "${env.BUILD_NUMBER}-${GIT_COMMIT_SHORT}"
}
parameters {
choice(
name: 'ENVIRONMENT',
choices: ['dev', 'test', 'staging', 'production'],
description: '选择部署环境'
)
booleanParam(
name: 'RUN_TESTS',
defaultValue: true,
description: '是否运行测试'
)
}
options {
buildDiscarder(logRotator(numToKeepStr: '10'))
timeout(time: 1, unit: 'HOURS')
timestamps()
}
stages {
stage('Checkout') {
steps {
checkout scm
sh '''
git log -1 --pretty=format:"%h - %an, %ar : %s"
'''
}
}
stage('Build') {
steps {
container('golang') {
sh '''
go mod download
CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o bin/${APP_NAME} .
'''
}
}
}
stage('Test') {
when {
expression { params.RUN_TESTS == true }
}
parallel {
stage('Unit Test') {
steps {
container('golang') {
sh 'go test -v -cover ./...'
}
}
}
stage('Integration Test') {
steps {
container('golang') {
sh 'go test -tags=integration -v ./...'
}
}
}
}
}
stage('Code Quality') {
steps {
script {
def scannerHome = tool 'SonarQubeScanner'
withSonarQubeEnv('SonarQube') {
sh "${scannerHome}/bin/sonar-scanner"
}
}
}
}
stage('Quality Gate') {
steps {
timeout(time: 1, unit: 'HOURS') {
waitForQualityGate abortPipeline: true
}
}
}
stage('Docker Build') {
steps {
container('docker') {
sh '''
docker build -t ${DOCKER_REGISTRY}/${APP_NAME}:${VERSION} .
docker tag ${DOCKER_REGISTRY}/${APP_NAME}:${VERSION} ${DOCKER_REGISTRY}/${APP_NAME}:latest
'''
}
}
}
stage('Security Scan') {
steps {
container('docker') {
sh '''
docker run --rm \
-v /var/run/docker.sock:/var/run/docker.sock \
aquasec/trivy:latest image \
--severity HIGH,CRITICAL \
${DOCKER_REGISTRY}/${APP_NAME}:${VERSION}
'''
}
}
}
stage('Push Image') {
steps {
container('docker') {
sh '''
echo ${DOCKER_REGISTRY_PSW} | docker login ${DOCKER_REGISTRY} -u ${DOCKER_REGISTRY_USR} --password-stdin
docker push ${DOCKER_REGISTRY}/${APP_NAME}:${VERSION}
docker push ${DOCKER_REGISTRY}/${APP_NAME}:latest
'''
}
}
}
stage('Deploy') {
when {
expression { params.ENVIRONMENT != 'production' }
}
steps {
sh """
kubectl set image deployment/${APP_NAME} \
${APP_NAME}=${DOCKER_REGISTRY}/${APP_NAME}:${VERSION} \
-n ${params.ENVIRONMENT}
kubectl rollout status deployment/${APP_NAME} -n ${params.ENVIRONMENT}
"""
}
}
stage('Deploy Production') {
when {
expression { params.ENVIRONMENT == 'production' }
}
steps {
input message: '确认部署到生产环境？', ok: '确认'
sh """
kubectl set image deployment/${APP_NAME} \
${APP_NAME}=${DOCKER_REGISTRY}/${APP_NAME}:${VERSION} \
-n production
kubectl rollout status deployment/${APP_NAME} -n production
"""
}
}
stage('Health Check') {
steps {
script {
def retries = 0
def maxRetries = 30
while (retries < maxRetries) {
try {
sh "curl -f http://${APP_NAME}.${params.ENVIRONMENT}.svc.cluster.local/health"
break
} catch (Exception e) {
retries++
if (retries >= maxRetries) {
error "健康检查失败，自动回滚"
sh "kubectl rollout undo deployment/${APP_NAME} -n ${params.ENVIRONMENT}"
}
sleep 10
}
}
}
}
}
}
post {
success {
echo " Pipeline succeeded for ${params.ENVIRONMENT} environment!"
// 发送通知（Slack、Email、钉钉）
}
failure {
echo " Pipeline failed!"
// 发送告警
}
always {
cleanWs()
}
}
}
3. 共享库（Shared Library）
创建共享库：
jenkins-shared-library/
├── vars/
│ ├── buildGo.groovy
│ ├── deployK8s.groovy
│ └── sendNotification.groovy
└── src/
└── org/
└── example/
└── Utils.groovy
vars/buildGo.groovy：
def call(Map config) {
pipeline {
agent any
stages {
stage('Build') {
steps {
sh """
go mod download
go build -o bin/${config.appName}
"""
}
}
stage('Test') {
steps {
sh 'go test -v ./...'
}
}
}
}
}
使用共享库：
@Library('jenkins-shared-library') _
buildGo(
appName: 'myapp',
version: '1.0.0'
)
GitLab CI实战
1. 基础配置
.gitlab-ci.yml：
# GitLab CI配置文件
image: golang:1.21
stages:
- build
- test
- package
- deploy
variables:
APP_NAME: myapp
DOCKER_REGISTRY: harbor.example.com
DOCKER_IMAGE: ${DOCKER_REGISTRY}/${APP_NAME}
before_script:
- echo "Starting CI/CD pipeline..."
build:
stage: build
script:
- go mod download
- go build -o bin/${APP_NAME}
artifacts:
paths:
- bin/${APP_NAME}
expire_in: 1 hour
only:
- main
- develop
test:
stage: test
script:
- go test -v -cover ./...
coverage: '/coverage: \d+.\d+% of statements/'
only:
- main
- develop
docker-build:
stage: package
image: docker:latest
services:
- docker:dind
script:
- docker login $DOCKER_REGISTRY -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD
- docker build -t ${DOCKER_IMAGE}:${CI_COMMIT_SHORT_SHA} .
- docker push ${DOCKER_IMAGE}:${CI_COMMIT_SHORT_SHA}
only:
- main
deploy-dev:
stage: deploy
image: bitnami/kubectl:latest
script:
- kubectl set image deployment/${APP_NAME} ${APP_NAME}=${DOCKER_IMAGE}:${CI_COMMIT_SHORT_SHA} -n dev
- kubectl rollout status deployment/${APP_NAME} -n dev
environment:
name: dev
url: https://dev.example.com
only:
- develop
deploy-prod:
stage: deploy
image: bitnami/kubectl:latest
script:
- kubectl set image deployment/${APP_NAME} ${APP_NAME}=${DOCKER_IMAGE}:${CI_COMMIT_SHORT_SHA} -n production
- kubectl rollout status deployment/${APP_NAME} -n production
environment:
name: production
url: https://example.com
when: manual
only:
- main
2. 高级特性
多阶段构建优化：
stages:
- build
- test
- security
- package
- deploy
# 缓存依赖
.go-cache:
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- /go/pkg/mod/
# 构建
build:
extends: .go-cache
stage: build
script:
- go build -o bin/myapp
# 并行测试
test:unit:
extends: .go-cache
stage: test
script:
- go test -v ./internal/...
test:integration:
extends: .go-cache
stage: test
script:
- go test -tags=integration -v ./...
test:e2e:
stage: test
script:
- npm run test:e2e
# 安全扫描
security:sast:
stage: security
script:
- gosec ./...
security:dependency:
stage: security
script:
- go list -json -m all | nancy sleuth
security:image:
stage: security
script:
- trivy image ${DOCKER_IMAGE}:${CI_COMMIT_SHORT_SHA}
动态环境：
deploy-review:
stage: deploy
script:
- |
cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
name: ${APP_NAME}-${CI_COMMIT_REF_SLUG}
namespace: review
spec:
replicas: 1
selector:
matchLabels:
app: ${APP_NAME}
review: ${CI_COMMIT_REF_SLUG}
template:
metadata:
labels:
app: ${APP_NAME}
review: ${CI_COMMIT_REF_SLUG}
spec:
containers:
- name: ${APP_NAME}
image: ${DOCKER_IMAGE}:${CI_COMMIT_SHORT_SHA}
ports:
- containerPort: 8080
EOF
environment:
name: review/${CI_COMMIT_REF_SLUG}
url: https://${CI_COMMIT_REF_SLUG}.review.example.com
on_stop: stop-review
only:
- branches
except:
- main
stop-review:
stage: deploy
script:
- kubectl delete deployment ${APP_NAME}-${CI_COMMIT_REF_SLUG} -n review
environment:
name: review/${CI_COMMIT_REF_SLUG}
action: stop
when: manual
3. 模板复用
创建模板（.gitlab-ci-templates.yml）：
.deploy-template:
image: bitnami/kubectl:latest
script:
- kubectl set image deployment/${APP_NAME} ${APP_NAME}=${DOCKER_IMAGE}:${CI_COMMIT_SHORT_SHA} -n ${ENVIRONMENT}
- kubectl rollout status deployment/${APP_NAME} -n ${ENVIRONMENT}
.test-template:
extends: .go-cache
script:
- go test -v ${TEST_PATH}
使用模板：
include:
- local: '.gitlab-ci-templates.yml'
test:unit:
extends: .test-template
variables:
TEST_PATH: ./internal/...
deploy-prod:
extends: .deploy-template
variables:
ENVIRONMENT: production
when: manual
Runner配置
1. 安装GitLab Runner
Docker安装：
# 1. 安装Runner
docker run -d --name gitlab-runner --restart always \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /srv/gitlab-runner/config:/etc/gitlab-runner \
gitlab/gitlab-runner:latest
# 2. 注册Runner
docker exec -it gitlab-runner gitlab-runner register
# 交互式配置：
# GitLab URL: https://gitlab.example.com
# Token: xxx（从GitLab项目设置获取）
# Description: docker-runner
# Tags: docker, linux
# Executor: docker
# Default Image: alpine:latest
Kubernetes安装：
# gitlab-runner.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: gitlab-runner
namespace: gitlab
data:
config.toml: |
concurrent = 10
check_interval = 3
[[runners]]
name = "kubernetes-runner"
url = "https://gitlab.example.com"
token = "RUNNER_TOKEN"
executor = "kubernetes"
[runners.kubernetes]
namespace = "gitlab"
image = "alpine:latest"
privileged = true
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: gitlab-runner
namespace: gitlab
spec:
replicas: 1
selector:
matchLabels:
app: gitlab-runner
template:
metadata:
labels:
app: gitlab-runner
spec:
containers:
- name: gitlab-runner
image: gitlab/gitlab-runner:latest
volumeMounts:
- name: config
mountPath: /etc/gitlab-runner
volumes:
- name: config
configMap:
name: gitlab-runner
2. Runner类型
Shared Runner（共享Runner）：
优点：
多项目共享
易于管理
弹性扩容
缺点：
✗ 资源竞争
✗ 安全性较低
Specific Runner（专用Runner）：
优点：
资源独占
安全性高
可定制环境
缺点：
✗ 维护成本高
✗ 资源利用率低
3. Executor类型
Docker Executor：
[[runners]]
name = "docker-runner"
executor = "docker"
[runners.docker]
image = "golang:1.21"
privileged = false
volumes = ["/cache"]
Kubernetes Executor：
[[runners]]
name = "kubernetes-runner"
executor = "kubernetes"
[runners.kubernetes]
namespace = "gitlab"
image = "golang:1.21"
privileged = true
cpu_limit = "1"
memory_limit = "2Gi"
性能优化
1. 并行执行
Jenkins：
stage('Test') {
parallel {
stage('Unit Test') {
steps {
sh 'go test ./internal/...'
}
}
stage('Integration Test') {
steps {
sh 'go test -tags=integration ./...'
}
}
stage('E2E Test') {
steps {
sh 'npm run test:e2e'
}
}
}
}
GitLab CI：
test:unit:
stage: test
script:
- go test ./internal/...
test:integration:
stage: test
script:
- go test -tags=integration ./...
test:e2e:
stage: test
script:
- npm run test:e2e
# 三个测试任务自动并行执行
2. 缓存依赖
GitLab CI：
build:
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- vendor/
- node_modules/
- .m2/repository/
script:
- go build
Jenkins：
stage('Build') {
steps {
cache(maxCacheSize: 1000, caches: [
arbitraryFileCache(
path: 'vendor',
cacheValidityDecidingFile: 'go.sum'
)
]) {
sh 'go build'
}
}
}
3. 增量构建
检测文件变更：
build:frontend:
script:
- |
if git diff --name-only $CI_COMMIT_BEFORE_SHA $CI_COMMIT_SHA | grep -q "^frontend/"; then
echo "前端代码变更，构建前端"
cd frontend && npm run build
else
echo "前端代码无变更，跳过构建"
fi
build:backend:
script:
- |
if git diff --name-only $CI_COMMIT_BEFORE_SHA $CI_COMMIT_SHA | grep -q "^backend/"; then
echo "后端代码变更，构建后端"
cd backend && go build
fi
4. 镜像优化
多阶段构建：
# 构建阶段
FROM golang:1.21 AS builder
WORKDIR /app
COPY go.* ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o myapp
# 运行阶段
FROM alpine:latest
RUN apk --no-cache add ca-certificates
COPY /app/myapp /myapp
CMD ["/myapp"]
镜像大小对比：
优化前：
golang:1.21 800MB
myapp:v1 820MB
优化后：
alpine:latest 5MB
myapp:v1 15MB
减少：98%
面试问答
Jenkins Pipeline声明式和脚本式有什么区别？
答案：
声明式Pipeline：
pipeline {
agent any
stages {
stage('Build') {
steps {
sh 'go build'
}
}
}
}
特点：
结构清晰，易读
语法简单
适合大多数场景
灵活性较低
脚本式Pipeline：
node {
stage('Build') {
sh 'go build'
}
}
特点：
灵活性高
可使用Groovy语法
适合复杂逻辑
语法复杂
易出错
选择建议：
推荐声明式：
标准CI/CD流程
团队协作
易于维护
使用脚本式：
复杂条件判断
动态生成Pipeline
高级自定义
如何优化CI/CD流水线的速度？
答案：
1. 并行执行：
# 并行测试
stages:
- test
test:unit:
stage: test
script: go test ./internal/...
test:integration:
stage: test
script: go test -tags=integration ./...
# 自动并行，节省50%时间
2. 缓存依赖：
build:
cache:
paths:
- vendor/
- node_modules/
script:
- go build
# 首次构建：5分钟
# 缓存后：1分钟
3. 增量构建：
# 只构建变更的模块
if [ "$CHANGED_MODULE" = "frontend" ]; then
npm run build
fi
4. Docker层缓存：
# 先复制依赖文件
COPY go.mod go.sum ./
RUN go mod download
# 后复制源码
COPY . .
RUN go build
# 依赖未变化时，复用缓存层
5. 使用更快的Runner：
build:
tags:
- high-performance
- docker
效果对比：
优化前：30分钟
优化后：10分钟
提速：3倍
GitLab CI和Jenkins有什么区别？如何选择？
答案：
| 维度 | Jenkins | GitLab CI |
|---|---|---|
| 集成性 | 需要配置集成 | 与GitLab深度集成 |
| 配置 | Jenkinsfile | .gitlab-ci.yml |
| 插件 | 丰富（2000+） | 内置功能 |
| 学习曲线 | 陡峭 | 平缓 |
| 灵活性 | 极高 | 中等 |
| UI | 传统/Blue Ocean | 现代化 |
| 适用场景 | 复杂流程 | 标准CI/CD |
选择建议：
选择Jenkins：
多代码仓库管理
复杂的自定义流程
需要大量插件
团队有Jenkins经验
选择GitLab CI：
使用GitLab管理代码
标准CI/CD流程
快速上手
一体化DevOps平台
如何保证CI/CD的安全性？
答案：
1. 凭据管理：
// Jenkins - 使用凭据
withCredentials([
usernamePassword(
credentialsId: 'docker-registry',
usernameVariable: 'USER',
passwordVariable: 'PASS'
)
]) {
sh 'docker login -u $USER -p $PASS'
}
// 不要在代码中硬编码密码！
2. 镜像扫描：
security:scan:
stage: security
script:
- trivy image myapp:latest --severity HIGH,CRITICAL
- |
if [ $? -ne 0 ]; then
echo "发现高危漏洞，停止部署"
exit 1
fi
3. 代码签名：
# 签名Docker镜像
export DOCKER_CONTENT_TRUST=1
docker push myapp:v1.0.0
4. 最小权限原则：
# Kubernetes RBAC
apiVersion: v1
kind: ServiceAccount
metadata:
name: gitlab-runner
namespace: gitlab
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: gitlab-runner-role
namespace: production
rules:
- apiGroups: ["apps"]
resources: ["deployments"]
verbs: ["get", "update", "patch"] # 仅允许更新部署
5. 审计日志：
记录：
- 谁触发了Pipeline
- 何时部署到生产环境
- 部署了哪个版本
- 是否通过了所有检查
如何处理CI/CD Pipeline的失败？
答案：
1. 自动重试：
// Jenkins
stage('Test') {
retry(3) {
sh 'go test ./...'
}
}
# GitLab CI
test:
script:
- go test ./...
retry:
max: 3
when:
- runner_system_failure
- stuck_or_timeout_failure
2. 失败通知：
post {
failure {
emailext(
subject: "Pipeline失败: ${env.JOB_NAME}",
body: "构建 ${env.BUILD_NUMBER} 失败",
to: 'team@example.com'
)
// Slack通知
slackSend(
channel: '#devops',
color: 'danger',
message: "Pipeline失败: ${env.JOB_NAME} #${env.BUILD_NUMBER}"
)
}
}
3. 自动回滚：
deploy:
script:
- kubectl apply -f deployment.yaml
- |
# 健康检查
for i in {1..30}; do
if curl -f http://myapp/health; then
echo "部署成功"
exit 0
fi
sleep 10
done
echo "健康检查失败，自动回滚"
kubectl rollout undo deployment/myapp
exit 1
4. 失败分析：
常见失败原因：
1. 测试失败 → 修复代码
2. 构建超时 → 优化构建脚本
3. 网络问题 → 重试
4. 资源不足 → 增加Runner资源
5. 依赖下载失败 → 使用缓存/私有仓库