建站
咨詢
售后
建站咨詢
新聞中心
這里有您想知道的互聯(lián)網(wǎng)營(yíng)銷解決方案
微服務(wù)CI/CD實(shí)踐-GitOps完整設(shè)計(jì)與實(shí)現(xiàn)
微服務(wù)CI/cd實(shí)踐-GitOps完整設(shè)計(jì)與實(shí)現(xiàn)
作者：Lizeyang 2021-09-07 08:23:45
云計(jì)算 當(dāng)我在Gitlab提交了代碼，會(huì)通過(guò)GitLab webhook 觸發(fā)Jenkins Scheduler 作業(yè)， 會(huì)將此次提交代碼所產(chǎn)生的hook data數(shù)據(jù)信息以POST的方式傳給Jenkins Job。
專注于為中小企業(yè)提供成都網(wǎng)站制作、成都網(wǎng)站設(shè)計(jì)服務(wù),電腦端+手機(jī)端+微信端的三站合一,更高效的管理,為中小企業(yè)川匯免費(fèi)做網(wǎng)站提供優(yōu)質(zhì)的服務(wù)。我們立足成都，凝聚了一批互聯(lián)網(wǎng)行業(yè)人才，有力地推動(dòng)了上千多家企業(yè)的穩(wěn)健成長(zhǎng)，幫助中小企業(yè)通過(guò)網(wǎng)站建設(shè)實(shí)現(xiàn)規(guī)模擴(kuò)充和轉(zhuǎn)變。
單應(yīng)用與環(huán)境
多應(yīng)用與環(huán)境
CI持續(xù)集成
首先，準(zhǔn)備一個(gè)代碼庫(kù)：
https://github.com/DevOpsCICDCourse/microservicescicd/blob/main/microservice-demo-service-master.zip
我們來(lái)梳理一下CI流水線的步驟：
- 由于此次實(shí)現(xiàn)的代碼倉(cāng)庫(kù)類型為單一存儲(chǔ)庫(kù)，即一個(gè)存儲(chǔ)庫(kù)存放多個(gè)服務(wù)模塊代碼，每個(gè)子目錄為一個(gè)服務(wù)模塊。
- 首先，我們的持續(xù)集成流水線需要能夠正確獲取，當(dāng)前的commit是哪個(gè)服務(wù)的代碼。
- 確定好服務(wù)，然后下載該服務(wù)的代碼，進(jìn)行編譯打包、單元測(cè)試、代碼掃描和構(gòu)建鏡像等步驟。
如何獲取commit的服務(wù)信息?這里我們使用GitLab WebHook功能和Jenkins 的job 構(gòu)建觸發(fā)器對(duì)接來(lái)實(shí)現(xiàn)。
工作流程是：當(dāng)我在Gitlab提交了代碼，會(huì)通過(guò)GitLab webhook 觸發(fā)Jenkins Scheduler 作業(yè)， 會(huì)將此次提交代碼所產(chǎn)生的hook data數(shù)據(jù)信息以POST的方式傳給Jenkins Job。此時(shí)Jenkins job可以編寫使用Generic Hook插件獲取此次POST請(qǐng)求傳輸過(guò)來(lái)的請(qǐng)求體Body信息。是一段JSON數(shù)據(jù)， 該job運(yùn)行后編寫Pipeline 解析JSON中的數(shù)據(jù)拿到所變更的服務(wù)模塊信息。最后觸發(fā)對(duì)應(yīng)服務(wù)的CI作業(yè)進(jìn)行構(gòu)建。
CI-Scheduler 作業(yè)
此作業(yè)只需要開(kāi)啟webhook， 配置觸發(fā)token(唯一性)。生成hookurl：http://jenkins.idevops.site/generic-webhook-trigger/invoke?token=microservicecicd-scheduler-CI
Jenkinsfile
- pipeline {
- agent any
- stages{
- stage("GetData"){
- steps{
- script {
- echo "${webHookData}"
- data = readJSON text: "${webHookData}"
- println(data)
- env.branchName = data.ref - "refs/heads/"
- env.commitId = data.checkout_sha
- env.projectId = data.project_id
- commits = data["commits"]
- println("${env.branchName}")
- println("${env.commitID}")
- println("${env.projectId}")
- //env.moduleName = "service01"
- changeServices = []
- for(commit in commits) {
- println(commit.id)
- //added
- for (add in commit.added) {
- s = add.split("/") as List
- if (s.size() > 1){
- if (changeServices.indexOf(s[0]) == -1){
- changeServices.add(s[0])
- }
- }
- }
- //modified
- for (m in commit.modified) {
- s = m.split("/") as List
- // println s
- // println s.size()
- // println s[0]
- if (s.size() > 1){
- // println changeServices.indexOf(s[0])
- if (changeServices.indexOf(s[0]) == -1){
- changeServices.add(s[0])
- }
- }
- }
- //removed
- for (r in commit.removed) {
- s = r.split("/") as List
- println s
- if (s.size() > 1){
- if (changeServices.indexOf(s[0]) == -1){
- changeServices.add(s[0])
- }
- }
- }
- }
- println(changeServices)
- //currentBuild.description = " Trigger by ${eventType} ${changeServices}
- }
- }
- }
- stage('DefineService') {
- steps {
- script{
- println(changeServices)
- //服務(wù)構(gòu)建順序控制
- services = ['service02', 'service01']
- for (service in services){
- if (changeServices.indexOf(service) != -1){
- jobName = 'microservicecicd-'+service+'-service-CI'
- build job: jobName, wait: false, parameters: [string(name: 'branchName', value: "${env.branchName}" ),
- string(name: 'commitId', value: "${env.commitId}" ),
- string(name: 'projectId', value: "${env.projectId}" )]
- }
- }
- }
- }
- }
- }
- }
GitLab 配置WebHook
開(kāi)啟webhook，配置hookurl：http://jenkins.idevops.site/generic-webhook-trigger/invoke?token=microservicecicd-scheduler-CI
CI流水線-CI作業(yè)
每個(gè)微服務(wù)創(chuàng)建一個(gè)CI作業(yè)，具有三個(gè)字符串參數(shù)：分支名稱、commitID、項(xiàng)目ID。
Jenkinsfile
- String branchName = "${env.branchName}"
- String moduleName = "${JOB_NAME}".split("/")[1].split("-")[1]
- String srcUrl = "http://gitlab.idevops.site/microservicecicd/microservicecicd-demo-service.git"
- String commitId = "${env.commitId}"
- String projectId = "${env.projectId}"
- pipeline {
- agent { node { label "build" } }
- stages {
- stage('GetCode') {
- steps {
- script {
- checkout([$class: 'GitSCM',
- branches: [[name: "${branchName}"]],
- doGenerateSubmoduleConfigurations: false,
- extensions: [[$class: 'SparseCheckoutPaths',
- sparseCheckoutPaths: [[path: "${moduleName}"],[path: 'Dockerfile']]]],
- submoduleCfg: [],
- userRemoteConfigs: [[credentialsId: 'gitlab-admin-user',
- url: "${srcUrl}"]]])
- }
- }
- }
- stage("Build&Test"){
- steps{
- script{
- echo "Build..........."
- sh """
- cd ${moduleName}
- mvn clean package
- """
- }
- }
- post {
- always {
- junit "${moduleName}/target/surefire-reports/*.xml"
- }
- }
- }
- stage("SonarScan"){
- steps{
- script{
- def sonarDate = sh returnStdout: true, script: 'date +%Y%m%d%H%M%S'
- sonarDate = sonarDate - "\n"
- withCredentials([string(credentialsId: 'sonar-admin-user', variable: 'sonartoken'),
- string(credentialsId: 'gitlab-user-token', variable: 'gitlabtoken')]) {
- // some block
- sh """
- cd ${moduleName}
- sonar-scanner \
- -Dsonar.projectKey=${JOB_NAME} \
- -Dsonar.projectName=${JOB_NAME} \
- -Dsonar.projectVersion=${sonarDate} \
- -Dsonar.ws.timeout=30 \
- -Dsonar.projectDescription="xxxxxxx" \
- -Dsonar.links.homepage=http://www.baidu.com \
- -Dsonar.sources=src \
- -Dsonar.sourceEncoding=UTF-8 \
- -Dsonar.java.binaries=target/classes \
- -Dsonar.java.test.binaries=target/test-classes \
- -Dsonar.java.surefire.report=target/surefire-reports \
- -Dsonar.host.url="http://sonar.idevops.site" \
- -Dsonar.login=${sonartoken} \
- -Dsonar.gitlab.commit_sha=${commitId} \
- -Dsonar.gitlab.ref_name=${branchName} \
- -Dsonar.gitlab.project_id=${projectId} \
- -Dsonar.dynamicAnalysis=reuseReports \
- -Dsonar.gitlab.failure_notification_mode=commit-status \
- -Dsonar.gitlab.url=http://gitlab.idevops.site \
- -Dsonar.gitlab.user_token=${gitlabtoken} \
- -Dsonar.gitlab.api_version=v4
- """
- }
- }
- }
- }
- stage("BuildImage"){
- steps{
- script{
- withCredentials([usernamePassword(credentialsId: 'aliyun-registry-admin', passwordVariable: 'password', usernameVariable: 'username')]) {
- env.nowDate = sh returnStdout: true, script: 'date +%Y%m%d%H%M%S'
- env.nowDate = env.nowDate - "\n"
- env.releaseVersion = "${env.branchName}"
- env.imageTag = "${releaseVersion}-${nowDate}-${commitId}"
- env.dockerImage = "registry.cn-beijing.aliyuncs.com/microservicecicd/microservicecicd-${moduleName}-service:${env.imageTag}"
- env.jarName = "${moduleName}-${branchName}-${commitId}"
- sh """
- docker login -u ${username} -p ${password} registry.cn-beijing.aliyuncs.com
- cd ${moduleName} && docker build -t ${dockerImage} -f ../Dockerfile --build-arg SERVICE_NAME=${jarName} .
- sleep 1
- docker push ${dockerImage}
- sleep 1
- docker rmi ${dockerImage}
- """
- }
- }
- }
- }
- }
- }
GitOps-CI擴(kuò)展部分
在原始CI作業(yè)的步驟基礎(chǔ)上，增加了一個(gè)更新環(huán)境的步驟。GitOps實(shí)踐會(huì)將當(dāng)前的基礎(chǔ)環(huán)境部署文件存放到一個(gè)Git倉(cāng)庫(kù)中。我們的CI作業(yè)在完成鏡像上傳后，同時(shí)更新環(huán)境部署文件中的鏡像標(biāo)簽信息。(所以我們需要先獲取該環(huán)境文件并更新上傳)
- stage("PushFile"){
- // when {
- // expression { "${env.branchName}".contains("RELEASE-") }
- // }
- steps{
- script{
- if ("${env.branchName}".contains("RELEASE-")){
- println("branchName = branchName")
- env.branchName = "master"
- } else {
- env.branchName = "feature"
- }
- for (i = 0; i < 3; i++) {
- //下載版本庫(kù)文件
- response = GetRepoFile(40,"${moduleName}%2fvalues.yaml", "${env.branchName}")
- //println(response)
- //替換文件中內(nèi)容
- yamlData = readYaml text: """${response}"""
- println(yamlData.image.version)
- println(yamlData.image.commit)
- yamlData.image.version = "${releaseVersion}-${env.nowDate}"
- yamlData.image.commit = "${commitId}"
- println(yamlData.toString())
- sh "rm -fr test.yaml"
- writeYaml charset: 'UTF-8', data: yamlData, file: 'test.yaml'
- newYaml = sh returnStdout: true, script: 'cat test.yaml'
- println(newYaml)
- //更新gitlab文件內(nèi)容
- base64Content = newYaml.bytes.encodeBase64().toString()
- // 會(huì)有并行問(wèn)題，同時(shí)更新報(bào)錯(cuò)
- try {
- UpdateRepoFile(40,"${moduleName}%2fvalues.yaml",base64Content, "${env.branchName}")
- break;
- } catch(e){
- sh "sleep 2"
- continue;
- }
- }
- }
- }
- }
- //封裝HTTP請(qǐng)求
- def HttpReq(reqType,reqUrl,reqBody){
- def gitServer = "http://gitlab.idevops.site/api/v4"
- withCredentials([string(credentialsId: 'gitlab-token', variable: 'gitlabToken')]) {
- result = httpRequest customHeaders: [[maskValue: true, name: 'PRIVATE-TOKEN', value: "${gitlabToken}"]],
- httpMode: reqType,
- contentType: "APPLICATION_JSON",
- consoleLogResponseBody: true,
- ignoreSslErrors: true,
- requestBody: reqBody,
- url: "${gitServer}/${reqUrl}"
- //quiet: true
- }
- return result
- }
- //獲取文件內(nèi)容
- def GetRepoFile(projectId,filePath,branchName){
- apiUrl = "projects/${projectId}/repository/files/${filePath}/raw?ref=${branchName}"
- response = HttpReq('GET',apiUrl,'')
- return response.content
- }
- //更新文件內(nèi)容
- def UpdateRepoFile(projectId,filePath,fileContent, branchName){
- apiUrl = "projects/${projectId}/repository/files/${filePath}"
- reqBody = """{"branch": "${branchName}","encoding":"base64", "content": "${fileContent}", "commit_message": "update a new file"}"""
- response = HttpReq('PUT',apiUrl,reqBody)
- println(response)
- }
images
GitOps-CD部分
CD-Scheduler作業(yè)
此作業(yè)其實(shí)也是接收GitLab的webhook請(qǐng)求， 與CI-scheduler作業(yè)類似。不同的是這個(gè)CD-scheduler作業(yè)是用來(lái)接收環(huán)境倉(cāng)庫(kù)的代碼變更。開(kāi)啟webhook， 配置觸發(fā)token。生成hookurl：http://jenkins.idevops.site/generic-webhook-trigger/invoke?token=microservicecicd-scheduler-CD
Jenkinsfile
- pipeline {
- agent any
- stages {
- stage('GetCommitService') {
- steps {
- script{
- echo 'Hello World'
- echo "${WebHookData}"
- // Git Info
- webhookdata = readJSON text: """${WebHookData}"""
- eventType = webhookdata["object_kind"]
- commits = webhookdata["commits"]
- branchName = webhookdata["ref"] - "refs/heads/"
- projectID = webhookdata["project_id"]
- commitID = webhookdata["checkout_sha"]
- changeServices = []
- for(commit in commits) {
- println(commit.id)
- //added
- for (add in commit.added) {
- s = add.split("/") as List
- if (s.size() > 1){
- if (changeServices.indexOf(s[0]) == -1){
- changeServices.add(s[0])
- }
- }
- }
- //modified
- for (m in commit.modified) {
- s = m.split("/") as List
- // println s
- // println s.size()
- // println s[0]
- if (s.size() > 1){
- // println changeServices.indexOf(s[0])
- if (changeServices.indexOf(s[0]) == -1){
- changeServices.add(s[0])
- }
- }
- }
- //removed
- for (r in commit.removed) {
- s = r.split("/") as List
- println s
- if (s.size() > 1){
- &nbs
名稱欄目：微服務(wù)CI/CD實(shí)踐-GitOps完整設(shè)計(jì)與實(shí)現(xiàn)
文章位置：http://www.5511xx.com/article/dhosjsc.html
