gitlab artifacts过期时间查看和清理

artifacts过期时间查看

• artifacts的默认过期时间是30天，可以通过gitlab UI界面查看

Main menu > Admin** > Settings > CI/CD > Continuous Integration and Deployment > Default artifacts expiration

• 在gitlab CI pipeline中也可以通过expire_in参数来设置artifacts的过期时间

• 在数据库中也可以查看artifacts的过期时间

  select project_id,size,job_id,created_at,updated_at,expire_at from ci_job_artifacts where project_id=xxx;

通过rails 控制台操作artifacts文件

• 查看未过期的artifacts(实例级别)

  builds_with_artifacts_that_never_expire = Ci::Build.with_downloadable_artifacts.where(artifacts_expire_at: nil).limit(50)
  builds_with_artifacts_that_never_expire.find_each do |build|
    puts "Build with id #{build.id} has artifacts that don't expire and belongs to project #{build.project.full_path}"
  end
  

• 查找从今天起7天后过期的构件的构建和项目

  builds_with_artifacts_that_expire_in_a_week = Ci::Build.with_downloadable_artifacts.where('artifacts_expire_at > ?', 7.days.from_now).limit(50)
  builds_with_artifacts_that_expire_in_a_week.find_each do |build|
    puts "Build with id #{build.id} has artifacts that expire at #{build.artifacts_expire_at} and belongs to project #{build.project.full_path}"
  end
  

• 查看artifacts文件大小的前20个项目

  include ActionView::Helpers::NumberHelper
  ProjectStatistics.order(build_artifacts_size: :desc).limit(20).each do |s|
    puts "#{number_to_human_size(s.build_artifacts_size)} \t #{s.project.full_path}"
  end
  

• 查看单个项目的前50个artifacts文件大小

  include ActionView::Helpers::NumberHelper
  project = Project.find_by_full_path('ops/test-2')
  Ci::JobArtifact.where(project: project).order(size: :desc).limit(50).map { |a| puts "ID: #{a.id} - #{a.file_type}: #{number_to_human_size(a.size)}" }
  

• 查看单个项目的前5个artifacts文件信息

  p = Project.find_by_id(2)
  arts = Ci::JobArtifact.where(project: p)
  
  list = arts.order(size: :desc).limit(5).each do |art|
      puts "Job ID: #{art.job_id} - Size: #{art.size}b - Type: #{art.file_type} - Created: #{art.created_at} - File loc: #{art.file}"
  end
  

• 删除早于特定日期的作业artifacts(保留job_log)（整个实例）

  # 清理1天以内的artifacts
  project = Project.find_by_full_path('ops/test-3')
  builds_with_artifacts =  project.builds.with_downloadable_artifacts
  
  builds_to_clear = builds_with_artifacts.where("finished_at > ?", 1.days.ago)
  builds_to_clear.find_each do |build|
    Ci::JobArtifacts::DeleteService.new(build).execute
    build.update!(artifacts_expire_at: Time.now)
  end
  
  
  # 清理10分钟以内的artifacts
  project = Project.find_by_full_path('ops/test-3')
  builds_with_artifacts =  project.builds.with_downloadable_artifacts
  
  builds_to_clear = builds_with_artifacts.where("finished_at > ?", 20.minutes.ago)
  builds_to_clear.find_each do |build|
    print "Ci::Build ID #{build.id}... "
    Ci::JobArtifacts::DeleteService.new(build).execute
    build.update!(artifacts_expire_at: Time.now)
  end
  

• 删除早于特定日期的作业artifacts(保留job_log)（整个项目）

  # 清理1天以内的artifacts
  project = Project.find_by_full_path('ops/test-3')
  builds_with_artifacts = Ci::Build.with_downloadable_artifacts
  
  builds_to_clear = builds_with_artifacts.where("finished_at < ?", 2.days.ago)
  builds_to_clear.find_each do |build|
    print "Ci::Build ID #{build.id}... "
    Ci::JobArtifacts::DeleteService.new(build).execute
    build.update!(artifacts_expire_at: Time.now)
  end
  

• 从特定日期前完成的作业中删除作业artifacts和日志（整个实例）

  # 清理1小时以内的artifacts
  builds_with_artifacts = Ci::Build.with_existing_job_artifacts(Ci::JobArtifact.trace)
  admin_user = User.find_by(username: 'root')
  
  builds_to_clear = builds_with_artifacts.where("finished_at > ?", 1.hours.ago)
  builds_to_clear.find_each do |build|
    print "Ci::Build ID #{build.id}... "
  
    if build.erasable?
      Ci::BuildEraseService.new(build, admin_user).execute
      puts "Erased"
    else
      puts "Skipped (Nothing to erase or not erasable)"
    end
  end
  

• 从特定日期前完成的作业中删除作业artifacts和日志（整个项目）

  # 清理5小时之内的artifacts
  project = Project.find_by_full_path('ops/test-2')
  builds_with_artifacts =  project.builds.with_existing_job_artifacts(Ci::JobArtifact.trace)
  admin_user = User.find_by(username: 'root')
  
  builds_to_clear = builds_with_artifacts.where("finished_at > ?", 10.minutes.ago) 
  builds_to_clear.find_each do |build|
    print "Ci::Build ID #{build.id}... "
  
    if build.erasable?
      Ci::BuildEraseService.new(build, admin_user).execute
      puts "Erased"
    else
      puts "Skipped (Nothing to erase or not erasable)"
    end
  end