Travis CI build status CircleCI build status CodeFactor Documentation Status image version pypi.org version pypi.org pyversions

Gitlab Bulk Checkout Tool

Simple utitlity written in Python for the local repositories. In a Time of SaaS, Isac and microservices, it can happen that you need many small repositories at your local Machine for Development, so it didn`t make fun to pull so many repositories.

Features

  • sort gitlab grops to fs folder
  • append a prefix to the repos
  • pull changes from origin

Using Tox Build Script

source ~/development/lib/virtualenv/tox/bin/activate
tox

The tox script create a installable dist under ./.tox/dist/*.tar.gz

Docker

For easy usage, the tool can be wrapped with docker, so you don`t need a local python env for using.

Build

docker build -t nolte/gitlab-bulkcheckout .

ssh agent https://kb.iu.edu/d/aeww

Usage

docker run -it \
  --user=${UID}:$(id -g $(whoami)) \
  -v $SSH_AUTH_SOCK:/ssh-agent \
  -e SSH_AUTH_SOCK=/ssh-agent \
  -e GROUPS_MAPPINGS=/app/gitlab_groupMapping.yml \
  -e GITLAB_TOKEN=$(pass /internet/gitlab.com/tokens/management) \
  -e PROJECTS_BASE=/tmp/bulkcheckout \
  -w /tmp/bulkcheckout \
  -v /tmp/bulkcheckout:/tmp/bulkcheckout \
  -v ${HOME}/.ssh:/home/builder/.ssh:ro \
  -v ${PWD}/gitlab_groupMapping.yml:/app/gitlab_groupMapping.yml:ro \
  nolte/gitlab-bulkcheckout -v checkoutbulk

Gitlab Bulk Checkout Tool

Getting Started

Usage

gitlab_bulkcheckout

Console script for gitlab_bulkcheckout.

gitlab_bulkcheckout [OPTIONS] COMMAND [ARGS]...

Options

-v, --verbose

Print debug information

--debug-file <debug_file>

File to be used as a stream for DEBUG logging

checkoutbulk

Checkout all listed projects from configured groups.

gitlab_bulkcheckout checkoutbulk [OPTIONS]

Options

-h, --host <host>

Url from the Gitlab Server, like https://gitlab.com

--private-token <private_token>

Gitlab Private Acccess Token

-m, --mapping-config-path <mapping_config_path>

Bulk Checkout configuration

-b, --base <base>

The Basedirectory for the local checkout

Environment variables

GITLAB_HOST
Provide a default for --host
GITLAB_TOKEN
Provide a default for --private-token
GROUPS_MAPPINGS
Provide a default for --mapping-config-path
PROJECTS_BASE
Provide a default for --base
version

Display the current version.

gitlab_bulkcheckout version [OPTIONS]

Config

example config
4564793:
  fs_prefix: "cookiecutter"
  path: templating

...
[gitlabGroupId]:
  # prefix for the local projects
  fs_prefix: "cookiecutter"
  # local folder for all projects from this group relative to $PROJECTS_BASE
  path: "templating"
...

Container Usage

image version docker stars docker pulls

For easy and quickly usage you can use the Dockerfile from Dockerhub.

docker pull nolte/gitlab-bulkcheckout:latest

For wrapping the tool, and checkout to your local FS it is required that you define some additional Container Run Parameters.

execute checkout
docker run -it \
  --user=${UID}:$(id -g $(whoami)) \
  -v ${SSH_AUTH_SOCK}:/ssh-agent \
  -e SSH_AUTH_SOCK=/ssh-agent \
  -e GROUPS_MAPPINGS=/app/gitlab_groupMapping.yml \
  -e GITLAB_TOKEN=$(pass /internet/gitlab.com/tokens/management) \
  -e PROJECTS_BASE=/tmp/bulkcheckout \
  -w /tmp/bulkcheckout \
  -v /tmp/bulkcheckout:/tmp/bulkcheckout \
  -v ${HOME}/.ssh:/home/builder/.ssh:ro \
  -v ${PWD}/gitlab_groupMapping.yml:/app/gitlab_groupMapping.yml:ro \
  nolte/gitlab-bulkcheckout -v checkoutbulk

Container Run Parameters

--user ${UID}:$(id -g $(whoami))
The Container User will mapped to your user and group from the Host System, see User.
-v $SSH_AUTH_SOCK (optional)
Required for forward the User SSH Agent from the Host System to the container.
-e SSH_AUTH_SOCK (optional)
see -v $SSH_AUTH_SOCK
-e GROUPS_MAPPINGS
The Path to the checkout config (see: gitlab_bulkcheckout, Config Volume, Config,)
-e GITLAB_TOKEN
Gitlab Access Token for using the Gitlab API (see: gitlab_bulkcheckout) This example use the Commandline Tool pass for manage this secret.
-e PROJECTS_BASE
The Local Checkout Location, this path should be mapped to your Host System, (see: gitlab_bulkcheckout).
-w /tmp/bulkcheckout
The Container Workingdir.
-v /tmp/bulkcheckout:/tmp/bulkcheckout
Some Folder mounted to your Host System, used as checkout Base.
-v ${HOME}/.ssh:/home/builder/.ssh:ro
Mount your local SSH folder, for checkout the repositories over ssh.
-v ${PWD}/gitlab_groupMapping.yml:/app/gitlab_groupMapping.yml:ro
The Used config file (see: Config)

Top-level package for project manager.

Indices and tables