As someone who frequently works with both GitHub and GitLab repositories, I often find the need to get an overview of various projects that exist within an organization or group. At my workplace, we use GitLab’s group- and subgroup functionality to organize our team-specific and customer-specific structures.

Recently, I needed to clone all 27 repositories associated with one of our microservice-projects (where else would you find so many repositories?). Doing this the traditional way by hand would require me to copy the URL for each repository and run “git clone \$URL” on the command line. This is a time-consuming and tedious process I want to avoid.

Fortunately,my habit for automating things lead me to discovering a tool called ghorg that simplifies this process.

With just a single command, I was able to clone entire GitHub organizations or GitLab (sub-)groups. Below is the command I used to do this:

ghorg clone customergroup/services --base-url= --scm=gitlab --token=$TOKEN --preserve-dir -p .

This command clones all repositories in the group customergroup/services, connecting to our private GitLab instance, using the specified token $TOKEN to authenticate. The --preserve-dir option clones the repos in a directory structure that matches GitLab groups, so repository foo will be cloned into customergroup/services/foo.

ghorg clone customergroup/services --base-url= --scm=gitlab --token=$TOKEN --preserve-dir -p .

 +-+-+-+-+ +-+-+ +-+-+-+-+-+
 |T|I|M|E| |T|O| |G|H|O|R|G|
 +-+-+-+-+ +-+-+ +-+-+-+-+-+

* SCM           : gitlab
* Type          : org
* Protocol      : https
* Location      : ./
* Concurrency   : 25
* Base URL      :
* Preserve Dir  : true
* Config Used   : none
* Ghorg version : v1.9.3
27 repos found in customergroup/services

Success cloning repo: -> branch: main
Success cloning repo: -> branch: master
Success cloning repo: -> branch: main
Success cloning repo: -> branch: main
Success cloning repo: -> branch: main
Success cloning repo: -> branch: master
Success cloning repo: -> branch: master

Interestingly, ghorg is not limited to GitLab or GitHub; it also offers support for other repository providers such as Bitbucket.

But I’m also a great fan of the gitlab-cli and it has a similar feature, too! Albeit the Gitlab-cli supports only Gitlab, naturally.

The same batch cloning process can be performed through the gitlab-cli tool with this command: glab repo clone --group customergroup/services -p

The syntax deviates slightly from the one used in “ghorg”. With --group $GROUP you define the particular group you want to clone and with the -p option you preserve the group-structure.

> glab repo clone --group customergroup/services -p
Cloning into 'customergroup/services/foo'...
remote: Enumerating objects: 59, done.
remote: Counting objects: 100% (5/5), done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 59 (delta 0), reused 0 (delta 0), pack-reused 54
Receiving objects: 100% (59/59), 13.36 KiB | 1.03 MiB/s, done.

Subsequently, both tools, ghorg and gitlab-cli, produced the same directory structure for the cloned repositories when tested, thus proving equivalently efficient.

> tree -d -L 3
└── customergroup
    └── services
        ├── foo
        ├── foo-audit-report
        ├── foo-tools
        ├── foo-base
        ├── foo-pages
        ├── foo-helm
        └── foo-application-base

Many other tools offer similar batch cloning features, like myrepos, vcsh or gitlabber, but I haven’t explored these alternatives yet. If anyone has experience with one of these tools, tell me! I’d love to hear about it.

Related posts: