Git mirrors of SVNKit and SqlJet repositories

Hi,

We use our SubGit product for self-hosting for more than a year already, but until now we didn’t have Git side of repository open for the public. Today I’ve found some time to make necessary changes to the Apache configuration and, voilà, our dear users may now use Git to get SVNKit or SqlJet source code. Just run:


$ git clone http://svn.svnkit.com/git/svnkit svnkit
$ git clone http://svn.svnkit.com/git/sqljet sqljet

To clone corresponding project’s git repository!

SubGit-based server side git-svn mirror works like a charm :)

From Svn to Gitolite

Hi,

Today I’d like to describe how to migrate from Svn to Git, minding existing infrastructure. One of the most common Subversion servers configurations is Linux server with Apache and mod_dav_svn module serving one or more Subversion repositories. Each Subversion repository may contain one or more projects. Of course, I will use SubGit to make migration smooth, i.e. without a need to force users to make a switch from Subversion to Git overnight.

For Git, I’ve chosen Gitolite as a Git management tool, as it is relatively easy to install and lot of documentation resources.

Background

Here is the first picture. In blue initial configuration is shown and in gray are what we will gradually build in this post. This sample migration takes place on Ubuntu Linux server:

Initial Setup

Step 1: Install Gitolite and create ‘Git’ user

To install Gitolite I’m using “install as root” method from Gitolite documentation. Few migration-specific additions are marked with the bold font.

On your workstation:

  • copy your ~/.ssh/id_rsa.pub file to /tmp/YourName.pub on the server. (The name of this file determines your gitolite username, so if you leave it as id_rsa.pub, your gitolite username will be id_rsa, which may not be what you want).

On your server, as root:

git clone git://github.com/sitaramc/gitolite
gitolite/src/gl-system-install
# defaults to being the same as:
# gitolite/src/gl-system-install /usr/local/bin /var/gitolite/conf /var/gitolite/hooks

# to upgrade gitolite, repeat the above commands. Make sure you use the
# same arguments for the last command each time.

# Git user should be in the same group as your Apache user:
useradd git -g www-data -m -s /bin/bash

# switch to the hosting user
su - git

# (now as git)
gl-setup /tmp/YourName.pub

# when prompted edit the following values in /home/git/.gitolite.rc file:
$REPO_UMASK = 0002; # default is 0077
$GL_GITCONFIG_KEYS = "core.sharedRepository"; # default is ""

# Change repositories directory permissions to let Apache user
# read it contents:
chmod ug+rx /home/git/repositories

On your workstation:

git clone git@server:gitolite-admin

Gitolite Installed

Step 2: Create Git repositories with Gitolite

Create empty Git repository for each of your Subversion project. In this post I assume that there are three Subversion projects in a single Subversion repository (p1, p2, p3) each with a standard trunk/branches/tags layout. You may have different distribution as well as different layout.

On your workstation, edit gitolite-admin/conf/gitolite.conf file:

repo    gitolite-admin
        RW+     =   alex
repo    testing
        RW+     =   @all

repo    p1
        RW+     =   alex
        config core.sharedRepository = true
repo    p2
        RW+     =   alex
        config core.sharedRepository = true
repo    p3
        RW+     =   alex
        config core.sharedRepository = true

Add more access rules if necessary, commit and push your change:


git add conf/gitolite.conf
git commit -m "repositories added"
git push

Git Repositories Created

Step 3: Set up smooth Svn to Git migration

On server download and install SubGit. Use either debian package distribution or zip archive.

sudo dpkg -i subgit_1.0.0-EAP-902_all.deb

Or

unzip subgit_1.0.0-EAP-902.zip

To set up migration, as www-data user, run:

$ sudo -u www-data subgit configure /var/svn/repos
SubGit version 1.0.0-EAP ('Miai') build #902
This is an EAP build, which you may not like to use in production environment.

Detecting paths eligible for translation...
Subversion to Git mapping has been configured in '/var/svn/repos':
/p1 : /var/svn/repos/git/p1.git
/p2 : /var/svn/repos/git/p2.git
/p3 : /var/svn/repos/git/p3.git

CONFIGURATION SUCCESSFUL

Adjust '/var/svn/repos/conf/subgit.conf' file
and then run
subgit install "/var/svn/repos"
to complete SubGit installation.

SubGit has detected Subversion projects and created default configuration. Now I will edit SubGit configuration file at /var/svn/repos/conf/subgit.conf to specify our Gitolite repositories locations instead of default ones and to mark repository as shared:

[core]
        # shared option must be set to 'true', 
        # as long as apache and gitolite are ran 
        # by different users (i.e. www-data and git)
        shared = true
        ...
        # authors.txt consists of mapping lines:
        # svnUser=gitUser<gitUser@email.com>
        # this file is optional
        authorsFile = conf/authors.txt
[git "p1"]
        translationRoot = p1
        repository = /home/git/repositories/p1.git
        ....
[git "p2"]
        translationRoot = p2
        repository = /home/git/repositories/p2.git
        ...
[git "p3"]
        translationRoot = p3
        repository = /home/git/repositories/p3.git
        ...

As soon as you’re happy with configuration, enable migration:

$ sudo -u www-data subgit install /var/svn/repos
SubGit version 1.0.0-EAP ('Miai') build #902
...
INSTALLATION SUCCESSFUL
...

That’s all, Gitolite and smooth Svn To Git migration is now configured!

Smooth Svn to Git migration

Commit changes to Subversion and Git users will pull them into their Git clones, push commits to Git repository and Subversion users will receive them. Try it :)

In case you have any questions or suggestions, please feel free to contact me at support@subgit.com
Thanks!

VisualSVN Subversion Server and Git

I’d like to start this blog with a few real-world examples on how to set up SubGit assuming infrastructure that is already in place. This post will take place in a strange world of Windows.

Initial configuration: VisualSVN Server on 64-bit Windows computer, provides read and write access to two Subversion repositories over HTTPS.

Primary Objective: Make Subversion repositories accessible for reading and modification with Git over HTTPS.
Secondary Objective: Reuse existing authentication settings, that are already configured for Subversion repositories.

In other words I’m about to set up instant bidirectional Svn to Git replication.

Configuration Details

This is a VisualSVN configuration I’ve created for this guide:

Initial VisualSVN Configuration

As you may see project repository is a single-project one with a standard trunk/branches/tags layout, and main repository contains two subproject, each with a standard layout.

VisualSVN is installed in C:\Program Files (x86)\VisualSVN Server directory.
Repositories are located in C:\Repositories directory.
VisualSVN uses standard Subversion authentication settings.

Add Git to Svn

First, install msysGit, you may download installer from their download page. Make sure you select this option when installing msysGit (it is important for enabling HTTP access later!):

Important Git Installation Option

Second, change an account VisualSVN service uses and modify C:\Repository directory (one where repositories are kept) permissions as described in a Visual SVN knowledge base entry. It is important to make VisualSVN service run on behalf of the account that has access to the Git you’ve just installed. Following the way of lesser resistance, I’ve used my personal account for that, but you may create a dedicated one.

After completing this step, I had VisualSVN service running on behalf of “HOST\alex” account and C:\Repository was writable for “HOST\alex”. Yes, my name is Alex.

Finally, download SubGit zip archive and unpack it into C:\SubGit directory. The run “subgit install” on Subversion repositories to enable replication:

> C:\SubGit\bin\subgit install C:\Repositories\main
> C:\SubGit\bin\subgit install C:\Repositories\project

Install SubGit

Install SubGit


Note, that you must run “subgit install” on behalf of the same user that you’ve configured VisualSVN service with.

Configure HTTP Access for Git

Important: VisualSVN comes with more or less truncated version of Apache, but fortunately it misses only one module of those needed for Git (mod_cgi). Download it (mod_cgi.so from Apache 2.2.21 for win32) and put into C:\Program Files (x86)\VisualSVN Server\bin directory.

Alternatively, you may get this missing module by installing Apache and taking mod_cgi.so file from the modules folder.

Then, edit C:\Program Files (x86)\VisualSVN Server\conf\httpd-custom.conf file:

LoadModule cgi_module bin/mod_cgi.so
LoadModule authz_user_module bin/mod_authz_user.so

SetEnvIf Request_URI "^/git/.*$" GIT_PROJECT_ROOT=C:/Repositories
SetEnvIf Request_URI "^/git/.*$" GIT_HTTP_EXPORT_ALL

ScriptAlias /git/ "C:/Program Files (x86)/Git/libexec/git-core/git-http-backend.exe/"

<Location /git>
  Options +ExecCGI

  Require valid-user
  AuthName "VisualSVN Server"
  AuthType Basic
  AuthBasicProvider file
  AuthUserFile "C:/Repositories/htpasswd"
</Location>

Note, that this configuration example uses “Basic” authentication option, granting read and write access to repositories for all Git users that are listed in htpasswd file. Of course you may configure something more complicated here.

Now restart VisualSVN Server service. That’s all!

Git for Svn

Try the following commands now:

# set this environment variable in case you're using SSL and 
# self-signed SSL certificate in VisualSVN (this is default).
> SET GIT_SSL_NO_VERIFY=true

> git clone https://alex@localhost/git/project project
> git clone https://alex@localhost/git/main/git/library lib

Thanks to SubGit, changes pushed from the cloned Git repository will be immediately propagated to the corresponding Svn repository and vice versa – new Svn revisions will be received by a pull performed from a cloned Git repository.

You may find more on SubGit at http://subgit.com/