For a long time in our team we’ve been storing logins, passwords, keys and other things like that in personal password managers or just plain-text files, spread around people’s machines, and no one had the full set. Finally, we decided to stop this chaos and start using one common passwords database.

Having evaluated several options, we chose KeePass. It’s not exactly meant for multi-user usage, but we came up with some sort of workaround.

Why KeePass

There are many password manager solutions available, but not all of them meet the following requirements:

• no clouds, offline in-house hosting;
• no vendor-lock on a proprietary/closed format or software;
• low maintenance;
• proper native desktop/mobile client applications, and preferably more than one to choose from.

So the choice is rather (very) limited.

Personally I’ve been using 1Password, v6 and v7, while it’s still not based on Electron, with ability to have an offline vault and without subscription. And that was my first candidate, especially that they have some additional functionality for teams, but then they announced v8, which is destined to be a total shit: cloud-only, subscription-only, Electron-based - so that’s no longer an option. Here’re also some similar thoughts on the matter.

Then there are some solutions that do meet most of the requirements, and for example we could consider using Bitwarden or Passbolt. But sadly neither of these two have proper native client applications.

So KeePass ended up being basically the only suitable candidate:

• offline database, which is just a file that you can host anywhere;
• database format (KDBX) and reference client application are open (every release comes with the sources archive);
• a lot of native client applications on all the platforms, including mobile ones.

Client applications

Most of the compatible client application are proper native (without Electron garbage) applications. Many of those are available free of charge.

The full(?) list of clients on all platforms can be found here, under the “Other downloads and links” section.

Out of the desktop clients I liked the KeePassXC (sources) the most. Unlike the reference client, which is written in C# and is for Windows only, KeePassXC is based on C++/Qt and works on Windows, Mac OS and Linux. It also seems to have better performance / more responsive (and nicer) GUI.

One thing I was worried about is whether KeePass has the functionality of storing 2FA/TOTP codes, but it certainly does have that:

Mobile clients I haven’t tried yet, but looks like on iOS the choice comes down to these two:

• KeePassium (sources)
• Strongbox (sources)

Both are paid applications and are not cheap, but they do have a one-time purchase lifetime license option.

Android clients I haven’t looked at, as I don’t have Android devices.

Database

As I mentioned, KeePass isn’t really meant to be used in a multi-user environment, as there are certain challenges in keeping the original database in order and tracking changes. There are, however, solutions like Pleasant Password Server, which can help with that.

As a workaround, we came up with the following plan:

1. The original / source of truth database is stored on a dedicated server in the internal network, not exposed to the internet;
2. The database file lives in a Git repository, and that’s how changes are tracked. Yes, it is a binary file, but still it’s not a terrible idea to version control it with Git;
3. Users (team members) clone the repository to their machines and get a local copy of the database.

Master password and key file

The database file (let’s call it passwords.kdbx) can be protected by two security/access factors (and/or with a YubiKey):

1. Password. As it’s the master password, it must be a long and a complex one, so it unlikely will be memorable, and so you’ll need to store it somewhere, but where - that’s an interesting question, because the first answer is yet another database/manager, the password for which you also need to store somewhere, and then the password for this “somewhere” also needs to be stored… yeah, interesting question;
2. Database key file. To open the database, in addition to entering the password, you also need to provide path to the key file. Obviously, it must never be stored together with the database file.

To add a key file, enable additional protection on creating the database:

And here’s how it will look like when you will be unlocking the database using both the password and the key file:

The paths to database and key file should be different here, as, once again, they should never be stored together. And of course the key file should not be part of the database Git repository either.

Server setup

For such purpose (hosting a Git repository) it is enough to create a small GNU/Linux VM. Certainly make sure that it is not exposed to the internet, so team members can connect to it only from the office network (or via VPN).

The server should definitely be protected with some additional security measures, comparing to your regular hosts. As a bare minimum disable password authentication, so only SSH keys are allowed:

$ sudo nano /etc/ssh/sshd_config

ChallengeResponseAuthentication no
PasswordAuthentication no
PermitEmptyPasswords no

Generate and add an SSH key for administrator account to /home/ADMINISTRATOR-USER-NAME/.ssh/authorized_keys (more about system users below), make sure that you can connect with it and only then restart the SSH service, to apply the sshd_config changes:

$ sudo systemctl restart sshd.service

It also wouldn’t hurt to setup fail2ban, just in case:

$ sudo apt install fail2ban
$ sudo nano /etc/fail2ban/jail.local

[sshd]
enabled = true
port = ssh
filter = sshd
logpath = /var/log/auth.log
maxretry = 5
findtime = 667
bantime = 11111
ignoreip = 127.0.0.1

$ sudo systemctl restart fail2ban.service

$ sudo fail2ban-client status
$ sudo fail2ban-client status sshd

Further server hardening I leave up to you.

System users and SSH keys

Speaking about users and home directories on the server, here’s a suggested configuration:

• keeper:keeper - administrator, can run sudo;
• roquelaire:shoppers - regular user, cannot run sudo.

The roquelaire user is the owner of the database repository, which is placed in his home folder, and his account is used to access the repository over SSH.

This account also will have multiple SSH keys - one per team member. It is important to note that this user should not be able to modify his authorized_keys, to prevent regular team members from adding more keys, as only administrator (keeper) should be allowed to do that. So roquelaire’s authorized_keys file should be kept for example in /home/keeper/.ssh-roquelaire and owned by keeper user. In that case, of course, keeper home folder should not be available to roquelaire:

$ sudo mv /home/roquelaire/.ssh /home/keeper/.ssh-roquelaire
$ sudo chown -R keeper:keeper /home/keeper/.ssh-roquelaire
$ sudo chmod 711 /home/keeper

Then for SSH access to work, the following needs to be added to /etc/ssh/sshd_config:

Match User roquelaire
    #AuthorizedKeysFile /home/keeper/.ssh-roquelaire/authorized_keys
    AuthorizedKeysCommand /usr/local/bin/get_roquelaire_keys
    AuthorizedKeysCommandUser keeper

And the /usr/local/bin/get_roquelaire_keys script does the following:

#!/bin/sh

cat /home/keeper/.ssh-roquelaire/authorized_keys

Repository

The database repository lives in /home/roquelaire/passwds:

$ cd /home/roquelaire
$ mkdir passwds && cd $_
$ git init --bare .

After you commit first version of the database file to it, the resulting structure will look like this:

$ tree -L 2 /home/roquelaire/
/home/roquelaire/
└── passwds
    ├── HEAD
    ├── branches
    ├── config
    ├── description
    ├── hooks
    ├── info
    ├── objects
    └── refs

As it’s a bare repository, the database file isn’t really in a downloadable form here. To clone the repository, team members need to generate an SSH key pair on their machine:

$ cd ~/.ssh
$ ssh-keygen -o -t rsa -b 4096 -C "USERNAME@YOUR-COMPANY.com"

Ideally, the e-mails need to be correct and unique per user, so it’s a good idea for administrator to ensure that before adding a new key.

Once the key pair is generated, user gives his public key to administrator, and administrator then adds it to /home/keeper/.ssh-roquelaire/authorized_keys file on the server.

Users, in turn, add a new record into their local ~/.ssh/config:

# you might want to add a local DNS record for convenience
Host passwords.YOUR-COMPANY.com
User roquelaire
IdentityFile ~/.ssh/FILENAME-OF-THE-PRIVATE-KEY

Having established SSH access to the server, users can now clone the repository:

$ git clone roquelaire@passwords.YOUR-COMPANY.com:passwds

If you are confused about how that works, remember that Git repository path on server is /home/roquelaire/passwds, and so passwds is indeed a top-level folder in the roquelaire user home directory.

Now the /path/to/passwds/passwords.kdbx file in user’s local repository is the actual database he can work with.

Syncing

Once users clone the main repository to their machines, getting updates and making changes becomes a familiar task of working with a Git repository.

When it comes to mobile clients, everyone can distribute their local copy of the database to their mobile devices, but since there is rarely a Git client on those, especially on iOS, one might have to use a cloud provider (iCloud Drive, OneDrive, Dropbox, etc) for syncing the database across his devices. It is not recommended (no 3rd-party servers should be involved), but it’s more or less tolerable, as long as the key file isn’t uploaded to the cloud storage together with the database.

Making changes

As with any other Git repository, when someone has some changes to make in the database file, he just adds them into his local copy and commits/pushes it to the main repository.

In ideal world everyone would write meaningful commit messages, so others would know what exactly has changed in the database.

In a super ideal world everyone would also sign their commits.

RSS feed

One way how users can know about updates in the database is, as with any other Git repository, by fetching them from the main repository and looking at the log/history.

Another way to let users know about updates would be to create an RSS feed, using Git post-update hook in /home/roquelaire/passwds/hooks/post-update:

#!/bin/bash

rssFile="/var/www/passwords.YOUR-COMPANY.com/rss.xml"

# just to overwrite the file
echo '<?xml version="1.0" encoding="UTF-8"?>' > $rssFile

cat << _EOF_ >> $rssFile
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
    <channel>
        <title>Keeper</title>
        <link>http://passwords.YOUR-COMPANY.com/</link>
        <atom:link href="http://passwords.YOUR-COMPANY.com/rss.xml" rel="self" type="application/rss+xml"/>
        <description>Keeper database updates</description>
        <language>en</language>
_EOF_
#for i in $(git log -n11 --pretty=format:"%H")
git log -n11 --pretty=format:"%H" | while read -r commitHash || [ -n "$commitHash" ];
do
    dt=$(git show $commitHash --no-patch --date=rfc --pretty="format:%ad")
    author=$(git show $commitHash --no-patch --pretty="format:%ae (%an)")
    msg=$(git show $commitHash --no-patch --pretty="format:%s")
    cat << _EOF_ >> $rssFile
        <item>
            <title>Database update $(echo $commitHash | cut -c1-8)</title>
            <guid isPermaLink="false">$commitHash</guid>
            <pubDate>$dt</pubDate>
            <author>$author</author>
            <description><![CDATA[$msg]]></description>
        </item>
_EOF_
done
cat << _EOF_ >> $rssFile
    </channel>
</rss>
_EOF_

And serve resulting rss.xml with NGINX (/etc/nginx/conf.d/default.conf):

server {
    listen [::]:80;
    listen 80;

    server_name  passwords.YOUR-COMPANY.com;

    # probably also throw in a HTML page
    # with some description and administrator contacts
    root /var/www/passwords.YOUR-COMPANY.com;

    location / {
        index  index.html index.htm;
    }
}

Now users can subscribe to the feed in their RSS clients:

Overall

What’s good

Everything seems to work quite good:

• one common database to store team’s secrets;
• no vendor-lock, not sharing anything with 3rd-party;
  • everything is hosted on internal server;
  • the database format and client applications are open sourced;
• server is available via SSH keys only;
  • every user has his own key;
  • the keys are managed by administrator;
• database is encrypted and protected by two factors: password and key file;
• changes are under version control with Git;
• very low maintenance cost;
  • no MySQL/PostgreSQL/etcSQL;
  • no backend/frontend, no yet another framework to get familiar with;
  • just a regular VM with Git repository and access via SSH;
• several nice native desktop and mobile client applications to choose from.

What’s not so good

There are certain disadvantages too:

• the process of getting the database file is neither trivial nor convenient for a user;
  • many people are somewhat CLI-challenged and can struggle with such tasks as generating an SSH key. Managers and other non-technical people will be simply horrified by the whole thing;
  • there is no simple way to share the master password and the key file with new users, especially if they are not in the same office with someone who already has it;
• it is the same shared master password and key file for everyone in the team;
• there is no access level separation, so everyone has access to all the entries, whilst some of those might need to be available only to managers/administrators;
• there needs to be an administrator - to add/remove users public SSH keys on the server (although, that’s actually a good thing);
• binary file changes are not “visible” in Git;
  • one can forget what he changed in the database, and Git will only show that there are some uncommitted changes;
  • describing changes in commit messages is not ideal/reliable, because everyone must write meaningful and detailed commit messages (which they won’t);
• people can still mess up and (accidentally) leak the database by using personal clouds (although chances that they will also leak password and key file are not that high).

That’s quite a lot of disadvantages, actually. So we will keep looking for a better solution, but like I said in the beginning, so far KeePass seems to be the best one available, given the requirements that we have. At the very least, it’s better than nothing.