CRDM Keycloak Setup

Docker Setup for the Keycloak instance of the "Center for Research Data Management". The keycloak service maintains separate realms for the main (tu-data realm) and the test instance (tu-data-test realm) of TU Wien Research Data. Further information about Keycloak can be found in the official documentation.

Secrets and Configuration

We all keep secrets.
Similarly, some things such as passwords should be kept secret when deploying in production!
For this reason, we use environment variables to store the confidential parts of our configuration. Since exporting the correct environment variables manually every time can be very tedious, we use .env files.

To quickly spin up a development instance, the current setup can handle the absence of a .env file by using fallback values provided in the docker-compose.yml file. These fallbacks will take care of giving the environment variables appropriate values. As mentioned above however, there are environment variables holding sensitive data (e.g. database password) and thus these fallbacks SHOULD NOT be used in a production setting.

Example Secrets File

An example .env file is available as .example.env. Its goal is to give a rough overview of what the environment variables and their values should look like. After copying the file (cp .example.env .env) and replacing the dummy values, it is ready for use. Yet again, DO NOT use these values in production; they are embrassingly simple!

Performing a Keycloak Realm(s) Export/Import

The primary reason to export/import realms is simple: keep a backup of the current setting. With this feature, we can back up the current keycloak data and restore them whenever required. Better safe than sorry!

Exporting Realms

Exporting one or multiple realms is important, in case backups or data transfer between servers is needed. The export-realms.sh script is responsible for dumping all the realms in a JSON file.

The Keycloak Admin Console offers also an easy way to export realms. So why not use that instead? Because the export file won't contain any user data. Besides that, the client secrets will be masked, which means that we won't be able to recreate the same exact instance using only this file.

Brief explanation of the arguments used for exporting realms

• --file
  Defines the file where we want to save the data. Remember to prepend to the filename the exact same value that is used for the volume mapping in the docker-compose.yml.

• --users
  Specifies the strategy to export realm users. We set the value to realm_file to export the users to the same file as the realm configuration.

Importing Realms

Having a dockerized Keycloak service that works out of the box and contains an imported realm with its default users is very useful. This not only simplifies the setup process, but also allows us to share a replicable Keycloak instance with others. The import-realms.sh script is responsible for reading and importing all the realms from a JSON file (that was naturally exported from Keycloak itself).

Brief explanation of the arguments used for importing realms

• --file
  Defines the file which we want to import the data from. Remember to prepend to the filename the exact same value that is used for the volume mapping in the docker-compose.yml.

• --override
  Specifies how to proceed when a realm with the same name already exists in the database. We set the value to false to avoid overwriting a realm if it already exists.

Automatic Deployment via GitLab (CI/CD)

The steps provided in .gitlab-ci.yml are executed on a configured GitLab runner. The basic workflow is contained in a separate script deploy.sh and looks like the following:

• Dump the relevant environment variables (i.e. GitLab variables) to a .env file.
  We use a prefix to group the variables so that the resulting .env contains only the ones that are relevant for the deployment. The prefixes used and dumped for the deployment are: POSTGRES_ for variables relevant to postgres, KC_ for keycloak, BACKUP_ for the postgres backups. See the dump-vars.sh script.

• SSH to the configured host and create the provided deployment path if it doesn't exist. Then check if a git repository exists there: clone if not, fetch & checkout if yes.

• Secure copy the required files to the configured host (.env and ssl certificates). SSH again to tear down the old deployment and spin up the updated docker containers.