Semantics of this Tutorial

General Information
Command line execution

Important distribution install $prefix=$HOME/citk/systems

Step #1: Installing a Distribution

For now the CITK tool chain "only" fully supports Ubuntu 14.04 (Trusty) and 16.04 (Xenial) 64 bit. The CITK tool chain is generally not dependent on a specific UNIX/Linux flavor. However, in order to maintain full support we decided to focus on Ubuntu Linux since it is one of the most popular distributions. If you feel like you need support for any other OS or Linux flavor please contact the developers here.

Before running any script (Jenkins) provided in this tutorial, please unset the following environment variables.

unset http_proxy
unset ftp_proxy
unset https_proxy

Assuming you have successfully completed the bootstrapping Tutorial we will now continue with the actual building and deployment steps. Remember: your $prefix is $HOME/citk/. Now we will clone and Install a Distribution. It contains a set of referenced Projects that are grouped together to form a system or sub-system for a certain task. For instance, we do collect all stable components in a versioned toolkit Distribution.

mkdir -p $HOME/citk/dist && cd $HOME/citk/dist
git clone .

After you cloned the repository you will find all available Distributions in the distributions folder. In order to setup a distribution on your local Jenkins copy, issue the following command, where DESIRED_DISTRIBUTION is a file from the distributions folder. This will create all necessary build jobs for your Distribution of choice.

In order to list all the system-specific dependencies currently not installed, you can advise the job-configurator to do so for your distribution.

$HOME/citk/jenkins/job-configurator --on-error=continue --cache-directory=$HOME/citk/gen-cache platform-requirements -p "ubuntu $(lsb_release -rs)" $HOME/citk/dist/distributions/DESIRED_DISTRIBUTION.distribution
Please note: Additional, system-specific, packages (*.deb, *.rpm) are listed in the corresponding system description, e.g., here. The build-generator will list missing packages. Install the packages prior to running the first build.
$HOME/citk/jenkins/job-configurator --on-error=continue --cache-directory=$HOME/citk/gen-cache generate -m toolkit -D toolkit.volume=$HOME/citk/systems -u {YOUR_USERNAME} -a {YOUR_API_TOKEN} $HOME/citk/dist/distributions/DESIRED_DISTRIBUTION.distribution

YOUR_USERNAME is the one you have created when installing the Jenkins instance by using the script. YOUR_API_TOKEN can be found out using the Jenkins web interface. Navigate to https://localhost:8080/user/{YOUR_USERNAME}/configure and use the "Show API Token" button to get an API token. You might as well use your password, as provided during the create_user phase.

Please note: Depending on the size of the cloned/checked-out repositories this can take a while. As soon as jobs appear in your local Jenkins installation, the generation step is done.

If the job-configurator ran successfully you will probably see a few warnings like the following — you can ignore them, e.g., "No Provider"

VERSION-SPEC fsmt:master {F226BB9}> (1 missing dependency):
    No provider for (:SETUPTOOLS "pep8").

Output you cannot ignore is a notification that credentials, aka login and password, are required in order to checkout the source code referenced in a distribution. This message is only shown if the distribution, or at least one project, is marked as "private". If this is not the case you may directly proceed with the next step. In any other case please expand the blue box below.

The following credentials have been used:
    * "projects.cit-ec" for job rst-proto-csra-master-toolkit-flobi-faceid-nightly

In general, private access for distributions or projects is not recommended since it has a huge negative impact on general reproducibility.

1) Please, go to https://localhost:8080/?auto_refresh=true and open the Credentials dialog on the left side.

2) Next select Global credentials (unrestricted)

3 ) Then select Add Credentials

In the next dialog add your credentials for the required project, e.g, projects.cit-ec (as shown in the generator output). Important: you have to expand the Advanced option on the right side in order to make the ID field visible. Please provide the id as printed in the generator output. For example:

The following credentials have been used:
    "projects.cit-ec" for jobs rst-proto-csra-master-toolkit-flobi-faceid-nightly

4) Save and proceed with the next step

Now, go to https://localhost:8080/?auto_refresh=true again and reload the page, you will see a couple of newly created build jobs (their names may be different).
Login using the "login menu" in the top right corner using the credentials you chose in the previous tutorial (if you are not already logged in).

Step #2: Deploy!

After these steps, trigger the *-orchestration-* job, which will orchestrate all other jobs according to their dependencies. You can trigger the orchestration-job by clicking on the most right stopwatch icon (see image below). After all jobs ran your distribution is installed and ready to be used in your $prefix.

Step #3: Expected Results

If everything went well, all job icons turned blue (success status) and you may proceed with the Running an Experiment Tutorial

Hint: If one or two jobs are red which means compilation/installation failed, please re-run the build flow job for a second time. If they stay red, please report a bug (see below)


Support/Issues can be requested/filed under