Table of contents

Introduction

Qvarn utils are a set of command line tools provided by QvarnLabs to help with Qvarn-related administrative tasks. At the moment they are of qvarn-dump, qvarn-copy and qvarn-stats. They are meant for migrating data to a fresh Qvarn instance during a service break or incrementally updating data to a fresh Qvarn instance.

Features

qvarn-stats shows how many resources of each type there are. The resource types are given as parameters.

qvarn-dump writes a list of all resources in a form that can easily compared to the output of a previous run.

qvarn-copy copies resources from one Qvarn to another.

Prerequisites

  • A Debian 9 system.
  • Root access to that system.
  • One or more Qvarn stacks to copy data.

Installation

Qvarn utils are distributed as a Debian package from QvarnLabs's APT repository. In order to install the package on Debian Stretch, add this repository to your source list. Create a file /etc/apt/sources.list.d/qvarnlabs.list with the following content:

deb http://apt.qvarnlabs.net/debian stretch main

It is a signed repository so we need to import the key it is signed with:

sudo apt-key adv --fetch-keys http://qvarn.org/apt.txt

Now update package index and install qvarn-utils:

sudo apt-get update
sudo apt-get install qvarn-utils

Usage

Access configuration

You need to store client id and secret for accessing a Qvarn in ~/.config/qvarn/createtoken.conf:

[https://qvarn.example.com]
client_id = @id.comes.here@
client_secret = secret.passphrase.comes.here

The first line (in []) is the base URL for the Qvarn instance. You can get the client id and secret when you create the client in the Gluu configuration.

You need to assign the client all the scopes in Gluu that it needs for accessing the Qvarn. All three tools need read access, and qvarn-copy also needs write access. Thus for a given resource type foo, with subresource bar, you would need to give the following scopes for read access:

uapi_foos_get
uapi_foos_id_get
uapi_foos_id_bar_get
uapi_foos_listeners_get
uapi_foos_listeners_id_get
uapi_foos_listeners_id_notifications_get
uapi_foos_listeners_id_notifications_id_get

For qvarn-copy you also need to give the following scopes:

uapi_foos_post
uapi_foos_id_bar_put
uapi_foos_listeners_post
uapi_foos_listeners_id_notifications_post

In addition, the following special scope needs to be assigned to qvarn-copy:

uapi_set_meta_fields

qvarn-dump

The first tool is qvarn-dump, which dumps all the resources of enumerated resource types, including listeners and notifications. This is especially useful to run after qvarn-copy has successfully finished, since running qvarn-dump on both Qvarns produces identical output. The output format of qvarn-dump is such that two dumps can be easily compared with standard Unix tools, such as diff. Also, two dumps can be catenated.

Command line arguments

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  --generate-manpage=TEMPLATE
                        fill in manual page TEMPLATE
  --output=FILE         write output to FILE, instead of standard output
  --secrets=FILE        get API client id and secret from FILE
  --token=TOKEN         use TOKEN as the access token
  --api=URL             use URL as the base URL for the Qvarn
  --max=N               use at most N resources of each type

  Configuration files and settings:
    --dump-setting-names
                        write out all names of settings and quit
    --dump-config       write out the entire current configuration
    --no-default-configs
                        clear list of configuration files to read
    --config=FILE       add FILE to config files
    --list-config-files
                        list all possible config files
    --help-all          show all options

  Logging:
    --log=FILE          write log entries to FILE (default is to not write log
                        files at all); use "syslog" to log to system log,
                        "stderr" to log to the standard error output, or
                        "none" to disable logging
    --log-level=LEVEL   log at LEVEL, one of debug, info, warning, error,
                        critical, fatal (default: debug)
    --log-max=SIZE      rotate logs larger than SIZE, zero for never (default:
                        0)
    --log-keep=N        keep last N logs (10)
    --log-mode=MODE     set permissions of new log files to MODE (octal;
                        default 0600)

  Peformance:
    --dump-memory-profile=METHOD
                        make memory profiling dumps using METHOD, which is one
                        of: none, or simple (no meliae support
                        anymore)(default: simple)
    --memory-dump-interval=SECONDS
                        make memory profiling dumps at least SECONDS apart

Examples

Run qvarn-dump in a virtual terminal:

qvarn-dump --api https://qvarn1.example.com org person card > qvarn.dump

qvarn-copy

The second tool is qvarn-copy. It copies resources from one Qvarn instance to another, under the assumption that the target Qvarn is not otherwise in use, and that none of its data will be changed, except by qvarn-copy. The tool is given a list of resource type names to copy, and it copies all resources of those types, and their notifications, while retaining resource IDs and revision IDs. The tool will not copy resources that already exist in the target, so that it can be stopped and re-run if needed.

Command line arguments

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  --generate-manpage=TEMPLATE
                        fill in manual page TEMPLATE
  --output=FILE         write output to FILE, instead of standard output
  --secrets=FILE        get API client id and secret from FILE
  --source-token=TOKEN  use TOKEN as the access token for the source Qvarn
  --target-token=TOKEN  use TOKEN as the access token for the target Qvarn
  --source=URL          use URL as the base URL for the source Qvarn
  --target=URL          use URL as the base URL for the target Qvarn
  --max=N               use at most N resources of each type

  Configuration files and settings:
    --dump-setting-names
                        write out all names of settings and quit
    --dump-config       write out the entire current configuration
    --no-default-configs
                        clear list of configuration files to read
    --config=FILE       add FILE to config files
    --list-config-files
                        list all possible config files
    --help-all          show all options

  Logging:
    --log=FILE          write log entries to FILE (default is to not write log
                        files at all); use "syslog" to log to system log,
                        "stderr" to log to the standard error output, or
                        "none" to disable logging
    --log-level=LEVEL   log at LEVEL, one of debug, info, warning, error,
                        critical, fatal (default: debug)
    --log-max=SIZE      rotate logs larger than SIZE, zero for never (default:
                        0)
    --log-keep=N        keep last N logs (10)
    --log-mode=MODE     set permissions of new log files to MODE (octal;
                        default 0600)

  Peformance:
    --dump-memory-profile=METHOD
                        make memory profiling dumps using METHOD, which is one
                        of: none, or simple (no meliae support
                        anymore)(default: simple)
    --memory-dump-interval=SECONDS
                        make memory profiling dumps at least SECONDS apart

Examples

qvarn-dump --source https://qvarn1.example.com \
    --target https://qvarn2.example.com org org person card

qvarn-stats

qvarn-stats counts the number of resources of each type named on the command line, plus any listeners and notifications associated with it.

Command line arguments

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  --generate-manpage=TEMPLATE
                        fill in manual page TEMPLATE
  --output=FILE         write output to FILE, instead of standard output
  --secrets=FILE        get API client id and secret from FILE
  --api=API             use URL as the base URL for Qvarn
  --totals              add up counts and display totals
  --no-totals           opposite of --totals

  Configuration files and settings:
    --dump-setting-names
                        write out all names of settings and quit
    --dump-config       write out the entire current configuration
    --no-default-configs
                        clear list of configuration files to read
    --config=FILE       add FILE to config files
    --list-config-files
                        list all possible config files
    --help-all          show all options

  Logging:
    --log=FILE          write log entries to FILE (default is to not write log
                        files at all); use "syslog" to log to system log,
                        "stderr" to log to the standard error output, or
                        "none" to disable logging
    --log-level=LEVEL   log at LEVEL, one of debug, info, warning, error,
                        critical, fatal (default: debug)
    --log-max=SIZE      rotate logs larger than SIZE, zero for never (default:
                        0)
    --log-keep=N        keep last N logs (10)
    --log-mode=MODE     set permissions of new log files to MODE (octal;
                        default 0600)

  Peformance:
    --dump-memory-profile=METHOD
                        make memory profiling dumps using METHOD, which is one
                        of: none, or simple (no meliae support
                        anymore)(default: simple)
    --memory-dump-interval=SECONDS
                        make memory profiling dumps at least SECONDS apart

Examples

qvarn-stats --totals --api https://qvarn1.example.com org person card 

Support

If you need help, or you have found a problem, email qvarn-support@qvarnlabs.com.