GWCelery deployment

Helm repository

The Helm chart for installing GWCelery and Redis is defined in this repository. Default values of the chart allow for a sandboxed deployment on minikube.

The first step is to add the necessary repository:

helm repo add --username <username> --password <token> --force-update \
    gwcelery-helm \ 
    https://git.ligo.org/api/v4/projects/16068/packages/helm/stable

Where:

• <username> is your marie.curie username

• <token> is a read_api scoped personal access token

Secret creation

You need to create a Kubernetes Secret containing the proxy certificate to authenticate to the GraceDB API. To create a proxy certificate lasting 1 year:

ligo-proxy-init -H 8760 <marie.curie>

To create the Secret called gwcelery-secrets:

./utility/add-gwcelery-secret.sh /tmp/x509up_u<uid>

where uid is your Linux user ID.

Attention

The connection to your local Hopskotch server is not authenticated. If you need to connect to a server requiring authentication (e.g. SCiMMA), you can pass the path to the .netrc, auth.toml, and `kafka_credential_map.json' files when creating the Secret.

user> ./add-gwcelery-secret.sh 
USAGE: add-gwcelery-secret.sh x509_file [netrc_file auth_file kafka_credential_map_file namespace]

Chart installation

Run the command below to install the GWCelery Helm chart. The chart also installs an instance of Redis as dependency.

K3sminikube

Attention

The step below is slightly different in the case of the LDAS fluxuser machines.

On your local K3s instance run:

user> helm  upgrade --install --set gwcelery.storageClassName=local-path \
    gwcelery gwcelery-helm/gwcelery

On the LDAS fluxuser machines run:

user> helm  upgrade --install --set gwcelery.storageClassName=local-path \
    --set gwcelery.ingress.host=fluxuser<X>.ldas.cit \
    --set gwcelery.gracedb.hostname=gracedb-dev<X>.ldas.cit
    gwcelery gwcelery-helm/gwcelery

user> helm  upgrade --install \
   gwcelery gwcelery-helm/gwcelery 

To check that the GWCelery Pods are properly running execute the command:

user> kubectl get pods | grep gwcelery
gwcelery-celery-749644655d-jfstt                   2/2     Running   0         5s
gwcelery-embright-0                                1/1     Running   0         5s
gwcelery-flask-5985979b8c-jhnx8                    1/1     Running   0         5s
gwcelery-flower-7cb5d567d-gd64q                    1/1     Running   0         5s
gwcelery-highmem-74666dcb7c-nskxn                  1/1     Running   0         5s
gwcelery-kafka-7c8f5899c-jrcqx                     1/1     Running   0         5s
gwcelery-multiprocessing-76848b7f65-8f82l          1/1     Running   0         5s
gwcelery-openmp-0                                  1/1     Running   0         5s
gwcelery-redis-master-0                            1/1     Running   0         5s
gwcelery-superevent-5b7599f67f-fh7kj               1/1     Running   0         5s
gwcelery-voevent-754df64b79-565gw                  1/1     Running   0         5s

Run a custom GWCelery tag

The GWCelery version to be used can be specified adding the following flags to the installation command:

--set gwcelery.image.repository="containers.ligo.org/emfollow/gwcelery"
--set gwcelery.image.tag="<my-tag>"

You can also specify your custom image, either hosted on a public repository or local.

Tip

In order to make your local image available to the minikube environment, please refer to this documentation. For instance, you could build it directly into the minikube cluster by using the following command:

user> minikube image build -t "gwcelery-custom/development:mytag"

Web endpoints

The GWCelery console and Flower monitoring are available on your local network (as in the GraceDB case, you might want to use a SOCKS proxy to access these services from your local browser).

Attention

In the case of the LDAS fluxuser machines, the endpoints are:

http://fluxuser<X>.ldas.cit/gwcelery/

http://fluxuser<X>.ldas.cit/flower/

In all other cases, first you need to add the following entry in your /etec/hosts file:

127.0.0.1   gwcelery gwcelery.default.svc.cluster.local

In minikube remember to open a tunnel in a separate terminal:

user> minikube tunnel

The endpoints for the GWCelery console and Flower monitoring are:

http://gwcelery.default.svc.cluster.local/gwcelery/
http://gwcelery.default.svc.cluster.local/flower/

Test

To test the installation you can either generate some event using the Mock Event Generator (MEG) as described here, or use the GWCelery console.

To inspect the logs of the various GWCelery workers, each deployed as a separate Pod, use the following command:

user> kubectl logs -f <pod-name> 

Uninstall

To uninstall the Helm chart run:

user> helm uninstall gwcelery

To manually remove the Secret containing your proxy certificate (for example to use a new certificate):

user> kubectl delete secret gwcelery-secrets 

To delete persistent volumes:

user> kubectl delete pvc embright-data-gwcelery-embright-0
user> kubectl delete pvc lalframes-gwcelery-openmp-0
user> kubectl delete pvc redis-data-gwcelery-redis-master-0   

Note that the last command will reset the Redis queue.