Run Pelican Server with Docker Image

This document explains how to run a Pelican server using Pelican docker image. If you want to use Pelican client functionalities, such as to download or upload an object, refer to the install page to download and install a Pelican binary instead.

Before starting

Pelican builds separate images for each Pelican server components, e.g. origin, cache, director, and registry. Depending on which Pelican server you want to run, you need to select a different Docker image from the list below.

• Pelican origin server: hub.opensciencegrid.org/pelican_platform/origin:latest
• Pelican cache server: hub.opensciencegrid.org/pelican_platform/cache:latest
• Pelican director server: hub.opensciencegrid.org/pelican_platform/director:latest
• Pelican registry server: hub.opensciencegrid.org/pelican_platform/registry:latest

Run Pelican server via Docker CLI

This section describes how to run various Pelican server images using the Docker CLI. If you haven’t installed Docker engine, follow the documentation from Docker  to install and start the Docker engine first.

Run Pelican origin server

To run the latest pelican origin server, run the following command:

docker run -it -p 8444:8444 -p 8443:8443 -v /path/to/your/data/:/tmp/pelican --name=pelican-origin hub.opensciencegrid.org/pelican_platform/origin:latest serve -v /tmp/pelican:/foo/bar -f <federation>

Where:

• docker run is a Docker CLI command that runs a new container from an image
• -it (--interactive --tty) runs the container in interactive mode and uses a tty terminal
• -p <host-port>:<container-port> (--publish) publishes a container’s port(s) to the host, allowing you to reach the container’s port via a host port. In this case, we can reach the container’s port 8444 via the host’s port 8444. Note that the admin website of Pelican servers run on port 8444 by default, and the objects transfer endpoint of the Pelican origin server runs on port 8443 by default.
• -v <host-location>:<container-location> (--volume) binds mount a volume from the host location(s) to the container’s location(s). This allow you to share files in your host machine to the container. In this case, we bind /path/to/your/data/ on your host machine to /tmp/pelican in the container. You need to replace /path/to/your/data/ to the directory where your data to publish is located.
• --name assigns a logical name to the container (e.g. pelican-origin). This allows you to refer to the container by name instead of by ID.
• hub.opensciencegrid.org/pelican_platform/origin:latest is the image to run
• serve is the command to run the Pelican binary
• -v /tmp/pelican:/foo/bar is the Pelican argument to bind /tmp/pelican directory in the container as namespace /foo/bar in Pelican. You need to change /foo/bar to a meaningful path that can represent your data, e.g. /chtc/public-data. You may pass additional arguments to Pelican server by appending them after this argument.
• -f <federation> sets the federation discovery URL, where <federation> is the URL to the federation the origin will be joining in. For instructions on how to find a federation to join, refer to Serve a Pelican Origin

Run Pelican cache server

To run the latest pelican cache server, run the following command:

docker run -it -p 8444:8444 -p 8442:8442 --name=pelican-cache hub.opensciencegrid.org/pelican_platform/cache:latest serve -f <federation>

Where most of the command overlaps with the one to run the Pelican origin server, with the following differences:

• -p 8444:8444 -p 8442:8442 publishes port 8444 and 8442 from the container to the same port on the host machine. Note that the admin website of the Pelican server runs on port 8444 by default, and the objects transfer endpoint of the Pelican cache server runs on port 8442 by default.

Run Pelican director server

To run the latest pelican director server, run the following command:

docker run -it -p 8444:8444 --name=pelican-director hub.opensciencegrid.org/pelican_platform/director:latest serve -f <federation>

Run Pelican registry server

To run the latest pelican registry server, run the following command:

docker run -it -p 8444:8444 --name=pelican-registry hub.opensciencegrid.org/pelican_platform/registry:latest serve -f <federation>

Stop Pelican container

To stop the Pelican container, run the following command:

# The `docker ps` command shows the processes running in Docker
docker ps
 
# This will display a list of containers that looks like the following:
CONTAINER ID   IMAGE  COMMAND   CREATED  STATUS   PORTS    NAMES
0be1a304b5d7   hub.opensciencegrid.org/pelican_platform/director:latest   "/bin/sh"   1 hour ago   Up 1 hour   0.0.0.0:8444->8444/tcp   pelican-director
 
# To stop the pelican container run the command
# docker stop <container-ID> or use
# docker stop <container-name>, which is `pelican-director` as previously defined
docker stop pelican-director

Configure Pelican server in a container

There are two ways to configure a Pelican server running in a container. One is through the environment variables, the other is by passing a pelican.yaml configuration file to the container. This section includes instructions for both.

Configure via environment variables

Most Pelican configuration parameters described in parameters page can be passed to a Pelican server as environment variables. For a configuration parameter, the corresponding environment variable is PELICAN_<PARAMETER_NAME>, where PELICAN is the prefix, and <PARAMETER_NAME> is the name of the parameter, with dots . replaced by underscores _. For example, a Pelican configuration named Logging.Level has a corresponding environment variable named PELICAN_LOGGING_LEVEL.

To pass environment variables to the Docker container, append -e flag to your docker run command. For example, if you want to pass PELICAN_LOGGING_LEVEL to the container, run:

docker run -it -e PELICAN_LOGGING_LEVEL=debug -p 8444:8444 -p 8443:8443 -v /path/to/your/data/:/tmp/pelican --name=pelican-origin hub.opensciencegrid.org/pelican_platform/origin:latest serve -v /tmp/pelican:/foo/bar -f <federation>

There are other ways to pass environment variables to a Docker container, see details in Docker documentation .

Configure via the configuration file

Pelican servers can also be configured via a configuration file in yaml. The default location that Pelican looks for a configuration file is /etc/pelican/pelican.yaml if Pelican runs as the root user (which is the case when the Pelican server is running in a container). If the Pelican server runs as a non-root user, the default location is ~/.config/pelican/pelican.yaml.

It is recommended that admins bind a Pelican configuration file on the host machine to the container running the Pelican server. Follow the steps below for instructions.

1. Prepare a yaml file on the host machine:

   touch pelican.yaml

2. Modify and save the pelican.yaml file on host machine with the following content:

   pelican.yaml

   Logging:
     Level: debug

3. Bind the configuration file on the host machine to the container when running the Pelican server image:

   docker run -it -p 8444:8444 -p 8443:8443 -v /path/to/your/data/:/tmp/pelican -v $PWD/pelican.yaml:/etc/pelican/pelican.yaml --name=pelican-origin hub.opensciencegrid.org/pelican_platform/origin:latest serve -v /tmp/pelican:/foo/bar -f <federation>

   where

   • -v $PWD/pelican.yaml:/etc/pelican/pelican.yaml is the flag to bind the pelican.yaml file under current working directory $PWD to /etc/pelican/pelican.yaml directory inside the container

4. You should note that your running Pelican origin server is logging debug level messages.

Additional Configurations for Running Pelican in a Container

Server hostname and port

Pelican server has a built-in web server to handle various API requests. By default, it uses server hostname and a port number as the server address (e.g. https://127.0.0.1:8444). In a container environment, the hostname is an alphanumeric value, e.g. 6ee28c5df997, and the corresponding Pelican server address is https://6ee28c5df997:8444. This value is for container internal communications and is not accessible from outside of the container. In local deployment, to access the container ports from the host machine, one can publish a port from inside the container to the host machine.

docker run -p 8444:8444 origin

However, Pelican server still recognize 6ee28c5df997 as its hostname, causing a mismatch.

In practice, there is usually a reverse-proxy service (such as Nginx, Traefik, etc.), to direct traffic from a human-readable domain name (e.g. https://example-origin.org ) to your container (e.g. 6ee28c5df997:8444). If Pelican server doesn’t catch this information and uses it as its server address, there will be a mismatch when the server advertises itself to other Pelican services.

Therefore, to keep things consistent, Pelican internally should use the same human-readable domain name as its web server’s external address. To do so, configure Server.ExternalWebURL to the domain name the outside world uses to access the Pelican web service, e.g. https://example-origin.org:8444  for a production service or https://localhost:8444  for a local deployment.

Notice that there is a port number, 8444, following the hostname in the URLs above. This is the default port for Pelican web server. When publishing ports from the container, you may choose a different port on the host machine to publish to. Two conventional options are 443 or 8080, which are default ports for a web service. If you publish 8444 port from the container to 443, or 8080 on the host machine, i.e. docker run -p 8444:443, you may remove the port number from Server.ExternalWebURL, i.e. https://example-origin.org. For other ports, you need to leave the port number in the URL.

Persisting Data Across Container Restarts

When running Pelican servers in containers, certain files and directories need to persist across container restarts to maintain server identity, configurations, and data. Without proper volume mounts, restarting a container will cause data loss, including cryptographic keys, databases, and configuration files.

This section describes which data should be persisted for each Pelican server type and how to configure volume mounts appropriately.

Quick Reference: Essential Volume Mounts

The table below shows the minimum recommended volume mounts for each server type at the directory level. For additional information about using config params to set the location of specific files in your container, see the section for your service.

Understanding Default Paths

Pelican uses different default paths depending on whether the server runs as root (which is the case in containers):

• Root user (including root-ly containers such as in Docker, and in unprivileged containers where pelican still appears to run as PID 0): Configuration and data are stored in /etc/pelican and /var/lib/pelican
• Non-root user: Configuration and data are stored in ~/.config/pelican

All examples below assume the server runs as root inside a container.

Mounting Directories vs Individual Files

The volume mount examples in this document show mounting entire directories (e.g., -v /host/pelican/config:/etc/pelican). This is the recommended approach for Docker deployments as it’s simpler and ensures all related files are persisted together.

However, in some deployment scenarios (such as Kubernetes with sealed secrets), you may need to mount individual files instead of entire directories. In such cases, consult the configuration parameter documentation linked for each file to determine or customize the file’s path. Each file’s default location is also specified in the sections below.

Data That Should Persist for All Server Types

The following data should be persisted for all Pelican server types (Origin, Cache, Director, Registry):

Issuer Keys Directory

Configuration: To customize the path, see the IssuerKeysDirectory parameter.

Purpose: Contains PEM-encoded ECDSA private keys used to sign credentials issued by the server. These keys establish the server’s cryptographic identity within the federation.

Default Path: /etc/pelican/issuer-keys

Best Practice: Pre-generate these keys outside the container and mount them into the container’s issuer keys directory. This prevents accidental re-generation of keys in the event of a typo in volume mounts, as the keys remain persisted on the host.

Volume Mount:

-v /host/pelican/issuer-keys:/etc/pelican/issuer-keys

For mounting individual key files:

-v /host/pelican/issuer.pem:/etc/pelican/issuer-keys/issuer.pem

Why it must persist: If these keys are lost, the server will generate new ones, causing authentication failures with other federation services that trust the old keys. For Caches/Origins, this is the source of the “namespace/service is registered with a different key” error. When this error is encountered, you must contact your federation’s administrators to have them delete the old registration. For security, this cannot be fixed without help from the federation administrators.

Server Database

Configuration: To customize the path, see the Server.DbLocation parameter.

Purpose: Stores server state and operational data.

Default Path: /var/lib/pelican/pelican.sqlite

Volume Mount:

-v /host/pelican/lib:/var/lib/pelican

For mounting individual database files:

-v /host/pelican/pelican.sqlite:/var/lib/pelican/pelican.sqlite

Web UI Password File

Configuration: To customize the path, see the Server.UIPasswordFile parameter.

Purpose: Stores the hashed+salted password for accessing the server’s web UI.

Default Path: /etc/pelican/server-web-passwd

Volume Mount:

-v /host/pelican/config:/etc/pelican

For mounting individual password files:

-v /host/pelican/server-web-passwd:/etc/pelican/server-web-passwd

TLS Certificates and Keys

Configuration: To customize paths, see Server.TLSCertificateChain, Server.TLSKey, and Server.TLSCACertificateFile parameters.

Purpose: TLS certificate chain and private key for HTTPS communication. Generally you must mount these into your container because while Pelican will generate them if they’re not provided, the Pelican-generated versions are self-signed and will not be trusted by most other web services.

Default Paths:

• Certificate chain: /etc/pelican/certificates/tls.crt
• Private key: /etc/pelican/certificates/tls.key
• CA certificate: /etc/pelican/certificates/tlsca.pem

Volume Mount:

-v /host/pelican/certificates:/etc/pelican/certificates

For mounting individual certificate files:

-v /host/tls.crt:/etc/pelican/certificates/tls.crt
-v /host/tls.key:/etc/pelican/certificates/tls.key
-v /host/tlsca.pem:/etc/pelican/certificates/tlsca.pem

Session Secret File

Configuration: To customize the path, see the Server.SessionSecretFile parameter.

Purpose: Secret used for encrypting/decrypting session data for the web UI and CSRF protection.

Default Path: /etc/pelican/session-secret

Volume Mount: Included in the /etc/pelican mount above.

Log Files (Optional)

Configuration: To send service logs to a file rather than stdout/stderr, see the Logging.LogLocation parameter.

Purpose: Server log files for troubleshooting and monitoring.

Default Path: Logs are written to stderr/stdout by default, but can be redirected to a file.

Volume Mount:

-v /host/pelican/logs:/var/log/pelican

Prometheus Monitoring Data (Optional)

Configuration: To customize the path, see the Monitoring.DataLocation parameter.

Purpose: Prometheus monitoring data and metrics.

Default Path: /var/lib/pelican/monitoring/data

Volume Mount:

-v /host/pelican/monitoring:/var/lib/pelican/monitoring

OIDC Client Credentials (If Using OIDC)

Configuration: To customize paths, see OIDC.ClientIDFile and OIDC.ClientSecretFile parameters.

Purpose: OIDC client ID and secret for OAuth2/OIDC authentication.

Default Paths:

• Client ID: /etc/pelican/oidc-client-id
• Client secret: /etc/pelican/oidc-client-secret

Volume Mount:

-v /host/pelican/config:/etc/pelican

For mounting individual credential files:

-v /host/oidc-client-id:/etc/pelican/oidc-client-id
-v /host/oidc-client-secret:/etc/pelican/oidc-client-secret

Origin-Specific Persistent Data

In addition to the common data above, Origins need to persist:

Origin Database

Configuration: To customize the path, see the Origin.DbLocation parameter.

Purpose: Origin-specific operational data and state.

Default Path: /var/lib/pelican/origin.sqlite

Volume Mount: Included in the /var/lib/pelican mount above.

Origin Data Directory

Configuration: Use the Origin.Exports parameter to define exports. For example, in /etc/pelican/pelican.yaml:

Origin:
  Exports:
    - StoragePrefix: /data
      FederationPrefix: /my-namespace

Purpose: The actual data being served by the Origin (if using POSIX storage).

Default Path: Configured via the Origin.Exports parameter.

Volume Mount Example:

-v /host/data:/data

Example: Running an Origin with Persistent Data

docker run -d \
  --name pelican-origin \
  -p 8444:8444 \
  -p 8443:8443 \
  -v /host/pelican/config:/etc/pelican \
  -v /host/pelican/lib:/var/lib/pelican \
  -v /host/pelican/certificates:/etc/pelican/certificates \
  -v /host/data:/data \
  hub.opensciencegrid.org/pelican_platform/origin:latest \
  serve

Cache-Specific Persistent Data

In addition to the common data, Caches need to persist:

Cache Storage Location

Configuration: Set Cache.StorageLocation to point to the mounted cache storage. In /etc/pelican/pelican.yaml:

Cache:
  StorageLocation: /var/cache/pelican

Purpose: Directory where cached objects are stored.

Default Path: /run/pelican/xrootd/cache

Important: The default path is in /run, which is typically a temporary filesystem. All Cache administrators should change this value to point at their mounted cache disks to ensure cached data persists across container restarts. If this configuration is not changed, cached data will fill up /run, potentially causing system instability and the Cache will lose all cached objects on restart.

Volume Mount:

-v /host/cache/storage:/var/cache/pelican

Why it must persist: Without persistence, the Cache loses all cached objects on restart, increasing load on Origins and slowing down data access due cache misses.

Example: Running a Cache with Persistent Data

docker run -d \
  --name pelican-cache \
  -p 8444:8444 \
  -p 8442:8442 \
  -v /host/pelican/config:/etc/pelican \
  -v /host/pelican/lib:/var/lib/pelican \
  -v /host/pelican/certificates:/etc/pelican/certificates \
  -v /host/cache/storage:/var/cache/pelican \
  hub.opensciencegrid.org/pelican_platform/cache:latest \
  serve

Note: All Cache configuration, including federation URL, should be provided via /etc/pelican/pelican.yaml which is mounted into the container.

Director-Specific Persistent Data

In addition to the common data, Directors need to persist:

GeoIP Database (Optional)

Configuration: To customize paths, see Director.GeoIPLocation and Director.MaxMindKeyFile parameters.

Purpose: MaxMind GeoIP database for geographic routing optimization.

Default Path: /var/cache/pelican/maxmind/GeoLite2-City.mmdb

Volume Mount:

-v /host/pelican/maxmind:/var/cache/pelican/maxmind

Example: Running a Director with Persistent Data

docker run -d \
  --name pelican-director \
  -p 8444:8444 \
  -v /host/pelican/config:/etc/pelican \
  -v /host/pelican/lib:/var/lib/pelican \
  -v /host/pelican/certificates:/etc/pelican/certificates \
  -v /host/pelican/maxmind:/var/cache/pelican/maxmind \
  hub.opensciencegrid.org/pelican_platform/director:latest \
  serve

Note: All Director configuration, including federation URL, should be provided via /etc/pelican/pelican.yaml which is mounted into the container.

Registry-Specific Persistent Data

Registries only require persisting the directories/files discussed in the “Data That Should Persist for All Server Types” section.

Example: Running a Registry with Persistent Data

docker run -d \
  --name pelican-registry \
  -p 8444:8444 \
  -v /host/pelican/config:/etc/pelican \
  -v /host/pelican/lib:/var/lib/pelican \
  -v /host/pelican/certificates:/etc/pelican/certificates \
  hub.opensciencegrid.org/pelican_platform/registry:latest \
  serve

Using Docker Compose for Persistent Data

For production deployments, Docker Compose provides a cleaner way to manage volumes. Here’s an example for an Origin:

version: '3.8'
 
services:
  pelican-origin:
    image: hub.opensciencegrid.org/pelican_platform/origin:latest
    container_name: pelican-origin
    ports:
      - "8444:8444"
      - "8443:8443"
    volumes:
      - pelican-config:/etc/pelican
      - pelican-lib:/var/lib/pelican
      - pelican-certs:/etc/pelican/certificates
      - /path/to/data:/data
    command: serve
    restart: unless-stopped
 
volumes:
  pelican-config:
  pelican-lib:
  pelican-certs:

Note: Configuration including exports and federation URL should be provided in a pelican.yaml file placed in the pelican-config volume.

Best Practices

1. Always persist /etc/pelican and /var/lib/pelican: These directories contain most of the critical persistent data.

2. Use named volumes or bind mounts consistently: Avoid mixing approaches to prevent confusion.

3. Back up databases regularly: The SQLite databases in /var/lib/pelican should be backed up regularly.

4. Protect private keys: The issuer keys and TLS private keys in /etc/pelican are sensitive and should be protected with appropriate file system permissions on the host.

5. Monitor disk usage: Cache storage and monitoring data can grow over time. Monitor disk usage and configure appropriate limits.

6. Test container restarts: After setting up persistent volumes, test that the container can be stopped and restarted without losing data or generating new keys.

7. Document your volume mounts: Keep a record of which host directories are mounted to which container paths for each server.