Configuration

Parameters

Pelican is configured using a YAML (opens in a new tab) file found at /etc/pelican/pelican.yaml. Each config value is represented by a key-value pair. Below is an example configuration file with a key-value pair, a nested key-value pair, and a list of key-value pairs.

/etc/pelican/pelican.yaml
OriginUrl: https://localhost:8444
 
OIDC:
	ClientIDFile: /etc/pelican/oidc-client-id
 
Registry:
  Institutions:
    - id: 0
      name: institution0
    - id: 1
      name: institution1
Filter by component
origin

registry

director

client

cache

Clear Filter

ConfigLocations

ConfigLocations provides administrators a way to define a list of directories containing Pelican configuration files. Within a given directory, files are read in lexicographical order, and any keys that are defined in multiple files will take the value from the last file read. Directories are read in the order provided by the list. For example, specifying:

ConfigLocations: ["/configs1", "/configs2"]

will read files first from /configs1 and then from /configs2. If a key is defined in both /configs1 and /configs2, the value from /configs2 will be used. If /configs1 contains files a.yaml and b.yaml where both define the same key, the value from b.yaml will be used.

Subdirectories of the provided directories are not read. Only the root config file's ConfigLocations key is used, and any redefinitions are ignored.

Components: [*]

Type: stringSlice

Default:

Debug

A bool indicating whether Pelican should emit debug messages in its log. NOTE: this will override whatever is set within your configuration file under Logging.Level!

Components: [*]

Type: bool

Default: false

TLSSkipVerify

When set to true, Pelican will skip TLS verification. This allows a "man in the middle" attack on the connection but can simplify testing. Intended for developers.

Components: [origin, registry, director]

Type: bool

Default: false

IssuerKey

A filepath to the file containing a PEM-encoded ecdsa private key which later will be parsed into a JWK and serves as the private key to sign various JWTs issued by this server.

A public JWK will be derived from this private key and used as the key for token verification.

Components: [client, registry, director]

Type: filename

Default: $ConfigBase/issuer.jwk

GeoIPOverrides

A list of IP addresses whose GeoIP resolution should be overridden with the supplied Lat/Long coordinates (in decimal form). This affects both server ads (for determining the location of origins and caches) and incoming client requests (for determing where a client request is coming from).

Configuration takes an IP address (both regular and CIDR) and a Coordinate made up of a lat/long pair in decimal format. For example:

GeoIPOverrides: - IP: "123.234.123.234" Coordinate: Lat: 43.073904 Long: -89.384859 - IP: "ABCD::1234/112" Coordinate: Lat: 39.8281 Long: -98.5795

Will result in the IP address "123.234.123.234" being mapped to Madison, WI, and IP addresses in the range ABCD::0000-FFFF will be mapped to a field in Kansas.

Components: [director]

Type: object

Default: none

DisableHttpProxy

[Deprecated] A legacy configuration for disabling the client's HTTP proxy. See Client.DisableHttpProxy for new config.

Components: [client]

Type: bool

Default: false

DisableProxyFallback

[Deprecated] A legacy configuration for disabling the client's proxy fallback. See Client.DisableProxyFallback for new config.

Components: [client]

Type: bool

Default: false

MinimumDownloadSpeed

[Deprecated] A legacy configuration for setting the client's minimum download speed. See Client.MinimumDownloadSpeed for new config.

Components: [client]

Type: int

Default: 102400

Transport


DialerTimeout
Transport

Maximum time allowed for establishing a connection to target host.

Components: [client, registry, origin]

Type: duration

Default: 10s

DialerKeepAlive
Transport

Maximum time a TCP connection should be kept alive without any activity.

Components: [client, registry, origin]

Type: duration

Default: 30s

MaxIdleConns
Transport

Maximum number of idle connections that the HTTP client should maintain in its connection pool.

Components: [client, registry, origin]

Type: int

Default: 30

IdleConnTimeout
Transport

Maximum duration an idle connection should remain open in the connection pool.

Components: [client, registry, origin]

Type: duration

Default: 90s

TLSHandshakeTimeout
Transport

Maximum time allowed for the TLS handshake to complete when making an HTTPS connection.

Components: [client, registry, origin]

Type: duration

Default: 15s

ExpectContinueTimeout
Transport

Timeout to control how long the client should wait for the "Expect: 100-continue" response from the server before sending the request body.

Components: [client, registry, origin]

Type: duration

Default: 1s

ResponseHeaderTimeout
Transport

Maximum time the client should wait for the response headers to be received from the server.

Components: [client, registry, origin]

Type: duration

Default: 10s

Logging


Level
Logging

A string defining the log level of the client. Options include (going from most info to least): Trace, Debug, Info, Warn, Error, Fatal, Panic.

Components: [*]

Type: string

Default: Error

LogLocation
Logging

A filename defining a file to write log outputs to, if the user desires.

Components: [*]

Type: filename

Default: none

DisableProgressBars
Logging

A bool defining if progress bars should be enabled or not.

Components: [Client]

Type: bool

Default: false

Logging.Origin


Cms
Logging.Origin

Trace level of XRootD cluster management service, one of the main XRootD executables. Cms basically is a file (or asset) discovery service. Each server has a cmsd daemon which connect to a master one informing it if a server is available. XRootD asks cms where a file could be found and cms works to report back the server for where the file is located. Accepted values: trace, debug, info, warn, error, fatal, panic

Components: [origin]

Type: string

Default: error

Scitokens
Logging.Origin

Trace level of scitokens debug output within XRootD configuration. This entails token management and security credentials within XRootD. Accepted values: trace, debug, info, warn, error, fatal, panic

Components: [origin]

Type: string

Default: error

Xrd
Logging.Origin

Trace level of the eXtended Request Daemon within XRootD, another main XRootD executable. This reports information the XRootD protocol and works with cms. Accepted values: trace, debug, info, warn, error, fatal, panic

Components: [origin]

Type: string

Default: error

Xrootd
Logging.Origin

Trace options for XRootD debug output within XRootD configuration. This prefix is reserved for the xroot protocol, which is the component that sits on sockets and talks to clients as they query file-system info, open files, and read data. This is the protocol for XRootD (like http) and handles connections and requests. Accepted values: trace, debug, info, warn, error, fatal, panic

Components: [origin]

Type: string

Default: info

Http
Logging.Origin

Logging level for the HTTP component of the origin. Increasing to debug will cause the Xrootd daemon to log. Accepted values: trace, debug, info, warn, error, fatal, panic all headers and requests.

Components: [origin]

Type: string

Default: error

Ofs
Logging.Origin

Logging level of Xrootd's "Open File System" (ofs) subsystem. The OFS manages the file descriptor table and redirection/ error handling. Accepted values: trace, debug, info, warn, error, fatal, panic

Components: [origin]

Type: string

Default: error

Oss
Logging.Origin

Logging level of Xrootd's "Open Storage System" (oss) subsystem. The OSS manages the interaction with the underlying POSIX storage (open, read, write, close, etc). Accepted values: trace, debug, info, warn, error, fatal, panic

Components: [origin]

Type: string

Default: error

Logging.Cache


Http
Logging.Cache

Logging level for the HTTP component of the cache. Increasing to debug will cause the Xrootd daemon to log all headers and requests. Accepted values: trace, debug, info, warn, error, fatal, panic

Components: [cache]

Type: string

Default: error

Ofs
Logging.Cache

Trace level of XRootD's Open File System. This component cares about files and directories from the administrative perspective. This component is build on top of the Open Storage System component, which deals with things like file creation and reads and writes for files and directories. Accepted values: trace, debug, info, warn, error, fatal, panic

Components: [cache]

Type: string

Default: error

Pfc
Logging.Cache

Trace level of XRootD Proxy File Cache (XCache), the caching mechanism used by XRootD. This component entails information for caches/caching within XRootD. This component instantiates its own Open Storage System (OSS) to write local files to. Accepted values: trace, debug, info, warn, error, fatal, panic

Components: [cache]

Type: string

Default: info

Pss
Logging.Cache

Trace level of XRootD Proxy System Service. Variables this component reports include: number of remotes file opens, number of opens that failed, number of remote file closes, and number of closes that failed. Accepted values: trace, debug, info, warn, error, fatal, panic

Components: [cache]

Type: string

Default: error

Scitokens
Logging.Cache

Trace level of scitokens debug output within XRootD configuration. This entails token management and security credentials within XRootD. Accepted values: trace, debug, info, warn, error, fatal, panic

Components: [cache]

Type: string

Default: error

Xrd
Logging.Cache

Trace level of the eXtended Request Daemon within XRootD, another main XRootD executable. This reports information the XRootD protocol and works with cms. Accepted values: trace, debug, info, warn, error, fatal, panic

Components: [cache]

Type: string

Default: error

Xrootd
Logging.Cache

Trace options for XRootD debug output within XRootD configuration. This prefix is reserved for the xroot protocol, which is the component that sits on sockets and talks to clients as they query file-system info, open files, and read data. This is the protocol for XRootD (like http) and handles connections and requests. Accepted values: trace, debug, info, warn, error, fatal, panic

Components: [cache]

Type: string

Default: error

Federation


DiscoveryUrl
Federation

A URL pointing to the federation's metadata discovery host. NOTE: this does not work if the url contains a path!

Components: [*]

Type: url

Default: none

TopologyUrl
Federation

A URL for the top level OSG Topology location (a legacy integration). This URL is needed to retrieve authorization file information.

Components: [origin, cache]

Type: url

Default: none

TopologyNamespaceUrl
Federation

A URL containing namespace information for origins and caches configured via the OSG Topology application (a legacy integration). The URL should point to the hosted namespace.json.

Components: [director, registry]

Type: url

Default: none

TopologyDowntimeUrl
Federation

A URL for determining OSG topology server downtime information. The result of querying this URL is an XML file containing downtime information.

Components: [director]

Type: url

Default: none

TopologyReloadInterval
Federation

The frequency, in minutes, that topology should be reloaded.

Components: [director, registry]

Type: duration

Default: none

Client


StoppedTransferTimeout
Client

A timeout indicating when a "stopped transfer" event should be triggered.

Components: [client]

Type: duration

Default: 100s

SlowTransferRampupTime
Client

A duration indicating the ramp up period for a slow transfer.

Components: [client]

Type: duration

Default: 30s

SlowTransferWindow
Client

A duration indicating the sliding window over which to consider transfer speeds for slow transfers.

Components: [client]

Type: duration

Default: 30s

DisableHttpProxy
Client

A bool indicating whether the client's HTTP proxy should be disabled.

Components: [client]

Type: bool

Default: false

WorkerCount
Client

An integer indicating the number of file transfer tasks that should be executed in parallel.

Components: [client]

Type: int

Default: 5

DisableProxyFallback
Client

A bool indicating whether the a proxy fallback should be used by the client.

Components: [client]

Type: bool

Default: false

MinimumDownloadSpeed
Client

The minimum speed (in bytes per second) allowed for a client download before an error is thrown.

Components: [client]

Type: int

Default: 102400

MaximumDownloadSpeed
Client

The maximum speed allowed for a client to download a given file (enforced via rate limits). This is not intended for use by production clients but rather for unit tests; 0 disables the rate limit.

Components: [client]

Type: int

Default: 0

Origin


DbLocation
Origin

A filepath to the intended location of the origin's database.

Components: [origin]

Type: filename

Default: $ConfigBase/origin.sqlite

Url
Origin

The origin's configured URL, as reported to XRootD. This is the file transfer endpoint for the origin.

Components: [origin]

Type: url

Default: https://${Server.Hostname}:${Origin.Port}

Port
Origin

The TCP port to be used by the origin service for serving files. If set to 0, then a random open port will be used.

Components: [origin]

Type: int

Default: 8443

Exports
Origin

A list describing the origin's exports. Each item in the list describes a single namespace the origin exports:

  • StoragePrefix: The relevant path from the object store, e.g. for posix /my/dir

  • FederationPrefix: The namespace prefix that data from StoragePrefix is made available under within the federation

  • Capabilities: A list of the capabilities the origin is willing to support for the given export. Capabilities include: ["Reads", "PublicReads", "Writes", "Listings", "DirectReads"] where each of these has the same effect as the corresponding "Origin.Enable*" configuration, except scoped to the given export. If "PublicReads" is included, "Reads" is inferred.

  • SentinelLocation: A filename under StoragePrefix path for Pelican to check the storage directory exists and is correctly mounted. The value must be a file and contain no directory. Leave it empty to skip the check.

    You should always choose a distinct name for SentinelLocation. It should not be reused for other servers. If running in a containerized environment it should not be the name of the underlying physical host as that may change and lead to confusion. You need to manually create a file under path to StoragePrefix with the same name as SentinelLocation.

    Note that this parameter is only available for the POSIX backend.

    Example:

    Origin.Exports - StoragePrefix: /home/foo/bar FederationPrefix: /demo/project Capabilities: ["Reads", "PublicReads", "Writes", "Listings", "DirectReads"] SentinelLocation: demoproject_origin_A

If Origin.StorageType == "s3", the following additional fields are available:

  • S3Bucket: [OPTIONAL] See Origin.S3Bucket for details
  • S3AccessKeyfile: [OPTIONAL] See Origin.S3AccessKeyfile for details
  • S3SecretKeyfile: [OPTIONAL] See Origin.S3SecretKeyfile for details

If Origin.StorageType == "globus", the following additional fields are available:

  • GlobusCollectionID: [REQUIRED] See Origin.GlobusCollectionID for details
  • GlobusCollectionName: [OPTIONAL] See Origin.GlobusCollectionName for details

Components: [origin]

Type: object

Default: none

StorageType
Origin

The type of storage underpinning the origin. Currently supported types are "posix", "https", "s3", "globus", and "xroot".

Components: [origin]

Type: string

Default: posix

FederationPrefix
Origin

The namespace prefix of the origin's contents within the federation.

NOTE: This config option is incompatible with multiple exports defined via Origin.Exports and is ignored when the origin exports multiple prefixes.

Components: [origin]

Type: string

Default: none

StoragePrefix
Origin

A string indicating the path to the volume exported by an origin's underlying storage. For example, if the origin has a StorageType of "posix", this constitutes the path on disk exported by the origin for the federation. If the origin has a StorageType of "s3", this value is not currently used.

NOTE: This config option is incompatible with multiple exports defined via Origin.Exports and is ignored when the origin exports multiple prefixes.

Components: [origin]

Type: string

Default: none

ExportVolumes
Origin

A list of docker-style export volumes for the origin. Each item in the list describes a single volume the origin exports. This configuration is meant mostly to be used by passing the -v flag from the command line. Paths exported with this configuration will inherit the origin's abilities, so individual export configurations are not possible.

Components: [origin]

Type: stringSlice

Default:

EnablePublicReads
Origin

A boolean indicating whether the origin permits reads without valid authorization. When false, reads from the origin will require a properly-scoped authorization token signed by the origin's issuer.

NOTE: This config option is meant to configure an origin's capabilities, but can be used to configure a namespace when the origin exports only a single prefix or when every exported namespace should inherit the same configuration.

Components: [origin]

Type: bool

Default: false

EnableReads
Origin

A boolean indicating whether the origin permits any reads. When false, the origin may still allow writes.

NOTE: This config option is meant to configure an origin's capabilities, but can be used to configure a namespace when the origin exports only a single prefix or when every exported namespace should inherit the same configuration.

Components: [origin]

Type: bool

Default: true

EnableWrites
Origin

A boolean indicating whether the origin permits writes. All writes require authorization.

NOTE: This config option is meant to configure an origin's capabilities, but can be used to configure a namespace when the origin exports only a single prefix or when every exported namespace should inherit the same configuration.

Components: [origin]

Type: bool

Default: true

EnableListings
Origin

A boolean indicating whether the origin permits object listings. When true, clients can list the contents of the origin.

NOTE: This config option is meant to configure an origin's capabilities, but can be used to configure a namespace when the origin exports only a single prefix or when every exported namespace should inherit the same configuration.

Components: [origin]

Type: bool

Default: true

EnableDirectReads
Origin

A boolean indicating whether the origin permits direct reads. When true, the origin indicates that it is willing to interact directly with clients. When false, the origin is indicating it is only willing to interact with clients via a cache service.

NOTE: This config option is meant to configure an origin's capabilities, but can be used to configure a namespace when the origin exports only a single prefix or when every exported namespace should inherit the same configuration.

Components: [origin]

Type: bool

Default: true

ExportVolume
Origin

[Deprecated] Origin.ExportVolume is being deprecated and will be removed in a future release. It is replaced by Origin.ExportVolumes. A path to the volume exported by an origin.

Components: [origin]

Type: string

Default: none

RunLocation
Origin

A directory where temporary configurations will be stored for the XRootD daemon started by the origin.

For non-root servers, if $XDG_RUNTIME_DIR is not set, a temporary directory will be created (and removed on shutdown).

Components: [origin]

Type: filename

Default: $XDG_RUNTIME_DIR/pelican/origin

NamespacePrefix
Origin

[Deprecated] Origin.NamespacePrefix is being deprecated and will be removed in a future release. It's configuration is being replaced by either Origin.Exports.FederationPrefix or by Origin.FederationPrefix. Note that Origin.FederationPrefix is incompatible with multiple exports and requires that the origin exports only a single path.

The filepath prefix at which an origin's contents are made globally available, eg /pelican/PUBLIC.

Components: [origin]

Type: string

Default: none

EnableWrite
Origin

[Deprecated] Origin.EnableWrite is being deprecated and will be removed in a future release. It is replaced by Origin.EnableWrites.

A boolean indicating if an origin allows write access.

Components: [origin]

Type: bool

Default: true

EnableFallbackRead
Origin

[Deprecated] Origin.EnableFallbackRead is being deprecated and will be removed in a future release. It is replaced by Origin.EnableDirectReads.

Set to true if the origin permits clients to directly read from it when no cache service is available.

Components: [origin]

Type: bool

Default: false

Multiuser
Origin

A bool indicating whether an origin is "multiuser", ie whether the underlying XRootD instance must be configured in multi user mode.

Components: [origin]

Type: bool

Default: false

EnableCmsd
Origin

A bool indicating whether the origin should enable the cmsd daemon.

Components: [origin]

Type: bool

Default: true

EnableMacaroons
Origin

A bool indicating whether the origin allows clients to authenticate using macaroons.

Components: [origin]

Type: bool

Default: false

DirectorTest
Origin

A bool indicating whether the director should send file transfer tests to the origin.

If Origin.StorageType is set to values other than POSIX, this parameter is set to false.

Components: [origin]

Type: bool

Default: true

SelfTest
Origin

A bool indicating whether the origin should perform self health checks.

If Origin.StorageType is set to values other than POSIX, this parameter is set to false.

Components: [origin]

Type: bool

Default: true

SelfTestInterval
Origin

The interval of which the origin starts a new file transfer test to itself.

Components: [origin]

Type: duration

Default: 15s

EnableUI
Origin

Indicate whether the origin should enable its web UI.

Components: [origin]

Type: bool

Default: true

EnableOIDC
Origin

Indicate whether the origin should allow users to login to the admin website via OAuth2/OIDC with third-party authentication providers such as CILogon.

If set to true, it is recommended that you also set Server.UIAdminUsers to a list of users to give admin privilege. This is because origin admin website doesn't have a public, non-admin view, and an empty AdminUsers list will lead to "permission denied" error for all users logged into origin admin website via OAuth.

Components: [origin]

Type: bool

Default: false

EnableBroker
Origin

Indicate whether the origin should utilize the broker service to avoid the need for incoming connections.

Components: [origin]

Type: bool

Default: false

EnableIssuer
Origin

Enable the built-in issuer daemon for the origin.

Components: [origin]

Type: bool

Default: false

ScitokensRestrictedPaths
Origin

Enable the built-in issuer daemon for the origin.

Components: [origin]

Type: stringSlice

Default:

ScitokensMapSubject
Origin

Enable the built-in issuer daemon for the origin.

Components: [origin]

Type: bool

Default: false

ScitokensDefaultUser
Origin

Enable the built-in issuer daemon for the origin.

Components: [origin]

Type: string

Default: none

ScitokensUsernameClaim
Origin

Enable the built-in issuer daemon for the origin.

Components: [origin]

Type: string

Default: none

ScitokensNameMapFile
Origin

Enable the built-in issuer daemon for the origin.

Components: [origin]

Type: string

Default: none

XRootDPrefix
Origin

The directory prefix for the XRootD origin configuration files.

Components: [origin]

Type: string

Default: origin

EnableVoms
Origin

Enable X.509 / VOMS-based authentication. This allows HTTP clients to present X.509 client credentials in order to authenticate. The configuration of the authorization for these clients must be done by the admin; Pelican does not support automatic VOMS authorization configuration.

Components: [origin]

Type: bool

Default: true

EnableDirListing
Origin

[Deprecated] Origin.EnableDirListing is being deprecated and will be removed in a future release. It is replaced by Origin.EnableListings.

Allows the origin to enable collection listings. Needs to be enabled for recursive downloads to work properly and for directories to be visible.

Components: [origin]

Type: bool

Default: false

Mode
Origin

[Deprecated] Origin.Mode is being deprecated and will be removed in a future release. It is replaced by Origin.StorageType.

The backend mode to be used by an origin. Current values that can be selected from are either "posix" or "s3".

Components: [origin]

Type: string

Default: posix

S3ServiceName
Origin

[Deprecated] Origin.S3ServiceName was previously used in part to determine an export's FederationPrefix, but upstream changes no longer rely on this value. As of Pelican 7.7.0, setting this value no longer has any effect. AWSv4 signatures used by S3 servers to handle authentication now hardcode "s3" as their service name.

When constructing signed URLs for S3, this value is used as a part of the signature. It is almost always "s3". For more information about S3 service names, see https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-auth-using-authorization-header.html

Components: [origin]

Type: string

Default: none

S3Region
Origin

Objects in S3 are associated with a "region", which is specifically a part of AWS's infrastructure. Often, S3 endpoints that are not provided by Amazon use "us-east-1" as their region. This value is used when constructing signed URLs for getting authenticated objects from a bucket.

For more information about how Amazon uses regions, see https://docs.aws.amazon.com/general/latest/gr/s3.html

This value is REQUIRED for S3 origins.

Components: [origin]

Type: string

Default: none

S3Bucket
Origin

Note: This value is only for setting up an origin that exports one storage prefix. For multiple exports, use Origin.Exports

Objects in S3 are stored in "buckets", which have unique names at each S3 service URL (ie the URL that provides access to your objects). Setting a bucket restricts the origin to only serving objects from that bucket.

However, if the origin is meant to export all of the buckets associated with a given service URL, this value can be left unset IF all of those buckets are public and the origin is using path-style URLS. When this is the case, objects can be fetched from the origin at the path /federation/prefix/bucket-name/object-name.

Components: [origin]

Type: string

Default: none

S3ServiceUrl
Origin

The URL that provides API access your objects. When the S3 instance is hosted by Amazon, this is often "https://s3.us-east-1.amazonaws.com".

This value is REQUIRED for S3 origins.

Components: [origin]

Type: string

Default: none

S3AccessKeyfile
Origin

Note: This value is only for setting up an origin that exports one storage prefix. For multiple exports, use Origin.Exports

A path to a file containing an S3 access keyfile (also sometimes called an API key) for authenticated buckets when an origin is run in S3 mode.

This value is OPTIONAL for S3 origins, and only applies when an exported bucket requires authentication. It should not be used if the bucket is public or if the origin is meant to export all public buckets from the S3 service URL.

Components: [origin]

Type: filename

Default: none

S3SecretKeyfile
Origin

Note: This value is only for setting up an origin that exports one storage prefix. For multiple exports, use Origin.Exports

A path to a file containing an S3 secret keyfile for authenticated buckets when an origin is run in S3 mode.

This value is OPTIONAL for S3 origins, and only applies when an exported bucket requires authentication. It should not be used if the bucket is public or if the origin is meant to export all public buckets from the S3 service URL.

Components: [origin]

Type: filename

Default: none

S3UrlStyle
Origin

The style of S3 urls used by the service URL host. This can be either "path" if objects are fetched at <service-url>/<bucket>/<object> or "virtual" if objects are fetched at <bucket>.<service-url>/<object>.

This value is REQUIRED for S3 origins, but defaults to "path" if not set.

Components: [origin]

Type: string

Default: path

HttpServiceUrl
Origin

If Origin.StorageType is set to https, the service URL is used as the base for requests to the backend. To generate the request, the Origin.FederationPrefix is removed from the object name, then the result is joined with the service URL and storage prefix. For example, if one sets Origin.HTTPServiceUrl=https://example.com, Origin.StoragePrefix=/testfiles and Origin.FederationPrefix=/foo, then a request for an object named /foo/bar will generate a request to https://example.com/testfiles/bar.

Components: [origin]

Type: string

Default: none

HttpAuthTokenFile
Origin

When set, all requests from the origin to the http backend will include the contents of the file as a bearer token in the Authorization header.

If the origin backend is configured with the globus storage type, any value set here will be overridden with the filepath to the first file ending in .tok found in the $(Origin.GlobusConfigLocation)/tokens directory

Components: [origin]

Type: filename

Default: none

XRootServiceUrl
Origin

When the origin is configured to export another XRootD storage backend by setting Origin.StorageType = xroot, the XRootServiceUrl is used as the base for root protocol requests and should point at the upstream XRootD server.

Components: [origin]

Type: string

Default: none

GlobusCollectionID
Origin

Note: This value is only for setting up an origin that exports one Globus collection. For multiple exports, use Origin.Exports

Required if Origin.StorageType == "globus" AND Origin.Exports is not set.

Globus stores objects in Collections. The unique identifier of a Collection is the Collection UUID. You can find the Collection UUID at the bottom of your Collection's overview page.

Components: [origin]

Type: string

Default: none

GlobusCollectionName
Origin

Note: This value is only for setting up an origin that exports one storage prefix. For multiple exports, use Origin.Exports

An optional human-readable name to describe the Collection. This should set to the "Display Name" of your Collection in Globus. It is recommended to set this name; otherwise the UUID will be used as the Collection name.

Components: [origin]

Type: string

Default: none

GlobusClientIDFile
Origin

Required if Origin.StorageType == "globus" and OIDC.Issuer is not Globus

A filepath to the file containing the Globus ClientID. You need to create a new project and register a new confidential OAuth client: https://app.globus.org/settings/developers/registration/confidential_client/select-project Once registered, in the client page, find "Client UUID" and copy it to a file.

If the server uses Globus as the OIDC authentication provider, Pelican will use OIDC configuration for Globus storage access.

Components: [origin]

Type: filename

Default: none

GlobusClientSecretFile
Origin

Required if Origin.StorageType == "globus" and OIDC.Issuer is not Globus

A filepath to the file containing the Globus ClientID. You need to create a new project and register a new confidential OAuth client following the instruction in Origin.GlobusClientIDFile. In the client page, you need to "Add Client Secret" and copy the secret to a file.

If the server uses Globus as the OIDC authentication provider, Pelican will use OIDC configuration for Globus storage access.

Components: [origin]

Type: filename

Default: none

GlobusConfigLocation
Origin

A filepath to the folder containing Globus config and access tokens

Components: [origin]

Type: filename

Default: $XDG_RUNTIME_DIR/pelican/xrootd/origin/globus

LocalCache


RunLocation
LocalCache

The directory for the runtime files of the local cache.

Components: [localcache]

Type: filename

Default: $XDG_RUNTIME_DIR/pelican/localcache

DataLocation
LocalCache

The directory for the location of the cache data files - this is where the actual data in the cache is stored for the local cache.

Components: [localcache]

Type: filename

Default: $PELICAN_LOCALCACHE_RUNLOCATION/cache

Socket
LocalCache

The location of the socket used for client communication for the local cache.

Components: [localcache]

Type: filename

Default: $PELICAN_LOCALCACHE_RUNLOCATION/cache.sock

Size
LocalCache

The maximum size of the local cache. If not set, it is assumed the entire device can be used. This parameter can be provided with units (e.g., 20GB, 150MB); if no unit is provided, then it is assumed to be in bytes.

Components: [localcache]

Type: string

Default: 0

HighWaterMarkPercentage
LocalCache

A percentage value where the cache cleanup routines will triggered. Once the cache usage of completed files hits the high water mark, files will be deleted until the usage hits the low water mark.

Components: [localcache]

Type: int

Default: 95

LowWaterMarkPercentage
LocalCache

A percentage value where the cache cleanup routines will complete. Once the cache usage of completed files hits the high water mark, files will be deleted until the usage hits the low water mark.

Components: [localcache]

Type: int

Default: 85

Cache


StorageLocation
Cache

An absolute path to the directory where xrootd will create its default namespace, meta, and data directories. For example, setting Cache.StorageLocation=/run/pelican/cache without specifying further Cache.DataLocations or Cache.MetaLocations values will result in the cache creating a directory structure like:

.
└── /run/pelican/cache/
    ├── data/
    │   ├── 00 # hexadecimal name values
    │   ├── 01
    │   ├── ...
    │   └── FF
    ├── meta/
    │   ├── 00 # hexadecimal name values
    │   ├── 01
    │   ├── ...
    │   └── FF
    └── namespace/
        ├── namespace1/
        │   ├── foo1.txt --> /run/pelican/cache/data/00
        │   └── foo2.txt --> /run/pelican/cache/data/01
        └── namespace2/
            └── bar.txt --> /run/pelican/cache/data/FF

In this setup, actual data files live at /run/pelican/cache/data and are given hexadecimal names, while references (symbolic links) to those files are stored in /run/pelican/cache/namespace. The meta directory is used for object metadata. Object requests to XRootD will be served from the namespace directories, and resolve the underlying object through these symbolic links.

We recommend tying the Cache.StorageLocation to a fast storage device, such as an SSD, to ensure optimal cache performance. If this directory does not already exist, it will be created by Pelican.

WARNING: The default value of /var/run/pelican should never be used for production caches, as this directory is typically cleared on system restarts, and may interfere with system services if it becomes full. Running a cache with the default value set will generate a warning at cache startup.

Components: [cache]

Type: string

Default: $XDG_RUNTIME_DIR/pelican/cache

NamespaceLocation
Cache

A cache's namespace directory is used to duplicate/recreate the federation's namespace structure, and stores symbolic links from object names to the actual data files (see Cache.StorageLocation for extra information). For example, requesting /foo/bar.txt from a cache will check for the existence of a symbolic link at ${Cache.NamespaceLocation}/foo/bar.txt, and if it exists, the cache will serve the data file at the location the symbolic link points to.

If this directory does not already exist, it will be created by Pelican.

WARNING: It's important that any values for Cache.DataLocations and Cache.MetaLocations are NOT subdirectories of Cache.NamespaceLocation, as this will make the raw data/meta files accessible through the cache's namespace structure, which is undefined behavior.

Components: [cache]

Type: string

Default: ${Cache.StorageLocation}/namespace

DataLocations
Cache

A list of absolute filesystem paths/directories where the cache's object data will be stored. This list of directories can be used to string together multiple storage devices to increase the cache's storage capacity, as long as each of the directories is accessible by the cache service. For example, setting Cache.DataLocations=["/mnt/cache1", "/mnt/cache2"] will result in splitting cache data between two mounted drives, /mnt/cache1 and /mnt/cache2. As such, these drives should be fast storage devices, such as SSDs.

For more information, see the xrootd oss documentation for the oss.space directive as well as the xrootd pfc documentation for the pfc.spaces directive.

If this directory does not already exist, it will be created by Pelican.

WARNING: It's important that any values for Cache.DataLocations are NOT subdirectories of Cache.NamespaceLocation, as this will make the raw data files accessible through the cache's namespace structure, which is undefined behavior.

Components: [cache]

Type: stringSlice

Default: ${Cache.StorageLocation}/data

MetaLocations
Cache

A list of absolute filesystem paths/directories where the cache's object metadata will be stored. Values in this list may point to separate drives as long as they're accessible by the cache service. For example, setting Cache.MetaLocations=["/mnt/meta1", "/mnt/meta2"] will result in splitting cache metadata between two the mounted drives. As such, these drives should be fast storage devices, such as SSDs.

For more information, see the xrootd oss documentation for the oss.space directive as well as the xrootd pfc documentation for the pfc.spaces directive.

If this directory does not already exist, it will be created by Pelican.

WARNING: It's important that any values for Cache.MetaLocations are NOT subdirectories of Cache.NamespaceLocation, as this will make the raw metadata files accessible through the cache's namespace structure, which is undefined behavior.

Components: [cache]

Type: stringSlice

Default: ${Cache.StorageLocation}/meta

LocalRoot
Cache

[Deprecated] Cache.LocalRoot is deprecated and replaced by Cache.StorageLocation.

Components: [cache]

Type: string

Default: $XDG_RUNTIME_DIR/pelican/cache

DataLocation
Cache

[Deprecated] Cache.DataLocation is being deprecated and will be removed in a future release. It is replaced by Cache.StorageLocation

Components: [cache]

Type: string

Default: $XDG_RUNTIME_DIR/pelican/cache

ExportLocation
Cache

A path that's relative to the Cache.NamespaceLocation where the cache will expose its contents. This path can be used to control which namespaces are available through the cache. For example, setting Cache.ExportLocation: /foo will only expose the /foo namespace to clients.

Components: [cache]

Type: string

Default: /

RunLocation
Cache

A directory where temporary configurations will be stored for the XRootD daemon started by the cache.

For non-root servers, if $XDG_RUNTIME_DIR is not set, a temporary directory will be created (and removed on shutdown).

Components: [cache]

Type: filename

Default: $XDG_RUNTIME_DIR/pelican/cache

SentinelLocation
Cache

A filename under Cache.DataLocation path for Pelican to check the storage directory exists and is correctly mounted. The value must be a file and contain no directory. Leave it empty to skip the check.

You should always choose a distinct name for Cache.SentinelLocation. It should not be reused for other servers. If running in a containerized environment it should not be the name of the underlying physical host as that may change and lead to confusion. You need to manually create a file under path to Cache.DataLocation with the same name as Cache.SentinelLocation.

Components: [cache]

Type: filename

Default: none

XRootDPrefix
Cache

The directory prefix for the XRootD cache configuration files.

Components: [cache]

Type: string

Default: cache

Url
Cache

The cache's configured URL, as reported to XRootD. This is the file transfer endpoint for the cache.

Components: [cache]

Type: url

Default: https://${Server.Hostname}:${Cache.Port}

Port
Cache

The TCP port the cache service should use. If set to 0, then a random open port will be used.

Components: [cache]

Type: int

Default: 8442

LowWatermark
Cache

A value of cache disk usage that stops the purging of cached files.

The value should be either a percentage integer of total available disk space (default is 90), or a number suffixed by k, m, g, or t. In which case, they must be absolute sizes in k (kilo-), m (mega-), g (giga-), or t (tera-) bytes, respectively.

Components: [cache]

Type: string

Default: 90

HighWaterMark
Cache

A value of cache disk usage that triggers the purging of cached files.

The value should be either a percentage integer of total available disk space (default is 95), or a number suffixed by k, m, g, or t. In which case, they must be absolute sizes in k (kilo-), m (mega-), g (giga-), or t (tera-) bytes, respectively.

Components: [cache]

Type: string

Default: 95

EnableVoms
Cache

Enable X.509 / VOMS-based authentication for the cache. This allows HTTP clients to present X.509 client credentials in order to authenticate. The configuration of the authorization for these clients must be done by the admin; Pelican does not support automatic VOMS authorization configuration.

Components: [cache]

Type: bool

Default: false

Concurrency
Cache

This value represents the maximum number of connections to a cache for XRootD throttling. When this value is set, it enables the XRootD throttling plugin and will set the maximum number of connections to this value.

Components: [cache]

Type: int

Default: none

EnableLotman
Cache

LotMan is a library that provides management of storage space in the cache.

Components: [cache]

Type: bool

Default: false

PermittedNamespaces
Cache

A list of namespaces the cache is allowed to pull from. If the list is empty or this option is unset, it's assumed that the cache is allowed to access any namespace that's advertised to the director. Otherwise, it will only be allowed to access the listed namespaces.

Components: [cache]

Type: stringSlice

Default:

SelfTest
Cache

A bool indicating whether the cache should perform self health checks.

Components: [cache]

Type: bool

Default: true

SelfTestInterval
Cache

The interval of which the cache starts a new file transfer test to itself.

Components: [cache]

Type: duration

Default: 15s

EnableOIDC
Cache

Indicate whether the cache should allow users to login to the admin website via OAuth2/OIDC with third-party authentication providers such as CILogon.

If set to true, it is recommended that you also set Server.UIAdminUsers to a list of users to give admin privilege. This is because cache admin website doesn't have a public, non-admin view, and an empty AdminUsers list will lead to "permission denied" error for all users logged into cache admin website via OAuth.

Components: [cache]

Type: bool

Default: false

BlocksToPrefetch
Cache

The number of 128 kilobyte blocks the cache will read ahead when recieving requests. This will put the data in the cache potentially before it is needed and reduce the latency to the client when a request is made. However, it can also cause many extra requests to an origin and potentially overload it when unnessesary. As such, this is turned off by default.

Components: [cache]

Type: int

Default: 0

DefaultCacheTimeout
Cache

The default value of the cache operation timeout if one is not specified by the client.

Newer clients should always specify a timeout; changing this default is rarely necessary.

Components: [cache]

Type: duration

Default: 9.5s

Director


DbLocation
Director

A filepath to the intended location of the director's database, where server downtime info is stored.

Components: [director]

Type: filename

Default: $ConfigBase/director.sqlite

DefaultResponse
Director

The default response type of a redirect for a director instance. Can be either "cache" or "origin". If a director is hosted at https://director.com, then a GET request to https://director.com/foo/bar.txt will either redirect to the nearest cache for namespace /foo if Director.DefaultResponse is set to "cache" or to the origin for /foo if it is set to "origin".

Components: [director]

Type: string

Default: cache

CachesPullFromCaches
Director

In the "origin" response, the director returns a list of origins that can serve the object. If Director.CachesPullFromCaches is set to true (default is false), the director then appends a list of caches that can serve the object to the original response.

Components: [director]

Type: bool

Default: false

CacheResponseHostnames
Director

A list of virtual hostnames for the director. If a request is sent by the client to one of these hostnames, the director assumes it should respond with a redirect to a cache.

If present, the hostname is taken from the X-Forwarded-Host header in the request. Otherwise, Host is used.

Components: [director]

Type: stringSlice

Default:

CacheSortMethod
Director

When the director recieves a client request that needs to be redirected to a cache, it will use this method to determine the ordering of the caches. The default method is "distance", which sorts caches by their spherical distance from the client.

Available methods include:

  • "distance": Sorts caches by their spherical distance from the client.
  • "distanceAndLoad": Sorts caches according to both their distance and a calculated load. This is currently a placeholder, and returns the same ordering as "distance".
  • "random": Sorts caches randomly.
  • "adaptive": Sorts caches according to stochastically-generated weights that consider a combination of factors, including a cache's distance from the client, its IO load, and whether the cache already has the requested object. See details at https://github.com/PelicanPlatform/pelican/discussions/1198. Note that if Director.CheckCachePresence is set to false, then the adaptive algorithm cannot use the cache locality information.

Components: [director]

Type: string

Default: distance

OriginResponseHostnames
Director

A list of virtual hostnames for the director. If a request is sent by the client to one of these hostnames, the director assumes it should respond with a redirect to an origin.

If present, the hostname is taken from the X-Forwarded-Host header in the request. Otherwise, Host is used.

Components: [director]

Type: stringSlice

Default:

MaxMindKeyFile
Director

A filepath to a MaxMind API key. The director service uses the MaxMind GeoLite City database (available here) to determine which cache is nearest to a client's IP address. The database, if not already found, will be downloaded automatically when a director is served and a valid key is present.

Components: [director]

Type: filename

Default: none

GeoIPLocation
Director

A filepath to the intended location of the MaxMind GeoLite City database. This option can be used either to load an existing database, or to configure the preferred download location if Pelican has a MaxMind API key.

Components: [director]

Type: filename

Default: $ConfigBase/maxmind/GeoLite2-city.mmdb

MinStatResponse
Director

A positive integer indicating minimum number of origin's responses required for a stat call.

Components: [director]

Type: int

Default: 1

MaxStatResponse
Director

A positive integer indicating maximum number of origin's responses required for a stat call. stat call will cancel the rest of the ongoing query if max response is hit.

Components: [director]

Type: int

Default: 1

EnableStat
Director

Before redirecting a cache (or, for direct reads or writes, a client) to an origin, query the origin to see if the object is present.

Enabling this option generates slightly more load on the origin; however, it provides improved error messages and allows a namespace to effectively be split across multiple origins.

Components: [director]

Type: bool

Default: true

CheckOriginPresence
Director

Before redirecting a cache (or, for direct reads or writes, a client) to an origin, query the origin to see if the object is present.

Enabling this option generates slightly more load on the origin; however, it provides improved error messages and allows a namespace to effectively be split across multiple origins.

Components: [director]

Type: bool

Default: true

AssumePresenceAtSingleOrigin
Director

If Director.CheckOriginPresence is enabled, the director will check for object presence at the origin before redirecting a client.

If this option is enabled and there's only one possible origin for the object, then the check will be skipped.

Enabling this option will reduce load on single-origin namespaces but clients will get less informative error messages.

Components: [director]

Type: bool

Default: true

CheckCachePresence
Director

Before redirecting a client to a cache, query the cache to see if the object is present at the cache.

Enabling this option improves the cache selection algorithm, allowing the director to prefer caches nearby the client with the object over caches without the object.

Components: [director]

Type: bool

Default: true

StatTimeout
Director

The timeout for a single stat request.

Components: [director]

Type: duration

Default: 2000ms

StatConcurrencyLimit
Director

The maximum number of concurrent stat request to a single origin server. Additional requests are blocked until total requests for the origin is below limit. See golang.org/x/sync/errgroup for detail

Components: [director]

Type: int

Default: 1000

AdvertisementTTL
Director

The time to live (TTL) of director's internal cache to store origins and caches advertisement.

Components: [director]

Type: duration

Default: 15m

OriginCacheHealthTestInterval
Director

The interval of which director issues a new file transfer test to all the registered origins and caches.

Components: [director]

Type: duration

Default: 15s

EnableBroker
Director

Whether the director should also run the connection brokering service.

Components: [director]

Type: bool

Default: true

FilteredServers
Director

A list of server resource names that the Director should consider in downtime, preventing the Director from issuing redirects to them. Additional downtimes are aggregated from Topology (when the Director is served in OSDF mode), and the Web UI.

Components: [director]

Type: stringSlice

Default:

SupportContactEmail
Director

An Email address to receive issues and help requests for the federation the director is hosting. The values will be displayed on the director web interface if provided. We highly recommend director admin to fill out this field.

Components: [director]

Type: string

Default: none

SupportContactUrl
Director

A URL where user can find support information. Can be your website, GitHub discussion, or third-party support portal for the federation the director is hosting. The values will be displayed on the director web interface if provided. We highly recommend director admin to fill out this field.

Components: [director]

Type: string

Default: none

EnableOIDC
Director

Indicate whether the director should allow users to login to the admin website via OAuth2/OIDC with third-party authentication providers such as CILogon.

If set to true, it is recommended that you also set Server.UIAdminUsers to a list of users to give admin privilege. This is because origin admin website doesn't have a public, non-admin view, and an empty AdminUsers list will lead to "permission denied" error for all users logged into origin admin website via OAuth.

Components: [director]

Type: bool

Default: false

X509ClientAuthenticationPrefixes
Director

A list of object prefixes where the origin uses X.509 client authentication.

If a cache requests an object starting with one of these prefixes, then it will be instructed by the director to use X.509 client authentication if available.

This setting allows for compatibility with specific legacy OSDF origins and is not needed for new origins.

Components: [director]

Type: stringSlice

Default:

CachePresenceTTL
Director

If Director.CheckCachePresence is enabled, the director will check with remote cache to see if the object is present before redirecting a client.

This parameter controls how long the director will cache the result of the lookup. Longer values will reduce the load generated on the caches but may reduce the accuracy of the result (as the contents of the cache will change over time).

Components: [director]

Type: duration

Default: 1m

CachePresenceCapacity
Director

If Director.CheckCachePresence is enabled, the director will check with remote cache to see if the object is present before redirecting a client.

This parameter controls how many responses, per-service, will be cached.

Components: [director]

Type: int

Default: 10000

Registry


DbLocation
Registry

A filepath to the intended location of the namespace registry's database.

Components: [registry]

Type: filename

Default: $ConfigBase/ns-registry.sqlite

RequireKeyChaining
Registry

Specifies whether namespaces requesting registration must possess a key matching any already-registered super/sub namespaces. For example, if true and a namespace /foo/bar is already registered, then registration of /foo or /foo/bar/baz can only be done using keys registered to /foo/bar.

Components: [registry]

Type: bool

Default: true

AdminUsers
Registry

[Deprecated] Registry.AdminUsers is deprecated and will be removed in the future releases. Please migrate to use Server.UIAdminUsers instead.

A string slice of "subject" claim of users to give admin permission for registry UI.

The "subject" claim should be the "CILogon User Identifier" from CILogon user page: https://cilogon.org/

Components: [registry]

Type: stringSlice

Default:

Institutions
Registry

A array of institution objects available to register. Users can only select from this list when they register a new namespace. Each object has name and id field where name is a human-readable name for the institution and id is a unique identifier for the institution. For Pelican running in OSDF alias, the id will be OSG ID.

For example:

- name: University of Wisconsin - Madison id: https://osg-htc.org/iid/01y2jtd41

Note that this value will take precedence over Registry.InstitutionsUrl if both are set.

Components: [registry]

Type: object

Default: none

CustomRegistrationFields
Registry

An array of objects specifying additional fields when registering namespaces.

The schema of the object is as follows:

- name: department_name type: enum required: true options: - name: Math id: math - name: Computer Science id: cs optionsUrl: https://example.com/options description: The department of the organization that holds this namespace

Note the following requirements:

  • name must be snake case with underline connecting words, i.e. department_name. The name displayed in the registration table will be converted from this field into a human-readble, space-separated name with the first letter(s) capitalized. i.e. department_name -> Department Name

  • type must be one of string, int, bool, datetime (Unix time in seconds), or enum.

  • options must be a non-empty yaml array for field with type enum. optionsUrl will be ignored if options is set.

    Example:

    options: - name: "Option A" id: "optionA"
  • description will show up in the web UI as helper text to help user understand the field

  • optionsUrl is a URL to provide a list of options for enum type field. The URL should respond to an anonymous GET request and return JSON response in the same format as the options field above

Components: [registry]

Type: object

Default: none

InstitutionsUrl
Registry

A url to get a list of available institutions for users to register their namespaces to. The url must accept a GET request with 200 response in JSON/YAML content with the following format:

JSON:

[ { "name": "University of Wisconsin - Madison", "id": " https://osg-htc.org/iid/01y2jtd41" } ]

YAML:

- name: University of Wisconsin - Madison id: " https://osg-htc.org/iid/01y2jtd41"

Where the id field will be stored in registry database and must be unique, and name field will be displayed in UI as the option.

Note that Pelican will cache the response of the url in a TTL cache with default refresh time of 15 minutes. Also note that `Registry.Institutions`` will take precedence over this value if both are set.

Components: [registry]

Type: url

Default: none

InstitutionsUrlReloadMinutes
Registry

Number of minutes that the Registry.InstitutionsUrl will be reloaded into the TTL cache.

Components: [registry]

Type: duration

Default: 15m

RequireCacheApproval
Registry

Only allow approved caches to join the federation and serve files. If set to true, caches can successfully self-register or registered via registry, but director won't direct traffic to the cache.

Components: [registry]

Type: bool

Default: false

RequireOriginApproval
Registry

Only allow approved origins to join the federation and serve files. If set to true, origins can successfully self-register or registered via registry, but director won't direct traffic to the origin, nor would files on the origin show up in the federation.

Components: [registry]

Type: bool

Default: false

Server


TLSCertificate
Server

A filepath to a file containing an X.509 host certificate to use for TLS authentication when running server components of Pelican.

If you override this filepath, you need to provide the matched-pair private key via Server.TLSKey and a Certificate Authority (CA) certificate via Server.TLSCACertificateFile.

Components: [origin, registry, director]

Type: filename

Default: $ConfigBase/certificates/tls.crt

TLSCACertificateFile
Server

A filepath to the TLS Certificate Authority (CA) certificate file, to be used by XRootD and internal HTTP client requests.

Do not override this filepath unless you want to provide your TLS host certificate

Components: [origin, registry, director]

Type: filename

Default: $ConfigBase/certificates/tlsca.pem

TLSCACertificateDirectory
Server

A filepath to the directory used for storing TLS Certificate Authority (CA) certificate to be used by XRootD only.

This is exclusive with Server.TLSCACertificateFile for XRootD and this value takes priority over Server.TLSCACertificateFile.

Components: [origin, registry, director]

Type: string

Default: none

TLSCAKey
Server

The name of a file containing a private key corresponding to the TLSCACertificate. Used when running server components of Pelican.

Components: [origin, registry, director]

Type: filename

Default: $ConfigBase/certificates/tlsca.key

TLSKey
Server

The name of a file containing a private key corresponding to the TLSCertificate. Used when running server components of Pelican.

Components: [origin, registry, director]

Type: filename

Default: $ConfigBase/certificates/tls.key

EnableUI
Server

Indicate whether a server should enable its web UI.

Components: [origin, registry, director, cache]

Type: bool

Default: true

WebPort
Server

The port number the Pelican web interface and internal web APIs will be bound to.

Components: [origin, director, registry]

Type: int

Default: 8444

WebHost
Server

A string-encoded IP address that the Pelican web engine is configured to listen on.

Components: [origin, director, registry]

Type: string

Default: 0.0.0.0

ExternalWebUrl
Server

A URL indicating the Pelican web interface and internal web APIs address as it appears externally.

Port number will be stripped if it's 443, from Server.WebPort or directly set through Server.ExternalWebUrl.

Components: [origin, director, registry]

Type: url

Default: https://${Server.Hostname}:${Server.WebPort} (for ${Server.WebPort} != 443)

Hostname
Server

The server's hostname, by default it's os.Hostname().

Components: [origin, director, registry]

Type: string

Default: none

IssuerUrl
Server

The URL and port at which the server's issuer can be accessed.

Components: [origin, director, registry]

Type: string

Default: none

IssuerHostname
Server

The hostname at which the server's issuer can be accessed.

Components: [origin, director, registry]

Type: string

Default: none

IssuerPort
Server

The port at which the server's issuer can be accessed.

Components: [origin, director, registry]

Type: int

Default: none

IssuerJwks
Server

A filepath indicating where the server's public JSON web keyset can be found.

Components: [origin, director, registry]

Type: filename

Default: none

Modules
Server

A list of modules to enable when running pelican in pelican serve mode.

Components: [*]

Type: stringSlice

Default:

UIActivationCodeFile
Server

If the server's web UI has not yet been configured, this file will contain the activation code necessary to turn it on.

Components: [origin, cache, registry, director]

Type: filename

Default: $ConfigBase/server-web-activation-code

UIPasswordFile
Server

A filepath specifying where the server's web UI password file should be stored.

Components: [origin, cache, registry, director]

Type: filename

Default: $ConfigBase/server-web-passwd

SessionSecretFile
Server

The filepath to the secret for encrypt/decrypt session data for Pelican web UI to initiate a session cookie.

This is used for sending redirect request for OAuth2 authentication follow. This is also used for CSRF auth key.

Components: [registry, director]

Type: filename

Default: $ConfigBase/session-secret The default content of the file is the hash of the concatenation of "pelican" and the DER form of ${IssuerKey}

RegistrationRetryInterval
Server

The duration of delay in origin/cache registration retry attempts if the initial registration call to registry was failed.

Components: [origin, cache]

Type: duration

Default: 10s

UILoginRateLimit
Server

The maximum number of requests a user can be made under the same IP address per second against the login endpoint

Components: [*]

Type: int

Default: 1

WebConfigFile
Server

A filepath to the file where web-based configuration changes are stored

Components: [*]

Type: filename

Default: $ConfigBase/web-config.yaml

UIAdminUsers
Server

A string slice of "subject" claim of users to give admin permission for the server admin website, who are authenticated through OAuth/OIDC.

The "subject" claim should be the "CILogon User Identifier" from CILogon user page: https://cilogon.org/

Components: [registry, origin, cache]

Type: stringSlice

Default:

StartupTimeout
Server

The amount of time the pelican server will wait for its components and services to startup. If the timeout is hit while waiting on a component, the server will shutdown.

Components: [origin, cache, registry, director]

Type: duration

Default: 10s

EnablePprof
Server

A boolean to enable or disable the pprof endpoints for debugging.

Components: [*]

Type: bool

Default: false

Issuer


TomcatLocation
Issuer

Location of the system tomcat installation

Components: [origin]

Type: string

Default: /opt/tomcat

ScitokensServerLocation
Issuer

Location of the scitoken server installation

Components: [origin]

Type: string

Default: /opt/scitokens-server

QDLLocation
Issuer

Location of the QDL language and scripts install on the system

Components: [origin]

Type: string

Default: /opt/qdl

IssuerClaimValue
Issuer

Contents of the issuer (iss) claim in the generated tokens

Components: [origin]

Type: string

Default: $(Server.ExternalWebUrl)

AuthenticationSource
Issuer

How users should authenticate with the issuer. Currently-supported values are:

  • none (default): No authentication is performed. All requests are successful and assumed to be a user named nobody.
  • OIDC: Use the server's OIDC configuration to authenticate with an external identity provider.

Components: [origin]

Type: string

Default: OIDC

OIDCAuthenticationRequirements
Issuer

A list of claim-value pairs that indicate required values from the OIDC ID token to authenticate.

Example:

- claim: idp_name value: University of Wisconsin-Madison

Would only allow tokens with "idp_name": "University of Wisconsin-Madison" set to authenticate.

Components: [origin]

Type: object

Default:

OIDCAuthenticationUserClaim
Issuer

The claim in the OIDC ID token to be used as the "username" for the issuer.

Components: [origin]

Type: string

Default: sub

UserStripDomain
Issuer

Some OIDC issuers generate a username of the form user@domain (such as john.doe@gmail.com); when UserStripDomain is enabled, Pelican will strip the domain when determining the username.

For example, the OIDC identity john.doe@gmail.com would map to john.doe.

Components: [origin]

Type: bool

Default: false

GroupSource
Issuer

How the issuer should determine group information based on the authenticated identity. Valid values are:

  • none (default): No group information should be used.
  • file: Read groups from an external, JSON-formatted file. The file should contain a single JSON object with keys corresponding to the "user" name and the value a list of strings that are interpreted as the user's groups.
  • oidc: Take group information from the identity token provided by the OIDC identity provider. Parses the value of the claim specified by Issuer.OIDCGroupClaim (defaults to "groups") as a list of groups. The value may either be a comma-separated string or an array of strings.

Components: [origin]

Type: string

Default: none

OIDCGroupClaim
Issuer

The claim in the OIDC ID token to be used as the group for the issuer. If the value is a string, it is assumed that a comma is used as a group delimiter; otherwise, an array of strings is assumed. Check the documentation of your OIDC provider to determine the appropriate claim name.

Components: [origin]

Type: string

Default: groups

GroupFile
Issuer

The location of a file containing group information. The file should contain a single JSON object with keys corresponding to the "user" name and the value a list of strings that are interpreted as the user's groups.

Components: [origin]

Type: string

Default: none

GroupRequirements
Issuer

Group membership requirements. A request must be mapped to one of the groups in this list to successfully authenticate.

Components: [origin]

Type: stringSlice

Default:

AuthorizationTemplates
Issuer

The authorizations that should be generated for an authenticated request. Value should be a list of authorized actions.

Each action is a key-value pair with the following keys defined:

  • actions: A list of authorized actions. Valid string values are read, modify, and create.
  • prefix: A prefix where the actions are authorized. If the prefix contains the substring $USER, the string is replaced with the authenticated username. If the prefix contains the substring $GROUP, then an authorization is emitted for each group authenticated.

For example, if the request is authenticated as user bbockelm with groups dept_a and dept_b, then the following configuration:

- actions: ["read", "create"] prefix: /projects/$GROUP - actions: ["read", "modify"] prefix: /home/$USER

will result in the following authorizations:

  • read /projects/dept_a
  • create /projects/dept_a
  • read /projects/dept_b
  • create /projects/dept_b
  • read /home/bbockelm
  • modify /home/bbockelm

Components: [origin]

Type: object

Default:

OIDC


ClientIDFile
OIDC

A filepath to a file containing an OIDC Client ID.

This is used by the namespace registry to allow OAuth2/OIDC login and authenticated namespace registration. By default, Pelican uses CILogon as the authentication provider. You need to first register an OIDC client at CILogon: https://cilogon.org/oauth2/register. If you'd like to use other authentication providers, you need to change other endpoint parameters under OIDC configuration to the endpoints of your provider, such as OIDC.AuthorizationEndpoint, OIDC.UserInfoEndpoint, etc.

OIDC.ClientIDFile is mutually exclusive with OIDC.ClientID. The value of OIDC.ClientID will override the value of OIDC.ClientIDFile if both are set.

This is a required parameter for the registry server. This is a required parameter for the origin/cache/director server if Origin.EnableOIDC/Cache.EnableOIDC/Director.EnableOIDC is set to true, respectively.

Components: [registry, origin, cache, director]

Type: filename

Default: $ConfigBase/oidc-client-id

ClientID
OIDC

The OIDC ClientID to use for the server. This is mutually exclusive with OIDC.ClientIDFile. The value of OIDC.ClientID will override the value of OIDC.ClientIDFile if both are set.

This is a required parameter for the registry server. This is a required parameter for the origin/cache/director server if Origin.EnableOIDC/Cache.EnableOIDC/Director.EnableOIDC is set to true, respectively.

Components: [registry, origin, cache, director]

Type: string

Default: none

ClientSecretFile
OIDC

A filepath to a file containing an OIDC Client Secret. This is used by the namespace registry to establish OIDC information for authenticated registration.

This is a required parameter for the registry server. This is a required parameter for the origin/cache/director server if Origin.EnableOIDC/Cache.EnableOIDC/Director.EnableOIDC is set to true, respectively.

Components: [registry, origin, cache, director]

Type: filename

Default: $ConfigBase/oidc-client-secret

DeviceAuthEndpoint
OIDC

A URL describing an OIDC Device Auth Endpoint. This is used by the namespace registry to establish OIDC information for authenticated registration. The default value is set to the URL from CILogon.

Components: [registry, origin, cache, director]

Type: url

Default: https://cilogon.org/oauth2/device_authorization

TokenEndpoint
OIDC

A URL describing an OIDC Token Endpoint. This is used by the namespace registry to establish OIDC information for authenticated registration. The default value is set to the URL from CILogon.

Components: [registry, origin, cache, director]

Type: url

Default: https://cilogon.org/oauth2/token

UserInfoEndpoint
OIDC

A URL describing an OIDC User Info Endpoint. This is used by the namespace registry to establish OIDC information for authenticated registration. The default value is set to the URL from CILogon.

Components: [registry, origin, cache, director]

Type: url

Default: https://cilogon.org/oauth2/userinfo

AuthorizationEndpoint
OIDC

A URL containing the OIDC authorization endpoint. The default value is set to the URL from CILogon.

Components: [registry, origin, cache, director]

Type: url

Default: https://cilogon.org/authorize

Issuer
OIDC

The URL of the OIDC issuer. If set, OIDC auto-discovery may be used to find other endpoints (token, user info, device auth). The URL should not contain a path unless your authentication server enables multi-tenant support.

If the OIDC auto-discovery failed, Pelican will fall back to use individual endpoints set in the configuration. For any unset endpoints, Pelican will use default values, which are from CILogon.

For CILogon, it's https://cilogon.org For Globus, it's https://auth.globus.org

Components: [registry, origin, cache, director]

Type: url

Default: none

ClientRedirectHostname
OIDC

The hostname for the OIDC client redirect URL that the OIDC provider will redirect to after the user is authenticated.

For development use only. Useful when developing in a container and you want to expose localhost instead of container hostname to your OAuth provider.

Components: [registry, origin, cache, director]

Type: string

Default: none

Xrootd


Port
Xrootd

[Deprecated] Xrootd.Port is deprecated and will be removed in the future release. Please migrate to use Origin.Port or Cache.Port instead.

The port over which XRootD should be made available. This setting is deprecated; please use the Cache.Port or Origin.Port, as appropriate, for the server.

Components: [origin, cache]

Type: int

Default: 8443

RunLocation
Xrootd

[Deprecated] Xrootd.RunLocation is deprecated and will be removed in a future release. Please migrate to use Cache.RunLocation or Origin.RunLocation instead.

A directory where temporary configurations will be stored for the XRootD daemon started by the origin or cache. For non-root servers, if $XDG_RUNTIME_DIR is not set, a temporary directory will be created (and removed on shutdown). This setting is deprecated; please use the Cache.RunLocation or Origin.RunLocation, as appropriate, for the server.

Components: [origin, cache]

Type: filename

Default: $XDG_RUNTIME_DIR/pelican

ConfigFile
Xrootd

The absolute path to an XRootD configuration file for customized XRootD configuration. This should only be used by admins with experience in configuring XRootD directly.

Xrootd.ConfigFile will be used as the continuation of the Pelican generated XRootD configuration, via the continue directive. Existing configuration values may be overwritten or appended.

Refer to Configuration File Continuation for details

Components: [origin, cache]

Type: filename

Default: none

RobotsTxtFile
Xrootd

Origins may be indexed by web search engines; to control the behavior of search engines, one may provide local policy via a robots.txt file.

If this file is not present, it will be auto-created with a default policy of blocking all indexing.

Components: [origin]

Type: filename

Default: $ConfigBase/robots.txt

ScitokensConfig
Xrootd

The location of a file configuring XRootD's token-based authorization subsystem. This file allows arbitrary changes to the authorization configuration and will be merged with any auto-generated configuration; it's recommended for use by experts only.

Components: [origin, cache]

Type: filename

Default: $ConfigBase/xrootd/scitokens.cfg

Mount
Xrootd

The mount path for an instance of XRootD.

Components: [origin]

Type: string

Default: none

MacaroonsKeyFile
Xrootd

The filepath to a Macaroons key for setting up authorization in XRootD.

Components: [origin]

Type: string

Default: none

Authfile
Xrootd

The filepath to an auth file for setting up authorization in XRootD.

Components: [origin]

Type: string

Default: none

AuthRefreshInterval
Xrootd

The interval used by XRootD (cache/origin) for refreshing Authfiles. This affects how often the server polls for upstream changes that might affect the authorization policy. For example, when applied to a cache, this affects how often origin permissions are polled for changes.

Components: [origin, cache]

Type: duration

Default: 5m

ManagerHost
Xrootd

A URL pointing toward the XRootD instance's Manager Host.

Components: [origin, cache]

Type: url

Default: none

ManagerPort
Xrootd

The port at which the XRootD instance's Manager Host is available.

Components: [origin, cache]

Type: int

Default: 1213

SummaryMonitoringHost
Xrootd

A URL pointing toward the XRootD instance's Summary Monitoring Host.

Components: [origin, cache]

Type: url

Default: none

SummaryMonitoringPort
Xrootd

The port at which the XRootD instance's Summary Monitoring Host is available.

Components: [origin, cache]

Type: int

Default: 9931

DetailedMonitoringHost
Xrootd

A URL pointing toward the XRootD instance's Detailed Monitoring Host.

Components: [origin, cache]

Type: url

Default: none

DetailedMonitoringPort
Xrootd

The port at which the XRootD instance's Detailed Monitoring Host is available.

Components: [origin, cache]

Type: int

Default: 9930

LocalMonitoringHost
Xrootd

A URL pointing toward the XRootD instance's Local Monitoring Host.

Components: [origin, cache]

Type: url

Default: none

Sitename
Xrootd

The sitename, as configured for XRootD.

Components: [origin]

Type: string

Default: none

MaxStartupWait
Xrootd

The maximum amount of time pelican will wait for the xrootd daemons to successfully start

Components: [origin, cache]

Type: duration

Default: 10s

Monitoring


DataLocation
Monitoring

A filepath where Prometheus should host its monitoring data.

Components: [origin]

Type: string

Default: $ConfigBase/monitoring/data

PortLower
Monitoring

The lower end of a range of monitoring ports for Prometheus configuration.

Components: [origin]

Type: int

Default: 9930

PortHigher
Monitoring

The lower end of a range of monitoring ports for Prometheus configuration.

Components: [origin]

Type: int

Default: 9999

AggregatePrefixes
Monitoring

A list of path-like prefixes, potentially containing a glob (wildcard character), indicating how the Prometheus-based monitoring should aggregate records when reporting. For example, if /foo/* is on the aggregate path list, then the monitoring data for a download of objects /foo/bar and /foo/baz will be aggregated into a single series, /foo.

Components: [origin]

Type: stringSlice

Default: /*

TokenExpiresIn
Monitoring

The duration of which the tokens for various Prometheus endpoints expire.

This includes tokens for director's Prometheus origin discovery endpoint, director's origin scraper, and server's self-scraper.

Components: [origin, director, registry]

Type: duration

Default: 1h

TokenRefreshInterval
Monitoring

The interval of which the token issuer for various Prometheus endpoints refreshes the token for monitoring.

The tokens that are affected by this config are the same as the one in Monitoring.TokenExpiresIn. This value must be less than Monitoring.TokenExpiresIn.

Components: [origin, director, registry]

Type: duration

Default: 5m

MetricAuthorization
Monitoring

If authorization (Bearer token) is required for accessing /metrics endpoint.

Components: [origin, cache, director, registry]

Type: bool

Default: true

PromQLAuthorization
Monitoring

If authorization (Bearer token or cookie) is required for accessing /prometheus/query endpoint.

Components: [origin, cache, director, registry]

Type: bool

Default: true

DataRetention
Monitoring

The duration of which Prometheus should retain the monitoring data.

Components: [origin, cache, director, registry]

Type: duration

Default: 360h

Shoveler


Enable
Shoveler

Enable XRootD monitoring shoveler: https://github.com/opensciencegrid/xrootd-monitoring-shoveler.

Components: [origin, cache]

Type: bool

Default: false

MessageQueueProtocol
Shoveler

Select which protocol to use in order to connect to the MQ. Options are amqp, stomp.

For amqp, the following configurations are required:

  • URL: amqps://username:password@example.com/vhost
  • Topic: mytopic
  • AMQPExchange: shoveled-xrd
  • AMQPTokenLocation: /etc/pelican/xrootd-monitoring-shoveler-token

For stomp, the following configurations are required:

  • URL: messagebroker.org:port
  • Topic: mytopic
  • StompUsername: username
  • StompPassword: password
  • StompCert: path/to/cert/file
  • StompCertKey: path/to/certkey/file

Components: [origin, cache]

Type: string

Default: amqp

URL
Shoveler

For amqp and stomp.

The URL to connect to the shoveler.

Components: [origin, cache]

Type: url

Default: none

Topic
Shoveler

For amqp and stomp.

The topic of the messages. For stomp, it defaults to xrootd.shoveler.

Components: [origin, cache]

Type: string

Default: none

AMQPExchange
Shoveler

For amqp only.

The exchange to shovel messages.

Components: [origin, cache]

Type: string

Default: shoveled-xrd

AMQPTokenLocation
Shoveler

For amqp only.

A filepath to the location of the JWT used for authenticating amqp connection.

Components: [origin, cache]

Type: filename

Default: $ConfigBase/shoveler-token

StompUsername
Shoveler

For stomp only.

Username for authentication.

Components: [origin, cache]

Type: string

Default: none

StompPassword
Shoveler

For stomp only.

Password for authentication.

Components: [origin, cache]

Type: string

Default: none

StompCert
Shoveler

For stomp only.

A filepath to the location of the TLS certifcate.

Components: [origin, cache]

Type: filename

Default: none

StompCertKey
Shoveler

For stomp only.

A filepath to the location of the private key associated with the certificate.

Components: [origin, cache]

Type: filename

Default: none

PortLower
Shoveler

The lower end of a range of Shoveler ports for Shoveler to set up UDP server.

Components: [origin]

Type: int

Default: 9930

PortHigher
Shoveler

The lower end of a range of Shoveler ports for Shoveler to set up UDP server.

Components: [origin]

Type: int

Default: 9999

OutputDestinations
Shoveler

A list of IP:Port destinations to forward XRootD monitoring packet to.

Components: [origin, cache]

Type: stringSlice

Default:

VerifyHeader
Shoveler

Whether to verify the header of the packet matches XRootD's monitoring packet format.

Components: [origin, cache]

Type: bool

Default: false

QueueDirectory
Shoveler

Directory to store overflow of queue onto disk. The queue keeps 100 messages in memory. If the shoveler is disconnected from the message bus, it will store messages over the 100 in memory onto disk into this directory. Once the connection has been re-established the queue will be emptied. The queue on disk is persistent between restarts, so a persistent directory should be used.

Components: [origin, cache]

Type: filename

Default: $ConfigBase/shoveler/queue

IPMapping
Shoveler

IP Mapping for remote IP addresses in forwarding to the destinations. You may either pass one IP address to map all messages to the configured origin, or a list of key-value pairs for one-to-one mapping.

One-to-all mapping:

IPMapping: - All: "172.0.0.4"

If a packet comes in with the private ip address of 192.168.0.4, the packet origin will be changed to 172.0.0.4 The port is always preserved.

One-to-one mapping:

IPMapping: - Source: "192.168.0.5" Dest: "172.0.0.5" - Source: "192.168.0.6" Dest: "129.93.10.7"

Components: [origin, cache]

Type: object

Default: none

Plugin


Token
Plugin

The specified token for pelican plugin staging.

Components: [plugin]

Type: string

Default: none

StagePlugin


Hook
StagePlugin

Flag to specify HTCondor hook behavior.

Components: [plugin]

Type: bool

Default: false

MountPrefix
StagePlugin

Prefix corresponding to the local mount point of the origin.

Components: [plugin]

Type: string

Default: none

OriginPrefix
StagePlugin

Prefix corresponding to the local origin.

Components: [plugin]

Type: string

Default: none

ShadowOriginPrefix
StagePlugin

Prefix corresponding to the shadow origin.

Components: [plugin]

Type: string

Default: none

Lotman


DbLocation
Lotman

The prefix indicating where LotMan should store its lot database. For the provided path, the database will be stored at <path>/.lot/lotman_cpp.sqlite.

Components: [cache]

Type: filename

Default: $ConfigBase

LibLocation
Lotman

The location of the system's installed LotMan library (libLotMan.so). When unset, the system will attempt to find Lotman at these fallback paths:

  • /usr/lib64/libLotMan.so
  • /usr/local/lib64/libLotMan.so
  • /opt/local/lib64/libLotMan.so

Components: [cache]

Type: filename

Default: none

EnableAPI
Lotman

Whether Lotman should enable its CRUD web endpoints. If true, administrators with an appropriately-signed token can interface with Lotman via HTTP. Otherwise, lots are only configurable via the Pelican configuration file at the cache.

Components: [cache]

Type: bool

Default: false

Lots
Lotman

Declarative configuration for LotMan. This is a list of objects, each of which describes a "lot". Every lot can be defined with the following:

  • LotName: REQUIRED. The name of the lot. This is used to identify the lot in the LotMan database.
  • Owner: REQUIRED. A string identifying the owner of the lot's data (as opposed to someone who can modify the lot itself). The Owner field should generally be set to the issue for the lot's namespace path. For example, if the lot tracks namespace /foo/bar, the owner might be set to https://registry.com/api/v1.0/registry/foo/bar.
  • Paths: OPTIONAL. A list of path objects, each of which describes a path that should be managed by the lot.
    • Path: REQUIRED. The path to be managed by the lot.
    • Recursive: REQUIRED. A boolean indicating whether the path should be managed recursively. If true, the lot will manage all files and directories under the specified path.
  • ManagementPolicyAttrs: REQUIRED. The lot's management policy attributes object. This contains information about resources the lot should be allocated, and how it should be managed.
    • DedicatedGB: REQUIRED. The amount of storage, in GB, that should be dedicated to the lot. This means the lot can assume it always has access to this quantity.
    • OpportunisticGB: REQUIRED. The amount of opportunistic storage, in GB, the lot should have access to, when storage is available.
    • MaxNumObjects: REQUIRED. The maximum number of objects a lot is allowed to store.
    • CreationTime: REQUIRED. A unix timestamp indicating when the lot should begin being considered valid. Times in the future indicate the lot should not be considered valid until that time.
    • ExpirationTime: REQUIRED. A unix timestamp indicating when the lot expires. Lots may continue to function after expiration, but lot data owners should recognize the storage is at-will and may be pre-empted at any time.
    • DeletionTime: REQUIRED. A unix timestamp indicating when the lot and its associated data should be deleted.

Note that example configurations can be found in lotman/resources/lots-config.yaml For more information about LotMan configuration, see: https://github.com/pelicanplatform/lotman

Components: [cache]

Type: object

Default: none