GridOS Connect Agent Setup

The GridOS Connect Agent is only supported on version 1.16.0 or later. Please ensure that your GridOS Connect deployment is upgraded to a supported version.

Connect Agent is a software version of the flow-server installed on-premises behind your own firewall. Flows can be deployed to Connect Agent, but there are no Heartbeat features like monitoring available directly on Connect Agent.

Connect Agent connects securely to the centralized flow-server, referred to as the GridOS Connect main cluster. The GridOS Connect Console serves as a unified interface for flows, deployed on the GridOS Connect cluster and all your Connect Agent deployments.

Integration-level management tasks are only available to users with the correct permissions. See the Users documentation for more details.

System Requirements

Hardware

Connect Agent can only be installed on a 64-bit version of Linux. This can be on a dedicated server or virtual machine but must meet the following criteria:

Hardware	Recommended minimum requirements
Processor	Dual 64-bit processors. More processors allow for increased, simultaneous flow executions.
Memory	4 GB of RAM. More RAM allows for increased, simultaneous flow executions and might be required to process large data payloads.
Hard disk	10GB. SSD based storage is recommended for the best performance.

Hardware

Recommended minimum requirements

Processor

Dual 64-bit processors. More processors allow for increased, simultaneous flow executions.

Memory

4 GB of RAM. More RAM allows for increased, simultaneous flow executions and might be required to process large data payloads.

Hard disk

10GB. SSD based storage is recommended for the best performance.

Software

You will need to install Docker (with Docker Compose), version 20.10.0 or higher.

Remove unused container images after an upgrade to free up storage:
```
docker image prune
```
Also be sure to remove unwanted log archive files. These are stored as .gz files in the logging directory: ${AGENT_INSTALL_DIR}/data/connect-agent/logs.

Create an ID

To get started with Connect Agent, first create an agent ID in Heartbeat:

Select Agents from the left-hand navigation menu:
Select the Create button in the top right corner and enter a name, which generates the agent ID.

This ID is required for your Connect Agent configuration and client mTLS certificate later in the Agent installation process.

Create Oauth Client Credentials Grant Zitadel Service User

Ensure you have created a Zitadel Service User, ensuring the following:

One of the user groups in the data.groups array for the Zitadel Service User ConfigMap MUST be associated with the Agent role.
You have the secrets readily available which are described in the bottom section of the Zitadel service user section:
- Client ID
- Client Secret
- Zitadel project ID

Connect Agent Certificate Management (PKI)

The Connect Agent uses mutual TLS (mTLS) as a secure connection between Agent and the GridOS Connect main cluster. For this to happen we need the following four items:

The server’s public certificate authority (CA) root certificate, referred to as the server-ca-cert.
The public client certificate to be used by the Connect Agent, referred to as the client-cert.
The client private key used by the Connect Agent, corresponding to the public client certificate, referred to as the client-private-key.
The public certificate authority for the issuer of the client-cert, referred to as the client-ca-cert.

In summary, in context of the mTLS configuration, the server is the Connect Management API server hosted on the GridOS Connect main cluster and the client is the Connect Agent.

Public Server CA Root Certificate

The Connect Management API server will be using a root certificate, which can be copied from the Agent view on the GridOS Connect Console:

GridOS Connect Console Agent configuration

Write this certificate data to a file named ca.crt, which we will soon need for the Agent configuration.

Client Certificate

The client-cert must be signed by the customer using the client-ca-cert. Currently we only support private CA’s and not public CA’s like Let’s Encrypt, GoDaddy, etc. There are many ways to create a client certificate key pair (client-cert & client-private-key). Regardless of the certificate creation method, the client-cert and client-private-key MUST fulfill the following requirements:

The client-cert subject field contains the organization (O field) with the exact same owner/org ID used for your Connect deployment. This same value is used for the required deployer parameters.
The client-cert subject field contains the organizational unit (OU field) with the exact string literal: Connect Deployment.
The client-cert subject field contains the common name (CN field) with the exact string matching the agent ID you created earlier.
The signed client-cert and client-private-key must be stored in separate PEM files to be used later on.
You must upload the client-ca-cert to the GridOS Connect Console. The client-ca-cert is needed for the GridOS Connect main cluster server to verify the client-cert when the Agent is communicating with the Connect main cluster. Use these steps to upload the client-ca-cert:
1. Visit your GridOS Connect Console.
2. Select Certificates from the left-hand naviagation menu: .
3. Click on the CREATE CERTIFICATE button in the top right corner.
4. Click UPLOAD FILE and select your client-ca-cert PEM file.
5. Click CREATE.

The following concrete examples are intended merely as suggestions and should not be treated as a guide! For instance, we expect the customer will not be creating their own client-ca-cert. Please refer to your organization’s own practices for PKI.

A client-ca-cert is required. We expect your company to have this in place, but if you must create one yourself you can do the following:

openssl genrsa -out ca.key 2048 (1)
openssl req -new -sha256 -key ca.key -out ca.csr -subj "/CN=ROOTCA" (2)
cat << EOF > ca-ext.cnf (3)
[ca_ext]
basicConstraints = critical,CA:true
keyUsage = critical, keyCertSign, cRLSign
subjectKeyIdentifier = hash
EOF
openssl x509 -req -days 3650 \
  -sha256 \
  -in ca.csr \
  -signkey ca.key \
  -out ca.crt \ (4)
  -extfile ca-ext.cnf \
  -extensions ca_ext

1	This creates the private key corresponding to the client-ca-cert.
2	This step is the Certificate Signing Request (CSR). The CSR file can be deleted after the client-ca-cert is created.
3	This `ca-ext.cnf` file can be deleted after the client-ca-cert is created.
4	The `ca.crt` file is the client-ca-cert PEM file.

Generate a Client certificate and private key, signed using the CA private key, E.g:

openssl genrsa -out client.key 2048 (1)
openssl req -new -key client.key -out client.csr -subj "/O=<YOUR-GRIDOS-CONNECT-ORG>/OU=Connect Deployment/CN=<YOUR-CONNECT-AGENT-ID>" (2)
cat << EOF > client-ext.cnf (3)
[req]
distinguished_name = req_distinguished_name
[req_distinguished_name]
[client_ext]
keyUsage = critical, digitalSignature
extendedKeyUsage = clientAuth
EOF
openssl x509 -req \
  -in client.csr \
  -CA ca.crt \
  -CAkey ca.key \
  -CAcreateserial \
  -out client.crt \ (4)
  -days 365 \
  -sha256 \
  -extfile client-ext.cnf \
  -extensions client_ext

1 This creates the private key corresponding to the client-cert.

This step is the Certificate Signing Request (CSR). The CSR file can be deleted after the client-cert is created.

Be sure to replace the placeholders <YOUR-GRIDOS-CONNECT-ORG> and <YOUR-CONNECT-AGENT-ID> with the proper values! The GridOS connect organization is the same as the owner-id used for the SDK deployer and the Agent ID was created earler

3 This client-ext.cnf file can be deleted after the client-cert is created.

4 The client.crt file is the client-cert PEM file.

Verify the client-cert.

openssl verify -CAfile ca.crt client.crt

You should now have two files required for the Agent configuration:

client.crt - the client-cert.
client.key- the client-private-key.

Together with these two files you will also have the server-ca-cert you created in the <<public-server-ca-root-certificate,above section>: ca.crt.

Concluding PKI

In total, three PKI files are needed for setting up the Connect Agent:

client.crt - the client-cert
client.key- the client-private-key
ca.crt - server-ca-cert

The private keys, i.e the files in these examples with the .key suffix, are sensitive information! The private key for the client certificate must only be available to the Connect Agent administrator!

Set Up the Connect Agent

GE Vernova Artifactory Access

Contact your GE Vernova Support representative for more information on how to get access to GE Vernova Artifactory.

Generate an Artifactory token:

Go to: https://dig-grid-artifactory.apps.ge.com/ui/user_profile
Once logged in, click on Generate an Identity Token. The description should help you remember its purpose, e.g: GridOS Connect Token.
Click Next. The generated token is used as your password credential, with your SSO as the username.

docker login dig-grid-artifactory.apps.ge.com

Deployment Artifacts for Air-Gapped Machines

If you are deploying to a machine with network access to the Artifactory, you can skip this section.

Visit the connect-releases repository.
Download the connect-agent-cache-container-{version}.tgz file to your deployment machine.

Load the cache container contents to your local environment:

# Load the Connect Agent cache image
docker load < connect-agent-cache-container-{version}.tgz
# Fetch the Connect agent docker images from the cache
docker run --rm -it \
  -v ${PWD}:/app/artifacts \
  --entrypoint bash \
  grid-oslo-docker/connect-agent-cache:{version} -c 'cp data/connect-agent-images.tar.gz artifacts'
# Extract connect-agent-images.tar
tar xvfz connect-agent-images.tar.gz
# Load the 4 Connect Agent images (vector, postgres, connect-agent and connect-agent-starter)
docker load < connect-agent-images.tar
# you can remove the intermediary container images and tarballs to free up storage
docker rmi grid-oslo-docker/connect-agent-cache:{version}
rm connect-agent-cache-container-*.tgz
rm connect-agent-images.tar*

Target Machine Deployment Preparations

On you deployment target Linux machine, create an installation directory. E.g:

export AGENT_INSTALL_DIR=~/connect-agent-installation
mkdir -p ${AGENT_INSTALL_DIR}/cert

Visit the connect-releases repository.
Download the connect-agent-{version}.zip file to your installation directory.

Extract the zip file to your installation directory:

cd ${AGENT_INSTALL_DIR}
unzip connect-agent-{version}.zip

Copy the server-ca-cert PEM file to ${AGENT_INSTALL_DIR}/cert/ca.crt.
Copy the client-cert PEM file to ${AGENT_INSTALL_DIR}/cert/client.crt.
Copy the client-private-key PEM file to ${AGENT_INSTALL_DIR}/cert/client.key.

Rename (or copy) the connect-agent-template.env file to its new name connect-agent.env.

Each new Connect Agent release will come with a (possible modified) version of this file. Hence, renaming it will ensure that the changes applied to this file will not be overwritten by a new release.

Open the new connect-agent.env file and fill in the following properties:

Property Description

Property	Description
`CONNECT_AGENT_ID`	The unique ID for this Connect Agent. This ID is provided when adding a new Connect Agent in the Connect Console.
`CONNECT_OWNER_ID`	The `ownerId` for the organisation. As used in the deployment directory as well as the deployement config.
`CONNECT_DOMAIN`	The FQDN for the main GridOS Connect cluster. See `YOUR_CONNECT_DOMAIN` in our installation documentation. This information can be found in the GridOS Connect Console, under the Agent settings. An image of this is show in the Public Server CA Root Certificate section above.
`DEPLOYMENT_ENVIRONMENT`	The environment name of the main GridOS Connect cluster. As used in the deployment directory as well as the deployement config.
`DECRYPTION_KEY`	Decryption key for credentials stored in `connect-agent-override.properties`. Optional. Only needed if the properties if the the properties contain sensitive information. See how encrypted values are generated for more information.
`POSTGRES_PASSWORD`	Select a cryptographically secure password to be used for the Connect Agent PostgreSQL server running on the Agent environment. Do not change after first installation.
`DEPLOYMENT_PUBLISHER_CLIENT_ID`	The client ID of the service user used for publishing to the upstream server API. Same as used for SDK deployer. This pertains to the section above.
`DEPLOYMENT_PUBLISHER_CLIENT_SECRET`	The client secret of the service user used for publishing to the upstream server API. Same as used for SDK deployer. This pertains to the section above.
`DEPLOYMENT_PUBLISHER_SCOPE`	The scope used for publishing to the upstream server API. Same as used for SDK deployer. This pertains to the section above.
`DEPLOYMENT_PUBLISHER_HTTP_PROXY_URI`	URI for HTTP proxy. (Optional)
`DEPLOYMENT_PUBLISHER_HTTP_PROXY_USERNAME`	The username which will be used for HTTP proxy authentication. (Optional)
`DEPLOYMENT_PUBLISHER_HTTP_PROXY_PASSWORD`	The password which will be used for HTTP proxy authentication. (Optional)
`POLL_FREQUENCY_SECONDS`	Number of seconds between each time the Connect Agent polls for new deployments (default is 120 seconds).

CONNECT_AGENT_ID

The unique ID for this Connect Agent. This ID is provided when adding a new Connect Agent in the Connect Console.

CONNECT_OWNER_ID

The ownerId for the organisation. As used in the deployment directory as well as the deployement config.

CONNECT_DOMAIN

The FQDN for the main GridOS Connect cluster. See YOUR_CONNECT_DOMAIN in our installation documentation. This information can be found in the GridOS Connect Console, under the Agent settings. An image of this is show in the Public Server CA Root Certificate section above.

DEPLOYMENT_ENVIRONMENT

The environment name of the main GridOS Connect cluster. As used in the deployment directory as well as the deployement config.

DECRYPTION_KEY

Decryption key for credentials stored in connect-agent-override.properties. Optional. Only needed if the properties if the the properties contain sensitive information. See how encrypted values are generated for more information.

POSTGRES_PASSWORD

Select a cryptographically secure password to be used for the Connect Agent PostgreSQL server running on the Agent environment. Do not change after first installation.

DEPLOYMENT_PUBLISHER_CLIENT_ID

The client ID of the service user used for publishing to the upstream server API. Same as used for SDK deployer. This pertains to the section above.

DEPLOYMENT_PUBLISHER_CLIENT_SECRET

The client secret of the service user used for publishing to the upstream server API. Same as used for SDK deployer. This pertains to the section above.

DEPLOYMENT_PUBLISHER_SCOPE

The scope used for publishing to the upstream server API. Same as used for SDK deployer. This pertains to the section above.

DEPLOYMENT_PUBLISHER_HTTP_PROXY_URI

URI for HTTP proxy. (Optional)

DEPLOYMENT_PUBLISHER_HTTP_PROXY_USERNAME

The username which will be used for HTTP proxy authentication. (Optional)

DEPLOYMENT_PUBLISHER_HTTP_PROXY_PASSWORD

The password which will be used for HTTP proxy authentication. (Optional)

POLL_FREQUENCY_SECONDS

Number of seconds between each time the Connect Agent polls for new deployments (default is 120 seconds).

Start the Connect Agent

To start the Connect Agent, run the command:

./connect-agent.sh

To verify that the Connect Agent is running, confirm that the connect-agent.sh script completed without errors, and then run:

docker ps

Verify that postgres and connect-agent are healthy and vector is Up.

The connect-agent.sh script passes arguments directly to docker compose. Refer to the examples provided below.

The start-connect-agent.sh script is included in this installation for backward compatibility with older versions. When run, it displays a message directing the user to use connect-agent.sh instead. The start-connect-agent.sh script is automatically removed after the first run of connect-agent.sh.

Stop the Connect Agent

You can stop the Connect Agent using the down argument.

./connect-agent.sh down

Connect Agent Logs

You can view the Connect Agent logs using the logs -f arguments.

./connect-agent.sh logs -f

Monitor the Connect Agent

You can monitor the Connect Agent using one of the following commands:

./connect-agent.sh top
./connect-agent.sh exec vector vector top

Manage the Connect Agent

The Connect Agent flows and resources are deployed differently than in a standard Connect installation. See the Connect Agent Deployments guide for more information.

After you deploy a flow to Connect Agent, its flow traces will appear in Heartbeat alongside your other flows. See the Filtering Connect Agent data for more information. Deployed flows will also appear in the Flows page in Heartbeat.

Manage Service Account

The connect-agent-override file defines service accounts for the Agent instance. Note that all secrets must be BCrypt-encoded. NOTE: The name of a service account is user-defined but must be unique for each account.

Basic Authentication

To authenticate using basic authentication, the username is the service account name, and the password is the plain-text password corresponding to the BCrypt-encoded password.

mc.flow-server.security.users.<Service-Account>.credentials=<bcryptEncodedPassword>

Flow Authentication with API Keys

The service account name is not used for authentication with API keys, but the service account still needs access to the flow. The API key is sent in the x-api-key header.

mc.flow-server.security.users.<Service-Account>.apiKey=<bcryptEncodedApiKey>

BCrypt Encoding of Passwords and API Keys

Use only ASCII characters in the passphrase. Other characters will not be handled correctly. To generate a BCrypt-encoded password or API key, use the following command:

docker run --rm httpd:alpine \
  sh -c 'htpasswd -nbBC 10 "" "<passphrase>"' | sed 's/\$2y\$/\$2a\$/' | tr -d ':\n'

Replace 10 with the desired cost factor (we recommend 10 or 12) and <passphrase> with the password or API key you want to encode.

Flow Access Control

Flow access control can be configured in the same file; however, it is recommended to use the flow-access.properties file, which can be deployed with the flows. See the Flow Access Properties documentation for more details.

Example configuration:

mc.flow-server.security.users.<Service-Account>.resourceAccesses=my-flow, my-other-flow
mc.flow-server.security.users.<Service-Account>.resourceAccesses=.*

Upgrade the Connect Agent

The Connect Agent installation may be upgraded by following the below steps:

Visit the connect-releases repository.
Optional: Download the connect-agent-cache-container-{version}.tgz file to your deployment machine.
Optional: Load the cache container contents to your local environment (see here for more details).
Download the connect-agent-{version}.zip file to your installation directory.
Extract the zip file to your installation directory (see here for more details).
Start the Connect Agent (see here for more details).

If you are deploying to a machine with network access to the Artifactory, you can skip the optional cache container steps described above.