Grafana Cloud setup

Getting credentials

Get your Grafana Cloud credentials:

1. Log in to Grafana Cloud
2. Click "Manage Stack" for your stack
3. Click "Details" to view your Instance ID (a number, e.g., 123456)
4. Go to your account settings
5. In the "Security" menu on the left hand side, click "Access Policies"
6. Click "Create access policy"
7. Enter a display name for your policy, e.g., "Zato Observability"
8. Select one of your stacks as the realm to which this policy will apply
9. In the scopes section, select the following: metrics:write, logs:write, traces:write
10. Click "Create"
11. After the policy is created, click "Add token"
12. Enter a name for your token, e.g., "Zato Token"
13. Click "Create"
14. Copy the generated token, you will need it in the next step

Encode your credentials (replace token with your actual token):

echo -n "1107449:token" | base64

Automating credential creation

Create a bootstrap access policy with accesspolicies:read, accesspolicies:write, accesspolicies:delete scopes for your organization realm, then use its token:

POLICY_RESPONSE=$(curl -s -X POST -H "Authorization: Bearer YOUR_BOOTSTRAP_TOKEN" -H "Content-Type: application/json" -d '{"name":"zato-otlp-test","displayName":"Zato OTLP","scopes":["metrics:write","logs:write","traces:write"],"realms":[{"type":"stack","identifier":"1107449"}]}' "https://www.grafana.com/api/v1/accesspolicies?region=prod-us-east-0") && ACCESS_POLICY_ID=$(echo "$POLICY_RESPONSE" | jq -r '.id') && echo "Access Policy ID: $ACCESS_POLICY_ID"

TOKEN_RESPONSE=$(curl -s -X POST -H "Authorization: Bearer YOUR_BOOTSTRAP_TOKEN" -H "Content-Type: application/json" -d '{"accessPolicyId":"'$ACCESS_POLICY_ID'","name":"zato-token-test","displayName":"Zato Token"}' "https://www.grafana.com/api/v1/tokens?region=prod-us-east-0") && OTLP_TOKEN=$(echo "$TOKEN_RESPONSE" | jq -r '.token') && echo "Token: $OTLP_TOKEN"

ENCODED=$(echo -n "1107449:$OTLP_TOKEN" | base64) && echo "Encoded credentials: $ENCODED"

Configuring otelcol-contrib

Create systemd override file:

sudo systemctl edit otelcol-contrib

Add environment variables:

[Service]
Environment="Zato_Grafana_Cloud_OTLP_Endpoint=https://otlp-gateway-prod-us-east-0.grafana.net/otlp"
Environment="Zato_Grafana_Cloud_Auth=your-base64-encoded-credentials"

Replace your-base64-encoded-credentials with the output from the base64 command above.

Save and exit.

Restart the collector:

sudo systemctl daemon-reload
sudo systemctl restart otelcol-contrib

Grafana Cloud Application Observability

After traces are sent successfully:

1. Go to your Grafana Cloud instance
2. Navigate to Application Observability in the menu
3. Services appear after the first traces are received