Ship container logs with the agent
Before you start
- Docker on the host
- A server secret ab_srv_…
The Abrova Trace agent is one container that reads the logs of every other container on the host and sends them to Abrova Trace in batches. You change nothing in your apps. In the Console you then search all hosts in one feed, filter by level and container, and start, stop or restart containers.
Get the server secret#
The agent runs on your server, so give it the project's server secret (ab_srv_…), not the app key. It is in Abrova Console under Project settings → Keys and is shown once, when you create or rotate it. Keep it in your server's secret store or environment.
Run one agent on the host#
docker run -d \ --name abrova-trace-agent \ --restart=always \ -v /var/run/docker.sock:/var/run/docker.sock:ro \ -v abrova-trace-agent-data:/var/lib/abrova-trace-agent \ -e ABROVA_TRACE_API_KEY=ab_srv_your_server_secret \ ghcr.io/abrova-co/abrova-trace-agent:latest
- The Docker socket lets the agent find containers and read their logs.
- The
abrova-trace-agent-datavolume keeps its disk buffer and read positions when the container is re-created, so no lines are lost or sent twice. - With
--restart=alwaysit comes back after a reboot.
If you run your services with Docker Compose, add the agent to the same file and keep the key in .env, which you don't commit:
# docker-compose.yml
services:
abrova-trace-agent:
image: ghcr.io/abrova-co/abrova-trace-agent:latest
container_name: abrova-trace-agent
restart: always
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- abrova-trace-agent-data:/var/lib/abrova-trace-agent
environment:
- ABROVA_TRACE_API_KEY=${ABROVA_TRACE_API_KEY} # ab_srv_… from .env
volumes:
abrova-trace-agent-data:docker save ghcr.io/abrova-co/abrova-trace-agent:latest -o agent.tar, copy the file, then docker load -i agent.tar on the server.See the logs in the Console#
Within a few seconds the agent registers every running and stopped container. Open Abrova Console → Trace → Logs:
- The Sources panel lists the host and its containers.
- New lines appear in the feed within about 5 seconds: the agent sends batches of up to 100 entries, or every 5 seconds, whichever comes first.
- Each entry has a level, even for plain text logs:
[ERROR],level=warn, JSONlevelfields and words likeexceptionorfailedare read for you, so the level filters work without code changes. - Stack traces and other multi-line output arrive as one entry, and tokens, passwords, API keys, e-mail addresses and phone numbers are masked before anything leaves the host.
To check the agent itself, read its own output on the server:
docker logs abrova-trace-agent
Search, saved searches and the query syntax are on Log search.
Choose what is sent#
Most of the log volume usually comes from a few noisy containers and from health checks. Trim them with environment variables on the agent:
| Variable | What it does |
|---|---|
| ABROVA_TRACE_EXCLUDE | Container names to skip, comma-separated. Default: abrova-trace-agent. A name matches when it is part of the container name. |
| ABROVA_TRACE_INCLUDE | Only these containers, comma-separated (empty = all). |
| ABROVA_TRACE_LABEL_FILTER | Only containers with this label, for example abrova-trace.project=shop. Takes priority over include and exclude. |
| ABROVA_TRACE_HEALTH_SAMPLE | Keep 1 in N successful health-check lines (default 20; 1 = all, 0 = none). |
| ABROVA_TRACE_DROP | Regular expressions separated by ||; matching entries are not sent. |
For example, to skip a health-check sidecar and a debug container, and drop static-file access lines:
docker rm -f abrova-trace-agent docker run -d \ --name abrova-trace-agent \ --restart=always \ -v /var/run/docker.sock:/var/run/docker.sock:ro \ -v abrova-trace-agent-data:/var/lib/abrova-trace-agent \ -e ABROVA_TRACE_API_KEY=ab_srv_your_server_secret \ -e ABROVA_TRACE_EXCLUDE=abrova-trace-agent,healthcheck,debug-shell \ -e 'ABROVA_TRACE_DROP=GET /static/||GET /favicon.ico' \ ghcr.io/abrova-co/abrova-trace-agent:latest
ABROVA_TRACE_EXCLUDE replaces the default, so list abrova-trace-agent again, or the agent ships its own logs.The first GB of logs each month is free; what you send above it is paid from your wallet. See Pricing.
Require the server secret#
Once the agent runs with the server secret, turn on Server routes require the server secret under Project settings → Products → Abrova Trace. Then the app key, which ships inside your apps, can no longer connect as an agent or register containers. Apps keep sending errors and logs with the app key as before. See Server secret & server routes.
Add more hosts#
With several servers, run the agent in agent mode on each extra host. It needs no key; it streams its containers to one collector over a WebSocket:
# On each extra server docker run -d \ --name abrova-trace-agent \ --restart=always \ -p 7008:7008 \ -v /var/run/docker.sock:/var/run/docker.sock:ro \ -e ABROVA_TRACE_HOSTNAME=server-2 \ ghcr.io/abrova-co/abrova-trace-agent:latest agent
On the main server, tell the collector where they are, as a comma-separated name:host:port list:
-e ABROVA_TRACE_REMOTE_AGENTS=server2:YOUR_SERVER2_HOST:7008
Let only the collector's address reach port 7008, for example ufw allow from COLLECTOR_IP to any port 7008. Details: Abrova Trace agent.
Troubleshooting#
The agent container keeps restarting
Read docker logs abrova-trace-agent. The usual causes are a wrong key, a server that can't reach trace.abrova.ir, or the Docker socket not mounted.
No logs appear
Check that your containers are not in ABROVA_TRACE_EXCLUDE (names match on any part) and that they match ABROVA_TRACE_INCLUDE or ABROVA_TRACE_LABEL_FILTER if you set them. Logs are batched: wait up to 5 seconds.
Logs arrive, but containers are missing or can't be controlled
Server routes require the server secret is on and the agent runs with the app key. Log lines still arrive, but registration, status and commands are refused. Redeploy the agent with the server secret (ab_srv_…).
Containers started before the agent are not listed
The agent registers containers when it starts. Restart it: docker restart abrova-trace-agent.
A line I need is missing
Check ABROVA_TRACE_DROP (matching entries are not sent) and ABROVA_TRACE_HEALTH_SAMPLE (only 1 in 20 successful health-check lines is kept by default; set it to 1 to keep all).
Logs were lost while the network was down
When Abrova Trace can't be reached, the agent writes batches to its data folder and sends them in order later, up to 50 MB (ABROVA_TRACE_SPOOL_MB). Mount the abrova-trace-agent-data volume so that buffer survives a re-created container.
- Logs above 1 GB a month cost from your wallet.
- Setting ABROVA_TRACE_EXCLUDE replaces the default: keep abrova-trace-agent in the list.
Where next: Logs overview