Fresh single-server Carbonio CE 4.5.4 on Ubuntu 24.04. After
`carbonio-bootstrap` + `service-discover setup-wizard` +
`pending-setups -a` everything in `pending-setups.d/done`, all services
`active (running)`. However in the built-in Carbonio Prometheus
( http://<host>:9090/targets) two jobs are permanently DOWN:
job=docs scrapeUrl= http://<public-ip>:/services/docs/editor/cool/getMetrics
job=storages scrapeUrl= http://<public-ip>:/metrics
Note the empty port — fallback to :80, connection refused.
## Root cause
In /etc/carbonio/carbonio-prometheus/prometheus.yml the docs/storages
jobs build __address__ via this relabel rule:
relabel_configs:
- source_labels:
[__meta_consul_address, __meta_consul_service_metadata_prom_port]
separator: ":"
target_label: __address__
So scrape URL = <consul-node-address> + ":" + <Meta.prom_port>.
In Consul, the service registration of carbonio-docs-editor and
carbonio-storages registered by the corresponding daemons has empty
or missing `Meta.prom_port`:
$ curl -s http://127.0.0.1:8500/v1/agent/service/carbonio-storages | jq '{Port,Address,Meta}'
{"Port": 10000, "Address": "", "Meta": {}}
$ curl -s http://127.0.0.1:8500/v1/agent/service/carbonio-docs-editor | jq '{Port,Address,Meta}'
{"Port": 10000, "Address": "", "Meta": {"service_id": "3120b478-..."}}
For comparison, carbonio-files registers correctly and is UP:
$ curl -s http://127.0.0.1:8500/v1/agent/service/carbonio-files | jq '{Port,Address,Meta}'
{"Port": 10000, "Address": "", "Meta": {"prom_port": "21500"}}
| Service | Meta.prom_port | Prom target |
|--------------------------|----------------|-------------|
| carbonio-files | "21500" | UP |
| carbonio-docs-editor | (missing) | DOWN |
| carbonio-storages | (missing) | DOWN |
So this is a registration-time omission in `carbonio-docs-editor`
25.04.8 and `carbonio-storages-ce` 1.0.16 packages — they don't pass
Meta.prom_port the way `carbonio-files-ce` does.
## Listener side note
Even if Meta.prom_port were set, scrape from the public node address
would still fail because:
$ ss -tlnp | grep coolwsd
LISTEN 127.0.0.1:10000 coolwsd
coolwsd binds 127.0.0.1 only. storages daemon binds 127.78.0.3 (mesh).
Neither is reachable from <public-ip>:<port>.
So a complete fix would need both:
1. Meta.prom_port populated in Consul registration
2. Either listener exposed on the node IP, OR the relabel rule
should use 127.0.0.1 / mesh-IP for these services.
## Reproducer
apt install carbonio-docs-editor carbonio-docs-connector-ce \
carbonio-storages-ce
# carbonio-bootstrap + service-discover setup-wizard + pending-setups -a
curl -s http://127.0.0.1:9090/api/v1/targets | \
jq '.data.activeTargets[] | select(.labels.job=="docs" or .labels.job=="storages")
| {job: .labels.job, url: .scrapeUrl, lastError}'
## Workaround
Exclude `docs` and `storages` from federation match[] / dashboards
that consume Carbonio Prometheus. End-user functionality is unaffected
(coolwsd is reachable internally; admin webmail can open documents
fine), only the metrics are not collectable.
## Questions
1. Is this a known issue, planned fix in a newer version?
2. Should `carbonio-docs-editor` and `carbonio-storages-ce` be
registering Meta.prom_port in their service definitions? If so,
where is the registration emitted from — service-discover wizard,
carbonio-bootstrap, the package postinst, or the daemons themselves?
3. Is there a supported way to make these targets scrape-able
without editing /etc/carbonio/carbonio-prometheus/prometheus.yml
(which would be overwritten on package update)?