At work I need to handle telemetry for a self-hosted Chroma. Configuration can be using the environment variables or configuration file. As far as I know, OpenTelemetry is the only choice and you can't even configure Chroma to some kind of log verbosity for it to print some log lines while running.

There is a page about observability in Chroma docs. For the environment variables section it gives Honeycomb as example. I have a little of experience of using Honeycomb so I thought maybe this would the easiest way to try it out.

On Honeycomb dashboard there is a dedicated page for the instructions on how to send data to a Honeycomb environment. It's basically setting up 3 environment variables.

export OTEL_EXPORTER_OTLP_ENDPOINT="https://api.honeycomb.io"
export OTEL_EXPORTER_OTLP_HEADERS="x-honeycomb-team=your-api-key"
export OTEL_SERVICE_NAME="your-service-name"

So I installed Chroma CLI. Create a config file:

open_telemetry:
  service_name: "chroma"
  endpoint: "https://api.honeycomb.io"
persist_path: "/data"

Save it as config.yml. I set the environment variable OTEL_EXPORTER_OTLP_HEADERS="x-honeycomb-team=your-api-key". Then I run chroma locally.

$ chroma run ./config.yml

I figure there will only be a trace or span if a client is connected to Chroma. So I created a new Python environment, install Chroma package, run Python prompt.

mkdir try-chroma; cd try-chroma
python3 -m venv
source venv/bin/activate
pip install chromadb

In Python prompt I connect to the Chroma.

import chromadb

chroma_client = chromadb.HttpClient(host='localhost', port=8000)

There is some error in Chroma logs saying that it failed to send data. Then I try adding port to open_telemetry.service_name`. Also doesn't work.

In [Honeycomb docs](https://docs.honeycomb.io/send-data/opentelemetry, grpc is mentioned. So I tried api.honeycomb.io without the https. Doesn't work.

Maybe the headers. I set it to {"x-honeycomb-team":"your-api-key"}. Still doesn't work. Tried the combination of https-or-grpc and header in json format and not. None of them works.

Then I found a page in Chroma docs about using Docker to deploy Chroma. In that page there is an example on how to use OpenTelemetry (Otel) Collector and Zipkin with Chroma. Honeycomb docs also have a page for using Otel Collector. So maybe it will help.