Error - Unexpected meta file version 2 on trying to read database

[pradeepdogga]

time="2018-05-31T23:09:03Z" level=error msg="read meta information /prometheus/01CEW002V9A5GWS7EAX4HD8DZE: unexpected meta file version 2"

Does this mean the exporter is not for version 2 of prometheus?

[JonRiv]

It actually is only compatible with version 2. Maybe have a retry ? it should work on a live db.
Let us know and we'll try to reproduce.

[pradeepdogga]

Thanks for the reply ! I am getting the same error again. Here are the steps :

1. Installed from https://github.com/vegasbrianc/prometheus , to get the metrics from cadvisor to be collected to prometheus. Started the docker swarm (with only one machine).
2. Verified that I am able to get stats from cadvisor.
3. Verified that prometheus (v2.1.0) is also receiving data from cadvisor.
4. Built the container from the Dockerfile of prometheus-export.
5. Ran the container and properly specified the path for prometheus database. It shows the error that meta file version is 2. I indeed checked whether the meta file had version 2 and it has.

Please let me know if I am doing something wrong.

[mmariani]

I think it's a bug with Prometheus 2.1, namely this: https://github.com/prometheus/tsdb/blob/master/repair.go#L18

In any case, I have the same message with v2.1 but not with v2.3.
Can you confirm?

$ sudo ./prometheus-export -path /var/lib/docker/volumes/prom_prometheus_data/_data -name go_goroutines | head
INFO[0000] Including labels                              labels="[*]"
INFO[0000] Excluding labels                              labels="[]"
{"__name__":"go_goroutines","__time__":"1528378649535","__value__":"7.000000","instance":"10.0.1.10:9100","job":"node-exporter"}
{"__name__":"go_goroutines","__time__":"1528378654536","__value__":"7.000000","instance":"10.0.1.10:9100","job":"node-exporter"}
{"__name__":"go_goroutines","__time__":"1528378659536","__value__":"7.000000","instance":"10.0.1.10:9100","job":"node-exporter"}
{"__name__":"go_goroutines","__time__":"1528378664536","__value__":"6.000000","instance":"10.0.1.10:9100","job":"node-exporter"}
{"__name__":"go_goroutines","__time__":"1528378669536","__value__":"7.000000","instance":"10.0.1.10:9100","job":"node-exporter"}
{"__name__":"go_goroutines","__time__":"1528378674536","__value__":"7.000000","instance":"10.0.1.10:9100","job":"node-exporter"}
{"__name__":"go_goroutines","__time__":"1528378679536","__value__":"7.000000","instance":"10.0.1.10:9100","job":"node-exporter"}
{"__name__":"go_goroutines","__time__":"1528378684536","__value__":"6.000000","instance":"10.0.1.10:9100","job":"node-exporter"}
{"__name__":"go_goroutines","__time__":"1528378689536","__value__":"7.000000","instance":"10.0.1.10:9100","job":"node-exporter"}
{"__name__":"go_goroutines","__time__":"1528378694536","__value__":"6.000000","instance":"10.0.1.10:9100","job":"node-exporter"}

Thanks,
Marco