A while back you may remember a request for read access to a few tables or a
snapshot of them from which one could work on extracting metrics of interest.
That request never went anywhere so when revisiting this subject recently I
went for a completely external approach using the API and some processing
code. Since InfluxDB and Grafana were mentioned as the tools which the OBS
team would use if they were to provide metrics I went ahead and used those.
The code is built atop the osc-plugin-factory/osclib  and primarily uses /
search/request API call to retrieve requests for given projects of interest.
The code then walks the combined state changes of all the requests to re-
create the staging state at any point in time. A variety of metrics are
generated along the way and some quite interesting graphs can be produced in
Grafana. Additionally, I plan to create an event log  for use by the
release team both for SLE and openSUSE to provide a clear overview of what is
being done by the various staging masters.
Long story short, I plan to cleanup and push the code which can be used to
update the metrics on the regular basis. The original responses indicated that
if something is setup there may be interest in moving forward with the
metrics. Is there interest in hosting an InfluxDB and Grafana instance that
could also be expanded with metrics added directly by OBS? If there is no
interest I will look into setting up a proper host for the release teams.
The URL to my local instances has been shared a few times, but I would rather
not include it in a public mailing list since it is not setup for any sort of
load or real public access. If individuals are interested feel free to reply
or contact me in IRC.
As a teaser, here are some of the work-in-progress graphs which only include
accepted submit requests. The negative dip on the first Factory graph is
documented in the OBS bug regarding dates . In fact the graph would be much
worse without some major workarounds to detect and avoid such corrupted data.
For those interested my VM request was fulfilled and I setup the machine .
Still some issues to be worked out and improvements that can be made to graph
titles and descriptions, but nice to have a solid basis publicly available.
Besides RAM limitations during Factory ingest the primary item to resolve is a
clean update path rather than re-requesting all data. See #4108: [api] allow
ordering to be controlled on /search/requests .