REST API Usage¶
Authentication¶
By default, the authentication is configured to the “basic” mode. You need to provide an Authorization header in your HTTP requests with a valid username(the password is not used). The “admin” username is granted all privileges, whereas any other username is recognize as having standard permissions.
You can customize permissions by specifying a different policy_file than the default one.
If you set the api.auth_mode value to keystone, the OpenStack Keystone middleware will be enabled for authentication. It is then needed to authenticate against Keystone and provide a X-Auth-Token header with a valid token for each request sent to Gnocchi’s API.
If you set the api.auth_mode value to remoteuser, Gnocchi will look at the HTTP server REMOTE_USER environment variable to get the username. Then the permissions model is the same as the “basic” mode.
Metrics¶
Gnocchi provides an object type that is called metric. A metric designates any thing that can be measured: the CPU usage of a server, the temperature of a room or the number of bytes sent by a network interface.
A metric only has a few properties: a UUID to identify it, a name, the archive policy that will be used to store and aggregate the measures.
Create¶
To create a metric, the following API request should be used:
POST /v1/metric HTTP/1.1 Content-Type: application/json Content-Length: 35 { "archive_policy_name": "high" }HTTP/1.1 201 Created Location: http://localhost/v1/metric/7d8f95b9-c262-454e-83a2-3035d9c6f6d5 Content-Length: 206 Content-Type: application/json { "archive_policy_name": "high", "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "id": "7d8f95b9-c262-454e-83a2-3035d9c6f6d5", "name": null, "resource_id": null, "unit": null }
Note
Once the metric is created, the archive policy attribute is fixed and unchangeable. The definition of the archive policy can be changed through the archive_policy endpoint though.
Read¶
Once created, you can retrieve the metric information:
GET /v1/metric/7d8f95b9-c262-454e-83a2-3035d9c6f6d5 HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK Content-Length: 532 Content-Type: application/json { "archive_policy": { "aggregation_methods": [ "max", "sum", "count", "std", "min", "mean" ], "back_window": 0, "definition": [ { "granularity": "0:00:01", "points": 3600, "timespan": "1:00:00" }, { "granularity": "0:01:00", "points": 10080, "timespan": "7 days, 0:00:00" }, { "granularity": "1:00:00", "points": 8760, "timespan": "365 days, 0:00:00" } ], "name": "high" }, "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "id": "7d8f95b9-c262-454e-83a2-3035d9c6f6d5", "name": null, "resource": null, "unit": null }
List¶
To retrieve the list of all the metrics created, use the following request:
GET /v1/metric HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK Content-Length: 1071 Content-Type: application/json [ { "archive_policy": { "aggregation_methods": [ "max", "sum", "count", "std", "min", "mean" ], "back_window": 0, "definition": [ { "granularity": "0:00:01", "points": 3600, "timespan": "1:00:00" }, { "granularity": "0:01:00", "points": 10080, "timespan": "7 days, 0:00:00" }, { "granularity": "1:00:00", "points": 8760, "timespan": "365 days, 0:00:00" } ], "name": "high" }, "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "id": "7d8f95b9-c262-454e-83a2-3035d9c6f6d5", "name": null, "resource_id": null, "unit": null }, { "archive_policy": { "aggregation_methods": [ "max", "sum", "count", "std", "min", "mean" ], "back_window": 0, "definition": [ { "granularity": "0:05:00", "points": 12, "timespan": "1:00:00" }, { "granularity": "1:00:00", "points": 24, "timespan": "1 day, 0:00:00" }, { "granularity": "1 day, 0:00:00", "points": 30, "timespan": "30 days, 0:00:00" } ], "name": "low" }, "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "id": "ff041e8c-605e-475f-b08f-4afd53f48a80", "name": null, "resource_id": null, "unit": null } ]
Pagination¶
Considering the large volume of metrics Gnocchi will store, query results are limited to max_limit value set in the configuration file. Returned results are ordered by metrics’ id values. To retrieve the next page of results, the id of a metric should be given as marker for the beginning of the next page of results.
Default ordering and limits as well as page start can be modified using query parameters:
GET /v1/metric?limit=100&sort=name:asc HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK Content-Length: 1071 Content-Type: application/json [ { "archive_policy": { "aggregation_methods": [ "max", "sum", "count", "std", "min", "mean" ], "back_window": 0, "definition": [ { "granularity": "0:00:01", "points": 3600, "timespan": "1:00:00" }, { "granularity": "0:01:00", "points": 10080, "timespan": "7 days, 0:00:00" }, { "granularity": "1:00:00", "points": 8760, "timespan": "365 days, 0:00:00" } ], "name": "high" }, "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "id": "7d8f95b9-c262-454e-83a2-3035d9c6f6d5", "name": null, "resource_id": null, "unit": null }, { "archive_policy": { "aggregation_methods": [ "max", "sum", "count", "std", "min", "mean" ], "back_window": 0, "definition": [ { "granularity": "0:05:00", "points": 12, "timespan": "1:00:00" }, { "granularity": "1:00:00", "points": 24, "timespan": "1 day, 0:00:00" }, { "granularity": "1 day, 0:00:00", "points": 30, "timespan": "30 days, 0:00:00" } ], "name": "low" }, "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "id": "ff041e8c-605e-475f-b08f-4afd53f48a80", "name": null, "resource_id": null, "unit": null } ]
Delete¶
Metrics can be deleted through a request:
DELETE /v1/metric/d0ce7f7f-6ab5-4725-9ffa-c05e589b109a HTTP/1.1 Content-Length: 0HTTP/1.1 204 No Content Content-Length: 0
See also Resources for similar operations specific to metrics associated with a resource.
Measures¶
Push¶
It is possible to send measures to the metric:
POST /v1/metric/7d8f95b9-c262-454e-83a2-3035d9c6f6d5/measures HTTP/1.1 Content-Type: application/json Content-Length: 198 [ { "timestamp": "2014-10-06T14:33:57", "value": 43.1 }, { "timestamp": "2014-10-06T14:34:12", "value": 12 }, { "timestamp": "2014-10-06T14:34:20", "value": 2 } ]HTTP/1.1 202 Accepted Content-Length: 0 Content-Type: application/json
If there are no errors, Gnocchi does not return a response body, only a simple status code. It is possible to provide any number of measures.
Important
While it is possible to send any number of (timestamp, value), they still need to honor constraints defined by the archive policy used by the metric, such as the maximum timespan.
Batch¶
It is also possible to batch measures sending, i.e. send several measures for different metrics in a simple call:
POST /v1/batch/metrics/measures HTTP/1.1 Content-Type: application/json Content-Length: 391 { "7d8f95b9-c262-454e-83a2-3035d9c6f6d5": [ { "timestamp": "2014-10-06T14:34:12", "value": 12 }, { "timestamp": "2014-10-06T14:34:20", "value": 2 } ], "ff041e8c-605e-475f-b08f-4afd53f48a80": [ { "timestamp": "2014-10-06T16:12:12", "value": 3 }, { "timestamp": "2014-10-06T18:14:52", "value": 4 } ] }HTTP/1.1 202 Accepted Content-Length: 0 Content-Type: application/json
Or using named metrics of resources:
POST /v1/batch/resources/metrics/measures HTTP/1.1 Content-Type: application/json Content-Length: 548 { "15e9c872-7ca9-11e4-a2da-2fb4032dfc09": { "cpu.util": { "archive_policy_name": "short", "measures": [ { "timestamp": "2014-10-06T14:34:12", "value": 12 }, { "timestamp": "2014-10-06T14:34:20", "value": 2 } ] } }, "6f24edd9-5a2f-4592-b708-ffbed821c5d2": { "cpu.util": { "archive_policy_name": "short", "measures": [ { "timestamp": "2014-10-06T14:34:12", "value": 6 }, { "timestamp": "2014-10-06T14:34:20", "value": 25 } ] } } }HTTP/1.1 202 Accepted Content-Length: 0 Content-Type: application/json
If some named metrics specified in the batch request do not exist, Gnocchi can try to create them as long as an archive policy rule matches:
POST /v1/batch/resources/metrics/measures?create_metrics=true HTTP/1.1 Content-Type: application/json Content-Length: 200 { "15e9c872-7ca9-11e4-a2da-2fb4032dfc09": { "disk.io.test": [ { "timestamp": "2014-10-06T14:34:12", "value": 71 }, { "timestamp": "2014-10-06T14:34:20", "value": 81 } ] } }HTTP/1.1 202 Accepted Content-Length: 0 Content-Type: application/json
Read¶
Once measures are sent, it is possible to retrieve aggregates using GET on the same endpoint:
GET /v1/metric/7d8f95b9-c262-454e-83a2-3035d9c6f6d5/measures HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK Content-Length: 269 Content-Type: application/json [ [ "2014-10-06T14:00:00+00:00", 3600.0, 19.033333333333335 ], [ "2014-10-06T14:33:00+00:00", 60.0, 43.1 ], [ "2014-10-06T14:34:00+00:00", 60.0, 7.0 ], [ "2014-10-06T14:33:57+00:00", 1.0, 43.1 ], [ "2014-10-06T14:34:12+00:00", 1.0, 12.0 ], [ "2014-10-06T14:34:20+00:00", 1.0, 2.0 ] ]
The list of points returned is composed of tuples with (timestamp, granularity, value) sorted by timestamp. The granularity is the timespan covered by aggregation for this point.
Refresh¶
Depending on the driver, there may be some lag after pushing measures before they are processed and queryable. To ensure your query returns all aggregates that have been pushed and processed, you can force any unprocessed measures to be handled:
GET /v1/metric/7d8f95b9-c262-454e-83a2-3035d9c6f6d5/measures?refresh=true HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK Content-Length: 269 Content-Type: application/json [ [ "2014-10-06T14:00:00+00:00", 3600.0, 19.033333333333335 ], [ "2014-10-06T14:33:00+00:00", 60.0, 43.1 ], [ "2014-10-06T14:34:00+00:00", 60.0, 7.0 ], [ "2014-10-06T14:33:57+00:00", 1.0, 43.1 ], [ "2014-10-06T14:34:12+00:00", 1.0, 12.0 ], [ "2014-10-06T14:34:20+00:00", 1.0, 2.0 ] ]
Note
Depending on the amount of data that is unprocessed, refresh may add some overhead to your query.
Filter¶
Time range¶
It is possible to filter the aggregates over a time range by specifying the start and/or stop parameters to the query with timestamp. The timestamp format can be either a floating number (UNIX epoch) or an ISO8601 formated timestamp:
GET /v1/metric/7d8f95b9-c262-454e-83a2-3035d9c6f6d5/measures?start=2014-10-06T14:34 HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK Content-Length: 184 Content-Type: application/json [ [ "2014-10-06T14:00:00+00:00", 3600.0, 19.033333333333335 ], [ "2014-10-06T14:34:00+00:00", 60.0, 7.0 ], [ "2014-10-06T14:34:12+00:00", 1.0, 12.0 ], [ "2014-10-06T14:34:20+00:00", 1.0, 2.0 ] ]
Aggregation¶
By default, the aggregated values that are returned use the mean aggregation method. It is possible to request for any other method defined by the policy by specifying the aggregation query parameter:
GET /v1/metric/7d8f95b9-c262-454e-83a2-3035d9c6f6d5/measures?aggregation=max HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK Content-Length: 256 Content-Type: application/json [ [ "2014-10-06T14:00:00+00:00", 3600.0, 43.1 ], [ "2014-10-06T14:33:00+00:00", 60.0, 43.1 ], [ "2014-10-06T14:34:00+00:00", 60.0, 12.0 ], [ "2014-10-06T14:33:57+00:00", 1.0, 43.1 ], [ "2014-10-06T14:34:12+00:00", 1.0, 12.0 ], [ "2014-10-06T14:34:20+00:00", 1.0, 2.0 ] ]
Granularity¶
It’s possible to provide the granularity argument to specify the granularity to retrieve, rather than all the granularities available:
GET /v1/metric/7d8f95b9-c262-454e-83a2-3035d9c6f6d5/measures?granularity=1 HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK Content-Length: 125 Content-Type: application/json [ [ "2014-10-06T14:33:57+00:00", 1.0, 43.1 ], [ "2014-10-06T14:34:12+00:00", 1.0, 12.0 ], [ "2014-10-06T14:34:20+00:00", 1.0, 2.0 ] ]
Resample¶
In addition to granularities defined by the archive policy, aggregates can be resampled to a new granularity.
GET /v1/metric/7d8f95b9-c262-454e-83a2-3035d9c6f6d5/measures?granularity=1&resample=5 HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK Content-Length: 125 Content-Type: application/json [ [ "2014-10-06T14:33:55+00:00", 5.0, 43.1 ], [ "2014-10-06T14:34:10+00:00", 5.0, 12.0 ], [ "2014-10-06T14:34:20+00:00", 5.0, 2.0 ] ]
Time-series data can also be grouped by calendar dates beyond a standard day. The resulting groupings are tied to the leading date of the group. For example, grouping on month returns a monthly aggregate linked to the first of the month.
GET /v1/metric/7d8f95b9-c262-454e-83a2-3035d9c6f6d5/measures?granularity=1&resample=W HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK Content-Length: 56 Content-Type: application/json [ [ "2014-10-05T00:00:00+00:00", "W", 19.033333333333335 ] ]
Available calendar groups are:
Y – by year
H – by half
Q – by quarter
M – by month
W – by week, starting on Sunday
Note
If you plan to execute the query often, it is recommended for performance to leverage an archive policy with the needed granularity instead of resampling the time series on each query.
Note
Depending on the aggregation method and frequency of measures, resampled data may lack accuracy as it is working against previously aggregated data.
Note
Gnocchi has an aggregates endpoint which provides resampling as well as additional capabilities.
Archive Policy¶
When sending measures for a metric to Gnocchi, the values are dynamically aggregated. That means that Gnocchi does not store all sent measures, but aggregates them over a certain period of time.
Gnocchi provides several aggregation methods that are builtin. The list of aggregation method available is: mean, sum, last, max, min, std, median, first, count and Npct (with 0 < N < 100). Those can be prefix by rate: to compute the rate of change before doing the aggregation.
An archive policy is defined by a list of items in the definition field. Each item is composed of: the timespan; the granularity, which is the level of precision that must be kept when aggregating data; and the number of points. The archive policy is determined using at least 2 of the points, granularity and timespan fields. For example, an item might be defined as 12 points over 1 hour (one point every 5 minutes), or 1 point every 1 hour over 1 day (24 points).
By default, new measures can only be processed if they have timestamps in the future or part of the last aggregation period. The last aggregation period size is based on the largest granularity defined in the archive policy definition. To allow processing measures that are older than the period, the back_window parameter can be used to set the number of coarsest periods to keep. That way it is possible to process measures that are older than the last timestamp period boundary.
For example, if an archive policy is defined with coarsest aggregation of 1 hour, and the last point processed has a timestamp of 14:34, it’s possible to process measures back to 14:00 with a back_window of 0. If the back_window is set to 2, it will be possible to send measures with timestamp back to 12:00 (14:00 minus 2 times 1 hour).
Create¶
The REST API allows to create archive policies in this way:
POST /v1/archive_policy HTTP/1.1 Content-Type: application/json Content-Length: 255 { "back_window": 0, "definition": [ { "granularity": "1h", "timespan": "7 day" }, { "granularity": "1s", "timespan": "1 hour" }, { "points": 48, "timespan": "1 day" } ], "name": "short" }HTTP/1.1 201 Created Location: http://localhost/v1/archive_policy/short Content-Length: 334 Content-Type: application/json { "aggregation_methods": [ "mean", "max", "sum", "count", "std", "min" ], "back_window": 0, "definition": [ { "granularity": "1:00:00", "points": 168, "timespan": "7 days, 0:00:00" }, { "granularity": "0:00:01", "points": 3600, "timespan": "1:00:00" }, { "granularity": "0:30:00", "points": 48, "timespan": "1 day, 0:00:00" } ], "name": "short" }
By default, the aggregation methods computed and stored are the ones defined with default_aggregation_methods in the configuration file. It is possible to change the aggregation methods used in an archive policy by specifying the list of aggregation method to use in the aggregation_methods attribute of an archive policy.
POST /v1/archive_policy HTTP/1.1 Content-Type: application/json Content-Length: 244 { "aggregation_methods": [ "-max", "-min" ], "back_window": 0, "definition": [ { "granularity": "1s", "timespan": "1 hour" }, { "points": 48, "timespan": "1 day" } ], "name": "short-without-max" }HTTP/1.1 201 Created Location: http://localhost/v1/archive_policy/short-without-max Content-Length: 258 Content-Type: application/json { "aggregation_methods": [ "sum", "count", "std", "mean" ], "back_window": 0, "definition": [ { "granularity": "0:00:01", "points": 3600, "timespan": "1:00:00" }, { "granularity": "0:30:00", "points": 48, "timespan": "1 day, 0:00:00" } ], "name": "short-without-max" }
The list of aggregation methods can either be:
a list of aggregation methods to use, e.g. [“mean”, “max”]
a list of methods to remove (prefixed by -) and/or to add (prefixed by +) to the default list (e.g. [“+mean”, “-last”])
If * is included in the list, it’s substituted by the list of all supported aggregation methods.
Read¶
Once the archive policy is created, the complete set of properties is computed and returned, with the URL of the archive policy. This URL can be used to retrieve the details of the archive policy later:
GET /v1/archive_policy/short HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK Content-Length: 334 Content-Type: application/json { "aggregation_methods": [ "max", "sum", "count", "std", "min", "mean" ], "back_window": 0, "definition": [ { "granularity": "1:00:00", "points": 168, "timespan": "7 days, 0:00:00" }, { "granularity": "0:00:01", "points": 3600, "timespan": "1:00:00" }, { "granularity": "0:30:00", "points": 48, "timespan": "1 day, 0:00:00" } ], "name": "short" }
List¶
It is also possible to list archive policies:
GET /v1/archive_policy HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK Content-Length: 2034 Content-Type: application/json [ { "aggregation_methods": [ "max", "sum", "count", "std", "min", "mean" ], "back_window": 0, "definition": [ { "granularity": "0:00:02", "points": 43200, "timespan": "1 day, 0:00:00" } ], "name": "no_granularity_match" }, { "aggregation_methods": [ "max", "sum", "count", "std", "min", "mean" ], "back_window": 0, "definition": [ { "granularity": "0:05:00", "points": 12, "timespan": "1:00:00" }, { "granularity": "1:00:00", "points": 24, "timespan": "1 day, 0:00:00" }, { "granularity": "1 day, 0:00:00", "points": 30, "timespan": "30 days, 0:00:00" } ], "name": "low" }, { "aggregation_methods": [ "max", "sum", "count", "std", "min", "mean" ], "back_window": 0, "definition": [ { "granularity": "0:01:00", "points": 1440, "timespan": "1 day, 0:00:00" }, { "granularity": "1:00:00", "points": 168, "timespan": "7 days, 0:00:00" }, { "granularity": "1 day, 0:00:00", "points": 365, "timespan": "365 days, 0:00:00" } ], "name": "medium" }, { "aggregation_methods": [ "max", "sum", "count", "std", "min", "mean" ], "back_window": 0, "definition": [ { "granularity": "0:00:01", "points": 3600, "timespan": "1:00:00" }, { "granularity": "0:01:00", "points": 10080, "timespan": "7 days, 0:00:00" }, { "granularity": "1:00:00", "points": 8760, "timespan": "365 days, 0:00:00" } ], "name": "high" }, { "aggregation_methods": [ "max", "sum", "count", "std", "min", "mean" ], "back_window": 0, "definition": [ { "granularity": "1:00:00", "points": 168, "timespan": "7 days, 0:00:00" }, { "granularity": "0:00:01", "points": 3600, "timespan": "1:00:00" }, { "granularity": "0:30:00", "points": 48, "timespan": "1 day, 0:00:00" } ], "name": "short" }, { "aggregation_methods": [ "max", "sum", "count", "std", "min", "mean" ], "back_window": 0, "definition": [ { "granularity": "1:00:00", "points": null, "timespan": null } ], "name": "unrelated" }, { "aggregation_methods": [ "sum", "count", "std", "mean" ], "back_window": 0, "definition": [ { "granularity": "0:00:01", "points": 3600, "timespan": "1:00:00" }, { "granularity": "0:30:00", "points": 48, "timespan": "1 day, 0:00:00" } ], "name": "short-without-max" } ]
Update¶
Existing archive policies can be modified to retain more or less data depending on requirements. If the policy coverage is expanded, aggregates are not retroactively calculated as backfill to accommodate the new timespan:
PATCH /v1/archive_policy/short HTTP/1.1 Content-Type: application/json Content-Length: 216 { "definition": [ { "granularity": "1h", "timespan": "7 day" }, { "granularity": "1s", "timespan": "1 hour" }, { "points": 48, "timespan": "1 day" } ] }HTTP/1.1 200 OK Content-Length: 334 Content-Type: application/json { "aggregation_methods": [ "max", "sum", "count", "std", "min", "mean" ], "back_window": 0, "definition": [ { "granularity": "1:00:00", "points": 168, "timespan": "7 days, 0:00:00" }, { "granularity": "0:00:01", "points": 3600, "timespan": "1:00:00" }, { "granularity": "0:30:00", "points": 48, "timespan": "1 day, 0:00:00" } ], "name": "short" }
The attribute back window of the update request is optional and can be used to change the back_window of an archive policies. If we reduce the back window, data is going to be deleted in the back end when the metrics daemon Janitor runs. On the other hand, if the value is increased, more raw data will be stored to enable reprocessing in case of data backfill.
Note
granularities cannot be changed to a different rate. Also, granularities cannot be added or dropped from a policy.
Delete¶
It is possible to delete an archive policy if it is not used by any metric:
DELETE /v1/archive_policy/some-archive-policy HTTP/1.1 Content-Length: 0HTTP/1.1 204 No Content Content-Length: 0
Note
An archive policy cannot be deleted until all metrics associated with it are removed by a metricd daemon.
Archive Policy Rule¶
Gnocchi provides the ability to define a mapping called archive_policy_rule. An archive policy rule defines a mapping between a metric and an archive policy. This gives users the ability to pre-define rules so an archive policy is assigned to metrics based on a matched pattern.
An archive policy rule has a few properties: a name to identify it, an archive policy name that will be used to store the policy name and metric pattern to match metric names.
An archive policy rule for example could be a mapping to default a medium archive policy for any volume metric with a pattern matching volume.*. When a sample metric is posted with a name of volume.size, that would match the pattern and the rule applies and sets the archive policy to medium. If multiple rules match, the longest matching rule is taken. For example, if two rules exists which match * and disk.*, a disk.io.rate metric would match the disk.* rule rather than * rule.
Create¶
To create a rule, the following API request should be used:
POST /v1/archive_policy_rule HTTP/1.1 Content-Type: application/json Content-Length: 90 { "archive_policy_name": "low", "metric_pattern": "disk.io.*", "name": "test_rule" }HTTP/1.1 201 Created Location: http://localhost/v1/archive_policy_rule/test_rule Content-Length: 82 Content-Type: application/json { "archive_policy_name": "low", "metric_pattern": "disk.io.*", "name": "test_rule" }
The metric_pattern is used to pattern match so as some examples,
* matches anything
disk.* matches disk.io
disk.io.* matches disk.io.rate
Read¶
Once created, you can retrieve the rule information:
GET /v1/archive_policy_rule/test_rule HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK Content-Length: 82 Content-Type: application/json { "archive_policy_name": "low", "metric_pattern": "disk.io.*", "name": "test_rule" }
List¶
It is also possible to list archive policy rules. The result set is ordered by the metric_pattern, in reverse alphabetical order:
GET /v1/archive_policy_rule HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK Content-Length: 84 Content-Type: application/json [ { "archive_policy_name": "low", "metric_pattern": "disk.io.*", "name": "test_rule" } ]
Update¶
It is possible to rename an archive policy rule:
PATCH /v1/archive_policy_rule/test_rule_rename HTTP/1.1 Content-Type: application/json Content-Length: 24 { "name": "new_name" }HTTP/1.1 200 OK Content-Length: 81 Content-Type: application/json { "archive_policy_name": "low", "metric_pattern": "disk.io.*", "name": "new_name" }
Delete¶
It is possible to delete an archive policy rule:
DELETE /v1/archive_policy_rule/test_rule_delete HTTP/1.1 Content-Length: 0HTTP/1.1 204 No Content Content-Length: 0
Resources¶
Gnocchi provides the ability to store and index resources. Each resource has a type. The basic type of resources is generic, but more specialized subtypes also exist, especially to describe OpenStack resources.
Create¶
To create a generic resource:
POST /v1/resource/generic HTTP/1.1 Content-Type: application/json Content-Length: 159 { "id": "75C44741-CC60-4033-804E-2D3098C7D2E9", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" }HTTP/1.1 201 Created Location: http://localhost/v1/resource/generic/75c44741-cc60-4033-804e-2d3098c7d2e9 ETag: "f8a965d5ed2b6e098ad149343930b36ec9a036ee" Last-Modified: Sat, 27 Jul 2024 07:23:00 GMT Content-Length: 473 Content-Type: application/json { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "ended_at": null, "id": "75c44741-cc60-4033-804e-2d3098c7d2e9", "metrics": {}, "original_resource_id": "75C44741-CC60-4033-804E-2D3098C7D2E9", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": null, "revision_start": "2024-07-27T07:23:00.500012+00:00", "started_at": "2024-07-27T07:23:00.500006+00:00", "type": "generic", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" }
The user_id and project_id attributes may be any arbitrary string. The timestamps describing the lifespan of the resource are optional, and started_at is by default set to the current timestamp.
The id attribute may be a UUID or some other arbitrary string. If it is a UUID, Gnocchi will use it verbatim. If it is not a UUID, the original value will be stored in the original_resource_id attribute and Gnocchi will generate a new UUID that is unique for the user. That is, if two users submit create requests with the same non-UUID id attribute, the resulting resources will have different UUID values in their respective id attributes.
You may use either of the id or the original_resource_id attributes to refer to the resource. The value returned by the create operation includes a Location header referencing the id.
Non-generic resources¶
More specialized resources can be created. For example, the instance is used to describe an OpenStack instance as managed by Nova.
POST /v1/resource/instance HTTP/1.1 Content-Type: application/json Content-Length: 392 { "display_name": "myvm", "ended_at": "2014-01-04 10:00:12", "flavor_id": "2", "host": "compute1", "id": "6868DA77-FA82-4E67-ABA9-270C5AE8CBCA", "image_ref": "http://image", "launched_at": "2017-12-10T08:10:42Z", "metrics": {}, "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "started_at": "2014-01-02 23:23:34", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" }HTTP/1.1 201 Created Location: http://localhost/v1/resource/instance/6868da77-fa82-4e67-aba9-270c5ae8cbca ETag: "87a4917f5cd23cb288ca21167dfca89ff3045afb" Last-Modified: Sat, 27 Jul 2024 07:23:00 GMT Content-Length: 647 Content-Type: application/json { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "display_name": "myvm", "ended_at": "2014-01-04T10:00:12+00:00", "flavor_id": "2", "host": "compute1", "id": "6868da77-fa82-4e67-aba9-270c5ae8cbca", "image_ref": "http://image", "launched_at": "2017-12-10T08:10:42+00:00", "metrics": {}, "original_resource_id": "6868DA77-FA82-4E67-ABA9-270C5AE8CBCA", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": null, "revision_start": "2024-07-27T07:23:00.562714+00:00", "server_group": null, "started_at": "2014-01-02T23:23:34+00:00", "type": "instance", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" }
All specialized types have their own optional and mandatory attributes, but they all include attributes from the generic type as well.
With metrics¶
Each resource can be linked to any number of metrics on creation:
POST /v1/resource/instance HTTP/1.1 Content-Type: application/json Content-Length: 414 { "display_name": "myvm2", "flavor_id": "2", "host": "compute1", "id": "6F24EDD9-5A2F-4592-B708-FFBED821C5D2", "image_ref": "http://image", "metrics": { "cpu.util": "7d8f95b9-c262-454e-83a2-3035d9c6f6d5" }, "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "server_group": "my_autoscaling_group", "started_at": "2014-10-06T14:00:02.000000", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" }HTTP/1.1 201 Created Location: http://localhost/v1/resource/instance/6f24edd9-5a2f-4592-b708-ffbed821c5d2 ETag: "94291fb234c1b46cb1e65136afdbab5d2e7ba06d" Last-Modified: Sat, 27 Jul 2024 07:23:01 GMT Content-Length: 670 Content-Type: application/json { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "display_name": "myvm2", "ended_at": null, "flavor_id": "2", "host": "compute1", "id": "6f24edd9-5a2f-4592-b708-ffbed821c5d2", "image_ref": "http://image", "launched_at": null, "metrics": { "cpu.util": "7d8f95b9-c262-454e-83a2-3035d9c6f6d5" }, "original_resource_id": "6F24EDD9-5A2F-4592-B708-FFBED821C5D2", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": null, "revision_start": "2024-07-27T07:23:01.692339+00:00", "server_group": "my_autoscaling_group", "started_at": "2014-10-06T14:00:02+00:00", "type": "instance", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" }
It is also possible to create metrics at the same time you create a resource to save some requests:
POST /v1/resource/instance HTTP/1.1 Content-Type: application/json Content-Length: 408 { "display_name": "myvm3", "flavor_id": "2", "host": "compute2", "id": "15e9c872-7ca9-11e4-a2da-2fb4032dfc09", "image_ref": "http://image", "metrics": { "cpu.util": { "archive_policy_name": "short" } }, "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "server_group": "my_autoscaling_group", "started_at": "2014-10-06T14:00:02.000000", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" }HTTP/1.1 201 Created Location: http://localhost/v1/resource/instance/15e9c872-7ca9-11e4-a2da-2fb4032dfc09 ETag: "6f153f82342f77569e43ddd87a7e74d329991b17" Last-Modified: Sat, 27 Jul 2024 07:23:01 GMT Content-Length: 670 Content-Type: application/json { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "display_name": "myvm3", "ended_at": null, "flavor_id": "2", "host": "compute2", "id": "15e9c872-7ca9-11e4-a2da-2fb4032dfc09", "image_ref": "http://image", "launched_at": null, "metrics": { "cpu.util": "17b898d0-8723-4287-8968-4f6921eeefeb" }, "original_resource_id": "15e9c872-7ca9-11e4-a2da-2fb4032dfc09", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": null, "revision_start": "2024-07-27T07:23:01.719711+00:00", "server_group": "my_autoscaling_group", "started_at": "2014-10-06T14:00:02+00:00", "type": "instance", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" }
Read¶
To retrieve a resource by its URL provided by the Location header at creation time:
GET /v1/resource/generic/75c44741-cc60-4033-804e-2d3098c7d2e9 HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK ETag: "f8a965d5ed2b6e098ad149343930b36ec9a036ee" Last-Modified: Sat, 27 Jul 2024 07:23:00 GMT Content-Length: 473 Content-Type: application/json { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "ended_at": null, "id": "75c44741-cc60-4033-804e-2d3098c7d2e9", "metrics": {}, "original_resource_id": "75C44741-CC60-4033-804E-2D3098C7D2E9", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": null, "revision_start": "2024-07-27T07:23:00.500012+00:00", "started_at": "2024-07-27T07:23:00.500006+00:00", "type": "generic", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" }
List¶
All resources can be listed, either by using the generic type that will list all types of resources, or by filtering on their resource type:
GET /v1/resource/generic HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK Content-Length: 967 Content-Type: application/json [ { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "ended_at": null, "id": "75c44741-cc60-4033-804e-2d3098c7d2e9", "metrics": {}, "original_resource_id": "75C44741-CC60-4033-804E-2D3098C7D2E9", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": null, "revision_start": "2024-07-27T07:23:00.500012+00:00", "started_at": "2024-07-27T07:23:00.500006+00:00", "type": "generic", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" }, { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "ended_at": "2014-01-04T10:00:12+00:00", "id": "6868da77-fa82-4e67-aba9-270c5ae8cbca", "metrics": {}, "original_resource_id": "6868DA77-FA82-4E67-ABA9-270C5AE8CBCA", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": null, "revision_start": "2024-07-27T07:23:00.562714+00:00", "started_at": "2014-01-02T23:23:34+00:00", "type": "instance", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" } ]
Specific resource type¶
No attributes specific to the resource type are retrieved when using the generic endpoint. To retrieve the details, either list using the specific resource type endpoint:
GET /v1/resource/instance HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK Content-Length: 649 Content-Type: application/json [ { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "display_name": "myvm", "ended_at": "2014-01-04T10:00:12+00:00", "flavor_id": "2", "host": "compute1", "id": "6868da77-fa82-4e67-aba9-270c5ae8cbca", "image_ref": "http://image", "launched_at": "2017-12-10T08:10:42+00:00", "metrics": {}, "original_resource_id": "6868DA77-FA82-4E67-ABA9-270C5AE8CBCA", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": null, "revision_start": "2024-07-27T07:23:00.562714+00:00", "server_group": null, "started_at": "2014-01-02T23:23:34+00:00", "type": "instance", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" } ]
With details¶
To retrieve a more detailed view of the resources, use details=true in the query parameter:
GET /v1/resource/generic?details=true HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK Content-Length: 1124 Content-Type: application/json [ { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "ended_at": null, "id": "75c44741-cc60-4033-804e-2d3098c7d2e9", "metrics": {}, "original_resource_id": "75C44741-CC60-4033-804E-2D3098C7D2E9", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": null, "revision_start": "2024-07-27T07:23:00.500012+00:00", "started_at": "2024-07-27T07:23:00.500006+00:00", "type": "generic", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" }, { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "display_name": "myvm", "ended_at": "2014-01-04T10:00:12+00:00", "flavor_id": "2", "host": "compute1", "id": "6868da77-fa82-4e67-aba9-270c5ae8cbca", "image_ref": "http://image", "launched_at": "2017-12-10T08:10:42+00:00", "metrics": {}, "original_resource_id": "6868DA77-FA82-4E67-ABA9-270C5AE8CBCA", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": null, "revision_start": "2024-07-27T07:23:00.562714+00:00", "server_group": null, "started_at": "2014-01-02T23:23:34+00:00", "type": "instance", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" } ]
Limit attributes¶
To limit response attributes, use attrs=id&attrs=started_at&attrs=user_id in the query parameter:
GET /v1/resource/generic?attrs=id&attrs=started_at&attrs=user_id HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK Content-Length: 291 Content-Type: application/json [ { "id": "75c44741-cc60-4033-804e-2d3098c7d2e9", "started_at": "2024-07-27T07:23:00.500006+00:00", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" }, { "id": "6868da77-fa82-4e67-aba9-270c5ae8cbca", "started_at": "2014-01-02T23:23:34+00:00", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" } ]
Pagination¶
Similar to metric list, query results are limited to max_limit value set in the configuration file. Returned results represent a single page of data and are ordered by resouces’ revision_start time and started_at values:
GET /v1/resource/generic?limit=2&sort=id:asc HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK Link: <http://localhost/v1/resource/generic?limit=2&marker=75c44741-cc60-4033-804e-2d3098c7d2e9&sort=id%3Aasc>; rel="next" Content-Length: 967 Content-Type: application/json [ { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "ended_at": "2014-01-04T10:00:12+00:00", "id": "6868da77-fa82-4e67-aba9-270c5ae8cbca", "metrics": {}, "original_resource_id": "6868DA77-FA82-4E67-ABA9-270C5AE8CBCA", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": null, "revision_start": "2024-07-27T07:23:00.562714+00:00", "started_at": "2014-01-02T23:23:34+00:00", "type": "instance", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" }, { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "ended_at": null, "id": "75c44741-cc60-4033-804e-2d3098c7d2e9", "metrics": {}, "original_resource_id": "75C44741-CC60-4033-804E-2D3098C7D2E9", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": null, "revision_start": "2024-07-27T07:23:00.500012+00:00", "started_at": "2024-07-27T07:23:00.500006+00:00", "type": "generic", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" } ]
List resource metrics¶
The metrics associated with a resource can be accessed and manipulated using the usual /v1/metric endpoint or using the named relationship with the resource:
GET /v1/resource/generic/6f24edd9-5a2f-4592-b708-ffbed821c5d2/metric/cpu.util/measures?start=2014-10-06T14:34 HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK Content-Length: 171 Content-Type: application/json [ [ "2014-10-06T14:00:00+00:00", 3600.0, 24.7 ], [ "2014-10-06T14:34:00+00:00", 60.0, 15.5 ], [ "2014-10-06T14:34:12+00:00", 1.0, 6.0 ], [ "2014-10-06T14:34:20+00:00", 1.0, 25.0 ] ]
Update¶
It’s possible to modify a resource by re-uploading it partially with the modified fields:
PATCH /v1/resource/instance/6868da77-fa82-4e67-aba9-270c5ae8cbca HTTP/1.1 Content-Type: application/json Content-Length: 20 { "host": "compute2" }HTTP/1.1 200 OK ETag: "c8d3dab1b198a49ad54d16cad56dac27e74bb9da" Last-Modified: Sat, 27 Jul 2024 07:23:01 GMT Content-Length: 647 Content-Type: application/json { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "display_name": "myvm", "ended_at": "2014-01-04T10:00:12+00:00", "flavor_id": "2", "host": "compute2", "id": "6868da77-fa82-4e67-aba9-270c5ae8cbca", "image_ref": "http://image", "launched_at": "2017-12-10T08:10:42+00:00", "metrics": {}, "original_resource_id": "6868DA77-FA82-4E67-ABA9-270C5AE8CBCA", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": null, "revision_start": "2024-07-27T07:23:01.128092+00:00", "server_group": null, "started_at": "2014-01-02T23:23:34+00:00", "type": "instance", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" }
It is also possible to associate additional metrics with a resource:
POST /v1/resource/generic/6f24edd9-5a2f-4592-b708-ffbed821c5d2/metric HTTP/1.1 Content-Type: application/json Content-Length: 42 { "memory": { "archive_policy_name": "low" } }HTTP/1.1 200 OK Content-Length: 1149 Content-Type: application/json [ { "archive_policy": { "aggregation_methods": [ "max", "sum", "count", "std", "min", "mean" ], "back_window": 0, "definition": [ { "granularity": "0:00:01", "points": 3600, "timespan": "1:00:00" }, { "granularity": "0:01:00", "points": 10080, "timespan": "7 days, 0:00:00" }, { "granularity": "1:00:00", "points": 8760, "timespan": "365 days, 0:00:00" } ], "name": "high" }, "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "id": "7d8f95b9-c262-454e-83a2-3035d9c6f6d5", "name": "cpu.util", "resource_id": "6f24edd9-5a2f-4592-b708-ffbed821c5d2", "unit": null }, { "archive_policy": { "aggregation_methods": [ "max", "sum", "count", "std", "min", "mean" ], "back_window": 0, "definition": [ { "granularity": "0:05:00", "points": 12, "timespan": "1:00:00" }, { "granularity": "1:00:00", "points": 24, "timespan": "1 day, 0:00:00" }, { "granularity": "1 day, 0:00:00", "points": 30, "timespan": "30 days, 0:00:00" } ], "name": "low" }, "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "id": "e0d0d176-4de2-4c16-b892-d2e4cf33f9a6", "name": "memory", "resource_id": "6f24edd9-5a2f-4592-b708-ffbed821c5d2", "unit": null } ]
History¶
And to retrieve a resource’s modification history:
GET /v1/resource/instance/6868da77-fa82-4e67-aba9-270c5ae8cbca/history HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK Content-Length: 1328 Content-Type: application/json [ { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "display_name": "myvm", "ended_at": "2014-01-04T10:00:12+00:00", "flavor_id": "2", "host": "compute1", "id": "6868da77-fa82-4e67-aba9-270c5ae8cbca", "image_ref": "http://image", "launched_at": "2017-12-10T08:10:42+00:00", "metrics": {}, "original_resource_id": "6868DA77-FA82-4E67-ABA9-270C5AE8CBCA", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": "2024-07-27T07:23:01.128092+00:00", "revision_start": "2024-07-27T07:23:00.562714+00:00", "server_group": null, "started_at": "2014-01-02T23:23:34+00:00", "type": "instance", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" }, { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "display_name": "myvm", "ended_at": "2014-01-04T10:00:12+00:00", "flavor_id": "2", "host": "compute2", "id": "6868da77-fa82-4e67-aba9-270c5ae8cbca", "image_ref": "http://image", "launched_at": "2017-12-10T08:10:42+00:00", "metrics": {}, "original_resource_id": "6868DA77-FA82-4E67-ABA9-270C5AE8CBCA", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": null, "revision_start": "2024-07-27T07:23:01.128092+00:00", "server_group": null, "started_at": "2014-01-02T23:23:34+00:00", "type": "instance", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" } ]
Delete¶
It is possible to delete a resource altogether:
DELETE /v1/resource/generic/75c44741-cc60-4033-804e-2d3098c7d2e9 HTTP/1.1 Content-Length: 0HTTP/1.1 204 No Content Content-Length: 0
It is also possible to delete a batch of resources based on attribute values, and returns a number of deleted resources.
Batch¶
To delete resources based on ids:
DELETE /v1/resource/generic HTTP/1.1 Content-Type: application/json Content-Length: 172 { "in": { "id": [ "3cbcca4f-af46-5285-b7f7-951affcb4772", "8a1bfad0-8a51-52ce-938b-c82cd028e91e", "4f46cf43-784d-5e77-9d66-acb2c23ac5b4" ] } }HTTP/1.1 200 OK Content-Length: 14 Content-Type: application/json { "deleted": 3 }
or delete resources based on time:
DELETE /v1/resource/generic HTTP/1.1 Content-Type: application/json Content-Length: 62 { ">=": { "started_at": "2024-07-27T07:23:01.982655+00:00" } }HTTP/1.1 200 OK Content-Length: 14 Content-Type: application/json { "deleted": 1 }
Resource Types¶
Gnocchi is able to manage resource types with custom attributes.
Create¶
To create a new resource type:
POST /v1/resource_type HTTP/1.1 Content-Type: application/json Content-Length: 424 { "attributes": { "display_name": { "required": true, "type": "string" }, "enabled": { "required": false, "type": "bool" }, "launched_at": { "required": false, "type": "datetime" }, "myid": { "type": "uuid" }, "prefix": { "max_length": 8, "min_length": 3, "required": false, "type": "string" }, "size": { "max": 32.8, "min": 5, "type": "number" } }, "name": "my_custom_type" }HTTP/1.1 201 Created Location: http://localhost/v1/resource_type/my_custom_type Content-Length: 451 Content-Type: application/json { "attributes": { "display_name": { "max_length": 255, "min_length": 0, "required": true, "type": "string" }, "enabled": { "required": false, "type": "bool" }, "launched_at": { "required": false, "type": "datetime" }, "myid": { "required": true, "type": "uuid" }, "prefix": { "max_length": 8, "min_length": 3, "required": false, "type": "string" }, "size": { "max": 32.8, "min": 5, "required": true, "type": "number" } }, "name": "my_custom_type", "state": "active" }
Read¶
Then to retrieve its description:
GET /v1/resource_type/my_custom_type HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK Content-Length: 451 Content-Type: application/json { "attributes": { "display_name": { "max_length": 255, "min_length": 0, "required": true, "type": "string" }, "enabled": { "required": false, "type": "bool" }, "launched_at": { "required": false, "type": "datetime" }, "myid": { "required": true, "type": "uuid" }, "prefix": { "max_length": 8, "min_length": 3, "required": false, "type": "string" }, "size": { "max": 32.8, "min": 5, "required": true, "type": "number" } }, "name": "my_custom_type", "state": "active" }
List¶
All resource types can be listed like this:
GET /v1/resource_type HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK Content-Length: 1125 Content-Type: application/json [ { "attributes": {}, "name": "generic", "state": "active" }, { "attributes": { "display_name": { "max_length": 255, "min_length": 0, "required": true, "type": "string" }, "flavor_id": { "max_length": 255, "min_length": 0, "required": true, "type": "string" }, "host": { "max_length": 255, "min_length": 0, "required": true, "type": "string" }, "image_ref": { "max_length": 255, "min_length": 0, "required": true, "type": "string" }, "launched_at": { "required": false, "type": "datetime" }, "server_group": { "max_length": 255, "min_length": 0, "required": false, "type": "string" } }, "name": "instance", "state": "active" }, { "attributes": { "display_name": { "max_length": 255, "min_length": 0, "required": true, "type": "string" }, "enabled": { "required": false, "type": "bool" }, "launched_at": { "required": false, "type": "datetime" }, "myid": { "required": true, "type": "uuid" }, "prefix": { "max_length": 8, "min_length": 3, "required": false, "type": "string" }, "size": { "max": 32.8, "min": 5, "required": true, "type": "number" } }, "name": "my_custom_type", "state": "active" }, { "attributes": {}, "name": "my_other_type", "state": "active" } ]
Update¶
Attributes can be added or removed:
PATCH /v1/resource_type/my_custom_type HTTP/1.1 Content-Type: application/json-patch+json Content-Length: 558 [ { "op": "add", "path": "/attributes/awesome-stuff", "value": { "required": false, "type": "bool" } }, { "op": "add", "path": "/attributes/required-stuff", "value": { "options": { "fill": true }, "required": true, "type": "bool" } }, { "op": "add", "path": "/attributes/required-datetime", "value": { "options": { "fill": "2017-12-11T08:12:42Z" }, "required": true, "type": "datetime" } }, { "op": "remove", "path": "/attributes/prefix" } ]HTTP/1.1 200 OK Content-Length: 537 Content-Type: application/json { "attributes": { "awesome-stuff": { "required": false, "type": "bool" }, "display_name": { "max_length": 255, "min_length": 0, "required": true, "type": "string" }, "enabled": { "required": false, "type": "bool" }, "launched_at": { "required": false, "type": "datetime" }, "myid": { "required": true, "type": "uuid" }, "required-datetime": { "required": true, "type": "datetime" }, "required-stuff": { "required": true, "type": "bool" }, "size": { "max": 32.8, "min": 5, "required": true, "type": "number" } }, "name": "my_custom_type", "state": "active" }
Delete¶
It can also be deleted if no more resources are associated to it:
DELETE /v1/resource_type/my_custom_type HTTP/1.1 Content-Length: 0HTTP/1.1 204 No Content Content-Length: 0
Note
Creating resource type means creation of new tables on the indexer backend. This is heavy operation that will lock some tables for a short amount of time. When the resource type is created, its initial state is creating. When the new tables have been created, the state switches to active and the new resource type is ready to be used. If something unexpected occurs during this step, the state switches to creation_error.
The same behavior occurs when the resource type is deleted. The state starts to switch to deleting, the resource type is no longer usable. Then the tables are removed and then finally the resource_type is really deleted from the database. If some unexpected error occurs the state switches to deletion_error.
Search¶
Gnocchi’s search API supports to the ability to execute a query across resources or metrics. This API provides a language to construct more complex matching contraints beyond basic filtering.
Usage and format¶
You can specify a time range to look for by specifying the start and/or stop query parameter, and the aggregation method to use by specifying the aggregation query parameter.
Query can be expressed in two formats: JSON or STRING.
The supported operators are: equal to (=, == or eq), lesser than (< or lt), greater than (> or gt), less than or equal to (<=, le or ≤) greater than or equal to (>=, ge or ≥) not equal to (!=, ne or ≠), addition (+ or add), substraction (- or sub), multiplication (*, mul or ×), division (/, div or ÷). In JSON format, these operations take only one argument, the second argument being automatically set to the field value. In STRING format, this is just <first argument> <operator> <second argument>
The operators or (or or ∨), and (and or ∧) and not are also supported. In JSON format, it takes a list of arguments as parameters. Using STRING, the format is <left group> and/or <right group> or not <right group>. With the STRING format, parenthesis can be used to create group.
An example of the JSON format:
["and",
["=", ["host", "example1"]],
["like", ["owner", "admin-%"]],
]
And its STRING format equivalent:
host = "example1" or owner like "admin-%"
Resource¶
It’s possible to search for resources using a query mechanism by using the
POST method and uploading a JSON formatted query or by passing a
STRING a formatted query URL-encoded in the filter parameter.
Single filter¶
When listing resources, it is possible to filter resources based on attributes values:
POST /v1/search/resource/instance HTTP/1.1 Content-Type: application/json Content-Length: 58 { "=": { "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" } }HTTP/1.1 200 OK Content-Length: 649 Content-Type: application/json [ { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "display_name": "myvm", "ended_at": "2014-01-04T10:00:12+00:00", "flavor_id": "2", "host": "compute1", "id": "6868da77-fa82-4e67-aba9-270c5ae8cbca", "image_ref": "http://image", "launched_at": "2017-12-10T08:10:42+00:00", "metrics": {}, "original_resource_id": "6868DA77-FA82-4E67-ABA9-270C5AE8CBCA", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": null, "revision_start": "2024-07-27T07:23:00.562714+00:00", "server_group": null, "started_at": "2014-01-02T23:23:34+00:00", "type": "instance", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" } ]
Or even:
POST /v1/search/resource/instance HTTP/1.1 Content-Type: application/json Content-Length: 30 { "like": { "host": "compute%" } }HTTP/1.1 200 OK Content-Length: 649 Content-Type: application/json [ { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "display_name": "myvm", "ended_at": "2014-01-04T10:00:12+00:00", "flavor_id": "2", "host": "compute1", "id": "6868da77-fa82-4e67-aba9-270c5ae8cbca", "image_ref": "http://image", "launched_at": "2017-12-10T08:10:42+00:00", "metrics": {}, "original_resource_id": "6868DA77-FA82-4E67-ABA9-270C5AE8CBCA", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": null, "revision_start": "2024-07-27T07:23:00.562714+00:00", "server_group": null, "started_at": "2014-01-02T23:23:34+00:00", "type": "instance", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" } ]
For the filter parameter version, the value is the URL-encoded version of
host like "compute%"
POST /v1/search/resource/instance?filter=host%20like%20%22compute%25%22 HTTP/1.1 Content-Type: application/json Content-Length: 0HTTP/1.1 200 OK Content-Length: 649 Content-Type: application/json [ { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "display_name": "myvm", "ended_at": "2014-01-04T10:00:12+00:00", "flavor_id": "2", "host": "compute1", "id": "6868da77-fa82-4e67-aba9-270c5ae8cbca", "image_ref": "http://image", "launched_at": "2017-12-10T08:10:42+00:00", "metrics": {}, "original_resource_id": "6868DA77-FA82-4E67-ABA9-270C5AE8CBCA", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": null, "revision_start": "2024-07-27T07:23:00.562714+00:00", "server_group": null, "started_at": "2014-01-02T23:23:34+00:00", "type": "instance", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" } ]
Multiple filters¶
Complex operators such as and and or are also available:
POST /v1/search/resource/instance HTTP/1.1 Content-Type: application/json Content-Length: 113 { "and": [ { "=": { "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" } }, { ">=": { "started_at": "2010-01-01" } } ] }HTTP/1.1 200 OK Content-Length: 649 Content-Type: application/json [ { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "display_name": "myvm", "ended_at": "2014-01-04T10:00:12+00:00", "flavor_id": "2", "host": "compute1", "id": "6868da77-fa82-4e67-aba9-270c5ae8cbca", "image_ref": "http://image", "launched_at": "2017-12-10T08:10:42+00:00", "metrics": {}, "original_resource_id": "6868DA77-FA82-4E67-ABA9-270C5AE8CBCA", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": null, "revision_start": "2024-07-27T07:23:00.562714+00:00", "server_group": null, "started_at": "2014-01-02T23:23:34+00:00", "type": "instance", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" } ]
filter version is
user_id = "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" and started_at >= "2010-01-01"
URL-encoded.
POST /v1/search/resource/instance?filter=user_id%20%3D%20%22BD3A1E52-1C62-44CB-BF04-660BD88CD74D%22%20and%20started_at%20%3E%3D%20%222010-01-01%22 HTTP/1.1 Content-Type: application/json Content-Length: 0HTTP/1.1 200 OK Content-Length: 649 Content-Type: application/json [ { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "display_name": "myvm", "ended_at": "2014-01-04T10:00:12+00:00", "flavor_id": "2", "host": "compute1", "id": "6868da77-fa82-4e67-aba9-270c5ae8cbca", "image_ref": "http://image", "launched_at": "2017-12-10T08:10:42+00:00", "metrics": {}, "original_resource_id": "6868DA77-FA82-4E67-ABA9-270C5AE8CBCA", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": null, "revision_start": "2024-07-27T07:23:00.562714+00:00", "server_group": null, "started_at": "2014-01-02T23:23:34+00:00", "type": "instance", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" } ]
With details¶
Details about the resource can also be retrieved at the same time:
POST /v1/search/resource/generic?details=true HTTP/1.1 Content-Type: application/json Content-Length: 58 { "=": { "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" } }HTTP/1.1 200 OK Content-Length: 1124 Content-Type: application/json [ { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "ended_at": null, "id": "75c44741-cc60-4033-804e-2d3098c7d2e9", "metrics": {}, "original_resource_id": "75C44741-CC60-4033-804E-2D3098C7D2E9", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": null, "revision_start": "2024-07-27T07:23:00.500012+00:00", "started_at": "2024-07-27T07:23:00.500006+00:00", "type": "generic", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" }, { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "display_name": "myvm", "ended_at": "2014-01-04T10:00:12+00:00", "flavor_id": "2", "host": "compute1", "id": "6868da77-fa82-4e67-aba9-270c5ae8cbca", "image_ref": "http://image", "launched_at": "2017-12-10T08:10:42+00:00", "metrics": {}, "original_resource_id": "6868DA77-FA82-4E67-ABA9-270C5AE8CBCA", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": null, "revision_start": "2024-07-27T07:23:00.562714+00:00", "server_group": null, "started_at": "2014-01-02T23:23:34+00:00", "type": "instance", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" } ]
Limit attributes¶
To limit response attributes, use attrs=id&attrs=started_at&attrs=user_id in the query parameter:
POST /v1/search/resource/generic?attrs=id&attrs=started_at&attrs=user_id HTTP/1.1 Content-Type: application/json Content-Length: 58 { "=": { "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" } }HTTP/1.1 200 OK Content-Length: 291 Content-Type: application/json [ { "id": "75c44741-cc60-4033-804e-2d3098c7d2e9", "started_at": "2024-07-27T07:23:00.500006+00:00", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" }, { "id": "6868da77-fa82-4e67-aba9-270c5ae8cbca", "started_at": "2014-01-02T23:23:34+00:00", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" } ]
History¶
It’s possible to search for old revisions of resources in the same ways:
POST /v1/search/resource/instance?history=true HTTP/1.1 Content-Type: application/json Content-Length: 53 { "=": { "id": "6868da77-fa82-4e67-aba9-270c5ae8cbca" } }HTTP/1.1 200 OK Content-Length: 1328 Content-Type: application/json [ { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "display_name": "myvm", "ended_at": "2014-01-04T10:00:12+00:00", "flavor_id": "2", "host": "compute1", "id": "6868da77-fa82-4e67-aba9-270c5ae8cbca", "image_ref": "http://image", "launched_at": "2017-12-10T08:10:42+00:00", "metrics": {}, "original_resource_id": "6868DA77-FA82-4E67-ABA9-270C5AE8CBCA", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": "2024-07-27T07:23:01.128092+00:00", "revision_start": "2024-07-27T07:23:00.562714+00:00", "server_group": null, "started_at": "2014-01-02T23:23:34+00:00", "type": "instance", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" }, { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "display_name": "myvm", "ended_at": "2014-01-04T10:00:12+00:00", "flavor_id": "2", "host": "compute2", "id": "6868da77-fa82-4e67-aba9-270c5ae8cbca", "image_ref": "http://image", "launched_at": "2017-12-10T08:10:42+00:00", "metrics": {}, "original_resource_id": "6868DA77-FA82-4E67-ABA9-270C5AE8CBCA", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": null, "revision_start": "2024-07-27T07:23:01.128092+00:00", "server_group": null, "started_at": "2014-01-02T23:23:34+00:00", "type": "instance", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" } ]
Time range¶
The timerange of the history can be set, too:
POST /v1/search/resource/instance?history=true HTTP/1.1 Content-Type: application/json Accept: application/json Content-Length: 227 { "and": [ { "=": { "host": "compute1" } }, { ">=": { "revision_start": "2024-07-27T07:23:00.562714+00:00" } }, { "or": [ { "<=": { "revision_end": "2024-07-27T07:23:01.128092+00:00" } }, { "=": { "revision_end": null } } ] } ] }HTTP/1.1 200 OK Content-Length: 1289 Content-Type: application/json [ { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "display_name": "myvm", "ended_at": "2014-01-04T10:00:12+00:00", "flavor_id": "2", "host": "compute1", "id": "6868da77-fa82-4e67-aba9-270c5ae8cbca", "image_ref": "http://image", "launched_at": "2017-12-10T08:10:42+00:00", "metrics": {}, "original_resource_id": "6868DA77-FA82-4E67-ABA9-270C5AE8CBCA", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": "2024-07-27T07:23:01.128092+00:00", "revision_start": "2024-07-27T07:23:00.562714+00:00", "server_group": null, "started_at": "2014-01-02T23:23:34+00:00", "type": "instance", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" }, { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "display_name": "myvm", "ended_at": null, "flavor_id": "2", "host": "compute1", "id": "ab0b5802-e79b-4c84-8998-9237f60d9cae", "image_ref": "http://image", "launched_at": null, "metrics": {}, "original_resource_id": "AB0B5802-E79B-4C84-8998-9237F60D9CAE", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": null, "revision_start": "2024-07-27T07:23:01.098905+00:00", "server_group": null, "started_at": "2024-07-27T07:23:01.098896+00:00", "type": "instance", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" } ]
This can be done with the filter parameter too:
host = 'compute1' and revision_start >= "2024-07-27T07:23:00.562714+00:00" and (revision_end <= "2024-07-27T07:23:01.128092+00:00" or revision_end == null)
POST /v1/search/resource/instance?history=true&filter=host%20%3D%20%27compute1%27%20and%20revision_start%20%3E%3D%20%222024-07-27T07%3A23%3A00.562714%2B00%3A00%22%20and%20%28revision_end%20%3C%3D%20%222024-07-27T07%3A23%3A01.128092%2B00%3A00%22%20or%20revision_end%20%3D%3D%20null%29 HTTP/1.1 Content-Type: application/json Accept: application/json Content-Length: 227 { "and": [ { "=": { "host": "compute1" } }, { ">=": { "revision_start": "2024-07-27T07:23:00.562714+00:00" } }, { "or": [ { "<=": { "revision_end": "2024-07-27T07:23:01.128092+00:00" } }, { "=": { "revision_end": null } } ] } ] }HTTP/1.1 200 OK Content-Length: 1289 Content-Type: application/json [ { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "display_name": "myvm", "ended_at": "2014-01-04T10:00:12+00:00", "flavor_id": "2", "host": "compute1", "id": "6868da77-fa82-4e67-aba9-270c5ae8cbca", "image_ref": "http://image", "launched_at": "2017-12-10T08:10:42+00:00", "metrics": {}, "original_resource_id": "6868DA77-FA82-4E67-ABA9-270C5AE8CBCA", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": "2024-07-27T07:23:01.128092+00:00", "revision_start": "2024-07-27T07:23:00.562714+00:00", "server_group": null, "started_at": "2014-01-02T23:23:34+00:00", "type": "instance", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" }, { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "display_name": "myvm", "ended_at": null, "flavor_id": "2", "host": "compute1", "id": "ab0b5802-e79b-4c84-8998-9237f60d9cae", "image_ref": "http://image", "launched_at": null, "metrics": {}, "original_resource_id": "AB0B5802-E79B-4C84-8998-9237F60D9CAE", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": null, "revision_start": "2024-07-27T07:23:01.098905+00:00", "server_group": null, "started_at": "2024-07-27T07:23:01.098896+00:00", "type": "instance", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" } ]
Magic¶
The special attribute lifespan which is equivalent to ended_at - started_at is also available in the filtering queries.
POST /v1/search/resource/instance HTTP/1.1 Content-Type: application/json Content-Length: 30 { ">=": { "lifespan": "30 min" } }HTTP/1.1 200 OK Content-Length: 649 Content-Type: application/json [ { "created_by_project_id": "", "created_by_user_id": "admin", "creator": "admin", "display_name": "myvm", "ended_at": "2014-01-04T10:00:12+00:00", "flavor_id": "2", "host": "compute1", "id": "6868da77-fa82-4e67-aba9-270c5ae8cbca", "image_ref": "http://image", "launched_at": "2017-12-10T08:10:42+00:00", "metrics": {}, "original_resource_id": "6868DA77-FA82-4E67-ABA9-270C5AE8CBCA", "project_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D", "revision_end": null, "revision_start": "2024-07-27T07:23:00.562714+00:00", "server_group": null, "started_at": "2014-01-02T23:23:34+00:00", "type": "instance", "user_id": "BD3A1E52-1C62-44CB-BF04-660BD88CD74D" } ]
Metric¶
It is possible to search for values in metrics. For example, this will look for all values that are greater than or equal to 50 if we add 23 to them and that are not equal to 55. You have to specify the list of metrics to look into by using the metric_id query parameter several times.
POST /v1/search/metric?metric_id=7d8f95b9-c262-454e-83a2-3035d9c6f6d5 HTTP/1.1 Content-Type: application/json Content-Length: 46 { "and": [ { ">=": [ { "+": 23 }, 50 ] }, { "!=": 55 } ] }HTTP/1.1 200 OK Content-Length: 127 Content-Type: application/json { "7d8f95b9-c262-454e-83a2-3035d9c6f6d5": [ [ "2014-10-06T14:33:00+00:00", 60.0, 43.1 ], [ "2014-10-06T14:33:57+00:00", 1.0, 43.1 ] ] }
And it is possible to search for values in metrics by using one or more granularities:
POST /v1/search/metric?metric_id=07c578b5-9d0d-4cb8-8367-148c24f602fa&granularity=1second&granularity=1800s HTTP/1.1 Content-Type: application/json Content-Length: 9 { "=": 12 }HTTP/1.1 200 OK Content-Length: 381 Content-Type: application/json { "07c578b5-9d0d-4cb8-8367-148c24f602fa": [ [ "2014-10-06T14:30:00+00:00", 1800.0, 12.0 ], [ "2014-10-06T14:34:12+00:00", 1.0, 12.0 ], [ "2014-10-06T14:34:14+00:00", 1.0, 12.0 ], [ "2014-10-06T14:34:16+00:00", 1.0, 12.0 ], [ "2014-10-06T14:34:18+00:00", 1.0, 12.0 ], [ "2014-10-06T14:34:20+00:00", 1.0, 12.0 ], [ "2014-10-06T14:34:22+00:00", 1.0, 12.0 ], [ "2014-10-06T14:34:24+00:00", 1.0, 12.0 ] ] }
Dynamic Aggregates¶
Gnocchi supports the ability to make on-the-fly reaggregations of existing metrics and the ability to manipulate and transform metrics as required. This is accomplished by passing an operations value describing the actions to apply to the metrics.
Note
operations can also be passed as a string, for example: “operations”: “(aggregate mean (metric (metric-id aggregation) (metric-id aggregation))”
Cross-metric Usage¶
Aggregation across multiple metrics have different behavior depending on whether boundary values are set (start and stop) and if needed_overlap is set.
Overlap percentage¶
Gnocchi expects that time series have a certain percentage of timestamps in common. This percent is controlled by the needed_overlap needed_overlap, which by default expects 100% overlap. If this percentage is not reached, an error is returned.
Note
If start or stop boundary is not set, Gnocchi will set the missing boundary to the first or last timestamp common across all series.
Backfill¶
The ability to fill in missing points from a subset of time series is supported by specifying a fill value. Valid fill values include any float, dropna, null, ffill, bfill full_ffill or full_bfill. In the case of null, Gnocchi will compute the aggregation using only the existing points. dropna is like null but remove NaN from the result. ffill fills NaN measures in one metric with previous non-NaN value, bfill fills NaN measures with next non-NaN value, so if the metric starts (or, in case of bfill, ends) with NaNs, those wouldn’t change and would be excluded from the resulting set, just like dropna does. To fill the remaining NaN values, producing a metric with as much timestamps as there are in all metrics combined, one can use full_ffill and full_bfill variants. In the case of full_ffill, it applies a forward then backward fill and for full_bfill, a backward then forward fill.
POST /v1/aggregates?fill=0&granularity=1 HTTP/1.1 Content-Type: application/json Content-Length: 95 { "operations": "(* (aggregate mean (metric 7d8f95b9-c262-454e-83a2-3035d9c6f6d5 mean)) 4)" }HTTP/1.1 200 OK Content-Length: 156 Content-Type: application/json { "measures": { "aggregated": [ [ "2014-10-06T14:33:57+00:00", 1.0, 14.0 ], [ "2014-10-06T14:34:12+00:00", 1.0, 80.0 ], [ "2014-10-06T14:34:20+00:00", 1.0, 36.0 ] ] } }
Search and aggregate¶
It’s also possible to do that aggregation on metrics linked to resources. In order to select these resources, the following endpoint accepts a query such as the one described in the resource search API.
POST /v1/aggregates?start=2014-10-06T14:34 HTTP/1.1 Content-Type: application/json Content-Length: 175 { "operations": [ "*", [ "aggregate", "mean", [ "metric", "cpu.util", "mean" ] ], 4 ], "resource_type": "instance", "search": { "=": { "server_group": "my_autoscaling_group" } } }HTTP/1.1 200 OK Content-Length: 172 Content-Type: application/json { "measures": { "aggregated": [ [ "2014-10-06T14:00:00+00:00", 3600.0, 50.86666666666667 ], [ "2014-10-06T14:34:12+00:00", 1.0, 49.0 ], [ "2014-10-06T14:34:20+00:00", 1.0, 46.4 ] ] } }
And metric name can be wildcard too.
POST /v1/aggregates?start=2014-10-06T14:34 HTTP/1.1 Content-Type: application/json Content-Length: 171 { "operations": [ "*", [ "aggregate", "mean", [ "metric", "cpu*", "mean" ] ], 4 ], "resource_type": "instance", "search": { "=": { "server_group": "my_autoscaling_group" } } }HTTP/1.1 200 OK Content-Length: 172 Content-Type: application/json { "measures": { "aggregated": [ [ "2014-10-06T14:00:00+00:00", 3600.0, 50.86666666666667 ], [ "2014-10-06T14:34:12+00:00", 1.0, 49.0 ], [ "2014-10-06T14:34:20+00:00", 1.0, 46.4 ] ] } }
Groupby¶
It is possible to group the resource search results by any attribute of the requested resource type, and then compute the aggregation:
POST /v1/aggregates?start=2014-10-06T14:34&groupby=host&groupby=flavor_id HTTP/1.1 Content-Type: application/json Content-Length: 149 { "operations": "(* (aggregate mean (metric cpu.util mean)) 4)", "resource_type": "instance", "search": "server_group='my_autoscaling_group'" }HTTP/1.1 200 OK Content-Length: 550 Content-Type: application/json [ { "group": { "flavor_id": "2", "host": "compute1" }, "measures": { "measures": { "aggregated": [ [ "2014-10-06T14:00:00+00:00", 3600.0, 43.333333333333336 ], [ "2014-10-06T14:34:00+00:00", 60.0, 58.0 ], [ "2014-10-06T14:34:12+00:00", 1.0, 80.0 ], [ "2014-10-06T14:34:20+00:00", 1.0, 36.0 ] ] } } }, { "group": { "flavor_id": "2", "host": "compute2" }, "measures": { "measures": { "aggregated": [ [ "2014-10-06T14:00:00+00:00", 3600.0, 58.4 ], [ "2014-10-06T14:30:00+00:00", 1800.0, 58.4 ], [ "2014-10-06T14:34:12+00:00", 1.0, 18.0 ], [ "2014-10-06T14:34:20+00:00", 1.0, 56.8 ] ] } } } ]
Gnocchi will only group by the last attribute’s value for each resource. Therefore, in cases that the resource’s attributes get changed over the provided time window, Gnocchi will only display the last attributes’ value. Below we have an example of a resource that had the flavor_id attribute changed from 1 to 2 after 2015.
POST /v1/aggregates?groupby=flavor_id HTTP/1.1 Content-Type: application/json Content-Length: 152 { "operations": "(aggregate mean (metric vm.uptime mean))", "resource_type": "instance", "search": "id = 'c14fafd8-2e50-4a1a-ac13-319c82fb2f65'" }HTTP/1.1 200 OK Content-Length: 251 Content-Type: application/json [ { "group": { "flavor_id": "2" }, "measures": { "measures": { "aggregated": [ [ "2015-03-06T14:00:00+00:00", 3600.0, 1.0 ], [ "2015-03-06T15:00:00+00:00", 3600.0, 1.0 ], [ "2099-01-01T01:00:00+00:00", 3600.0, 1.0 ], [ "2099-01-01T02:00:00+00:00", 3600.0, 1.0 ] ] } } } ]
Between the measurements from 2015 and 2099, the flavor_id was changed from 1 to 2, but in the response, the flavor_id is as if it was always 2. In these cases, if you want to get all the attributes’ values over the time window, and group each measurement by the attribute value the resource has when the measurement was collected, you can add the use_history=true in the aggregates API request, like the following example.
POST /v1/aggregates?groupby=flavor_id&use_history=true HTTP/1.1 Content-Type: application/json Content-Length: 152 { "operations": "(aggregate mean (metric vm.uptime mean))", "resource_type": "instance", "search": "id = 'c14fafd8-2e50-4a1a-ac13-319c82fb2f65'" }HTTP/1.1 200 OK Content-Length: 326 Content-Type: application/json [ { "group": { "flavor_id": "1" }, "measures": { "measures": { "aggregated": [ [ "2015-03-06T14:00:00+00:00", 3600.0, 1.0 ], [ "2015-03-06T15:00:00+00:00", 3600.0, 1.0 ] ] } } }, { "group": { "flavor_id": "2" }, "measures": { "measures": { "aggregated": [ [ "2099-01-01T01:00:00+00:00", 3600.0, 1.0 ], [ "2099-01-01T02:00:00+00:00", 3600.0, 1.0 ] ] } } } ]
When using the use_history=True, the re-aggregation process will not work as expected, as the values in the timespan that represents the granularity being consulted are added together to enable the split into different portions according to their participation in the timeframe being consulted. Therefore, if one wants to use the re-aggregation process, he/she needs to consider using the use_history option as False.
List of supported <operations>¶
Get one or more metrics¶
(metric <metric-id> <aggregation>)
(metric ((<metric-id> <aggregation>) (<metric-id> <aggregation>) ...))
metric-id: the id of a metric to retrieve
aggregation: the aggregation method to retrieve
Note
When used alone, this provides the ability to retrieve multiple metrics in a single request.
Rolling window aggregation¶
(rolling <aggregation method> <rolling window> (<operations>))
aggregation method: the aggregation method to use to compute the rolling window.
(mean, median, std, min, max, sum, var, count)
rolling window: number of previous values to aggregate
Aggregation across metrics¶
aggregate <aggregation method> ((<operations>), (<operations>), ...))
aggregation method: the aggregation method to use to compute the aggregate between metrics
(mean, median, std, min, max, sum, var, count)
Resample¶
(resample <aggregation method> <granularity> (<operations>))
aggregation method: the aggregation method to use to compute the aggregate between metrics
(mean, median, std, min, max, sum, var, count)
granularity: the granularity (e.g.: 1d, 60s, ...)
Note
If you plan to execute the query often, it is recommended for performance to leverage an archive policy with the needed granularity instead of resampling the time series on each query.
Math operations¶
(<operator> <operations_or_float> <operations_or_float>)
operator: %, mod, +, add, -, sub, *, ×, mul, /, ÷, div, **, ^, pow
Boolean operations¶
(<operator> <operations_or_float> <operations_or_float>)
operator: =, ==, eq, <, lt, >, gt, <=, ≤, le, =, ≥, ge, !=, ≠, ne
Function operations¶
(abs (<operations>))
(absolute (<operations>))
(neg (<operations>))
(negative (<operations>))
(cos (<operations>))
(sin (<operations>))
(tan (<operations>))
(floor (<operations>))
(ceil (<operations>))
(clip (<operations>))
(clip_min (<operations>))
(clip_max (<operations>))
(rateofchange (<operations>))
Examples¶
Aggregate then math¶
The following computes the mean aggregates with all metrics listed in metrics and then multiples it by 4.
POST /v1/aggregates?start=2014-10-06T14:34&stop=2017-10-06T14:34 HTTP/1.1 Content-Type: application/json Content-Length: 209 { "operations": [ "*", [ "aggregate", "mean", [ "metric", [ "7d8f95b9-c262-454e-83a2-3035d9c6f6d5", "mean" ], [ "17b898d0-8723-4287-8968-4f6921eeefeb", "mean" ] ] ], 4 ] }HTTP/1.1 200 OK Content-Length: 172 Content-Type: application/json { "measures": { "aggregated": [ [ "2014-10-06T14:00:00+00:00", 3600.0, 50.86666666666667 ], [ "2014-10-06T14:34:12+00:00", 1.0, 49.0 ], [ "2014-10-06T14:34:20+00:00", 1.0, 46.4 ] ] } }
Between metrics¶
Operations between metrics can also be done, such as:
POST /v1/aggregates?start=2014-10-06T14:34&stop=2017-10-06T14:34 HTTP/1.1 Content-Type: application/json Content-Length: 196 { "operations": [ "absolute", [ "**", [ "metric", "7d8f95b9-c262-454e-83a2-3035d9c6f6d5", "mean" ], [ "metric", "17b898d0-8723-4287-8968-4f6921eeefeb", "mean" ] ] ] }HTTP/1.1 200 OK Content-Length: 199 Content-Type: application/json { "measures": { "aggregated": [ [ "2014-10-06T14:00:00+00:00", 3600.0, 1280933994231924.0 ], [ "2014-10-06T14:34:12+00:00", 1.0, 715541.7527999327 ], [ "2014-10-06T14:34:20+00:00", 1.0, 35501249116611.47 ] ] } }
List the top N resources that consume the most CPU during the last hour¶
The following is configured so that stop - start = granularity will get only one point per instance.
This will give all information needed to order by cpu.util timeseries which can be filtered down to N results.
POST /v1/aggregates?start=2014-10-06T14:00&stop=2014-10-06T15:00&groupby=original_resource_id&groupby=display_name&granularity=3600.0 HTTP/1.1 Content-Type: application/json Content-Length: 126 { "operations": "(metric cpu.util mean)", "resource_type": "instance", "search": "server_group='my_autoscaling_group'" }HTTP/1.1 200 OK Content-Length: 496 Content-Type: application/json [ { "group": { "display_name": "myvm2", "original_resource_id": "6F24EDD9-5A2F-4592-B708-FFBED821C5D2" }, "measures": { "measures": { "6f24edd9-5a2f-4592-b708-ffbed821c5d2": { "cpu.util": { "mean": [ [ "2014-10-06T14:00:00+00:00", 3600.0, 10.833333333333334 ] ] } } } } }, { "group": { "display_name": "myvm3", "original_resource_id": "15e9c872-7ca9-11e4-a2da-2fb4032dfc09" }, "measures": { "measures": { "15e9c872-7ca9-11e4-a2da-2fb4032dfc09": { "cpu.util": { "mean": [ [ "2014-10-06T14:00:00+00:00", 3600.0, 14.6 ] ] } } } } } ]
Aggregation across metrics (deprecated)¶
Note
This API have been replaced by the more flexible aggregates API
Gnocchi supports on-the-fly aggregation of previously aggregated data of metrics.
It can be done by providing the list of metrics to aggregate:
GET /v1/aggregation/metric?metric=7d8f95b9-c262-454e-83a2-3035d9c6f6d5&metric=17b898d0-8723-4287-8968-4f6921eeefeb&start=2014-10-06T14:34&aggregation=mean HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK Content-Length: 144 Content-Type: application/json [ [ "2014-10-06T14:00:00+00:00", 3600.0, 12.716666666666667 ], [ "2014-10-06T14:34:12+00:00", 1.0, 12.25 ], [ "2014-10-06T14:34:20+00:00", 1.0, 11.6 ] ]
Note
This aggregation is done against the aggregates built and updated for a metric when new measurements are posted in Gnocchi. Therefore, the aggregate of this already aggregated data may not have sense for certain kind of aggregation method (e.g. stdev).
By default, the measures are aggregated using the aggregation method provided, e.g. you’ll get a mean of means, or a max of maxs. You can specify what method to use over the retrieved aggregation by using the reaggregation parameter:
GET /v1/aggregation/metric?metric=7d8f95b9-c262-454e-83a2-3035d9c6f6d5&metric=17b898d0-8723-4287-8968-4f6921eeefeb&aggregation=mean&reaggregation=min HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK Content-Length: 182 Content-Type: application/json [ [ "2014-10-06T14:00:00+00:00", 3600.0, 10.833333333333334 ], [ "2014-10-06T14:33:57+00:00", 1.0, 3.5 ], [ "2014-10-06T14:34:12+00:00", 1.0, 4.5 ], [ "2014-10-06T14:34:20+00:00", 1.0, 9.0 ] ]
It’s also possible to do that aggregation on metrics linked to resources. In order to select these resources, the following endpoint accepts a query such as the one described in the resource search API.
POST /v1/aggregation/resource/instance/metric/cpu.util?start=2014-10-06T14:34&aggregation=mean HTTP/1.1 Content-Type: application/json Content-Length: 47 { "=": { "server_group": "my_autoscaling_group" } }HTTP/1.1 200 OK Content-Length: 144 Content-Type: application/json [ [ "2014-10-06T14:00:00+00:00", 3600.0, 12.716666666666667 ], [ "2014-10-06T14:34:12+00:00", 1.0, 12.25 ], [ "2014-10-06T14:34:20+00:00", 1.0, 11.6 ] ]
Like for searching resource, the query
server_group = "my_autoscaling_group"
can be passed in filter parameter
POST /v1/aggregation/resource/instance/metric/cpu.util?start=2014-10-06T14:34&aggregation=mean&filter=server_group%20%3D%20%22my_autoscaling_group%22 HTTP/1.1 Content-Type: application/json Content-Length: 0HTTP/1.1 200 OK Content-Length: 144 Content-Type: application/json [ [ "2014-10-06T14:00:00+00:00", 3600.0, 12.716666666666667 ], [ "2014-10-06T14:34:12+00:00", 1.0, 12.25 ], [ "2014-10-06T14:34:20+00:00", 1.0, 11.6 ] ]
It is possible to group the resource search results by any attribute of the requested resource type, and then compute the aggregation:
POST /v1/aggregation/resource/instance/metric/cpu.util?groupby=host&groupby=flavor_id HTTP/1.1 Content-Type: application/json Content-Length: 47 { "=": { "server_group": "my_autoscaling_group" } }HTTP/1.1 200 OK Content-Length: 613 Content-Type: application/json [ { "group": { "flavor_id": "2", "host": "compute1" }, "measures": [ [ "2014-10-06T14:00:00+00:00", 3600.0, 10.833333333333334 ], [ "2014-10-06T14:33:00+00:00", 60.0, 3.5 ], [ "2014-10-06T14:34:00+00:00", 60.0, 14.5 ], [ "2014-10-06T14:33:57+00:00", 1.0, 3.5 ], [ "2014-10-06T14:34:12+00:00", 1.0, 20.0 ], [ "2014-10-06T14:34:20+00:00", 1.0, 9.0 ] ] }, { "group": { "flavor_id": "2", "host": "compute2" }, "measures": [ [ "2014-10-06T14:00:00+00:00", 3600.0, 14.6 ], [ "2014-10-06T14:30:00+00:00", 1800.0, 14.6 ], [ "2014-10-06T14:33:57+00:00", 1.0, 25.1 ], [ "2014-10-06T14:34:12+00:00", 1.0, 4.5 ], [ "2014-10-06T14:34:20+00:00", 1.0, 14.2 ] ] } ]
Similar to retrieving aggregates for a single metric, the refresh parameter can be provided to force all POSTed measures to be processed across all metrics before computing the result. The resample parameter may be used as well.
Note
Tranformations (eg: resample, absolute, …) are done prior to any reaggregation if both parameters are specified.
Also, aggregation across metrics have different behavior depending on whether boundary values are set (‘start’ and ‘stop’) and if ‘needed_overlap’ is set.
Gnocchi expects that we have a certain percent of timestamps common between time series. This percent is controlled by needed_overlap, which by default expects 100% overlap. If this percentage is not reached, an error is returned.
Note
If start or stop boundary is not set, Gnocchi will set the missing boundary to the first or last timestamp common across all series.
The ability to fill in missing points from a subset of time series is supported by specifying a fill value. Valid fill values include any float, dropna, null, ffill, bfill full_ffill or full_bfill. In the case of null, Gnocchi will compute the aggregation using only the existing points. dropna is like null but remove NaN from the result. ffill fills NaN measures in one metric with previous non-NaN value, bfill fills NaN measures with next non-NaN value, so if the metric starts (or, in case of bfill, ends) with NaNs, those wouldn’t change and would be excluded from the resulting set, just like dropna does. To fill the remaining NaN values, producing a metric with as much timestamps as there are in all metrics combined, one can use full_ffill and full_bfill variants. In the case of full_ffill, it applies a forward then backward fill and for full_bfill, a backward then forward fill.
GET /v1/aggregation/metric?metric=7d8f95b9-c262-454e-83a2-3035d9c6f6d5&metric=17b898d0-8723-4287-8968-4f6921eeefeb&fill=0&granularity=1 HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK Content-Length: 127 Content-Type: application/json [ [ "2014-10-06T14:33:57+00:00", 1.0, 14.3 ], [ "2014-10-06T14:34:12+00:00", 1.0, 12.25 ], [ "2014-10-06T14:34:20+00:00", 1.0, 11.6 ] ]
Capabilities¶
The list aggregation methods that can be used in Gnocchi are extendable and can differ between deployments. It is possible to get the supported list of aggregation methods from the API server:
GET /v1/capabilities HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK Content-Length: 2473 Content-Type: application/json { "aggregation_methods": [ "58pct", "rate:51pct", "rate:33pct", "17pct", "rate:last", "96pct", "rate:68pct", "14pct", "rate:48pct", "rate:31pct", "rate:20pct", "8pct", "21pct", "rate:first", "49pct", "rate:11pct", "67pct", "59pct", "3pct", "rate:24pct", "rate:61pct", "rate:93pct", "1pct", "72pct", "rate:38pct", "63pct", "97pct", "rate:82pct", "86pct", "46pct", "77pct", "87pct", "28pct", "rate:49pct", "rate:53pct", "median", "rate:64pct", "rate:52pct", "rate:83pct", "rate:92pct", "90pct", "rate:59pct", "rate:23pct", "52pct", "2pct", "44pct", "rate:62pct", "89pct", "84pct", "rate:10pct", "rate:89pct", "rate:75pct", "20pct", "50pct", "80pct", "rate:65pct", "rate:30pct", "41pct", "rate:54pct", "rate:76pct", "47pct", "rate:sum", "65pct", "9pct", "57pct", "62pct", "rate:66pct", "23pct", "rate:median", "rate:17pct", "rate:8pct", "rate:98pct", "rate:69pct", "31pct", "rate:97pct", "36pct", "6pct", "rate:6pct", "73pct", "rate:std", "45pct", "35pct", "55pct", "rate:35pct", "max", "24pct", "rate:22pct", "count", "56pct", "rate:46pct", "rate:7pct", "13pct", "19pct", "29pct", "rate:77pct", "rate:mean", "rate:3pct", "rate:2pct", "rate:21pct", "rate:19pct", "rate:count", "rate:94pct", "rate:70pct", "rate:18pct", "88pct", "18pct", "rate:45pct", "rate:25pct", "rate:1pct", "69pct", "12pct", "51pct", "76pct", "64pct", "78pct", "32pct", "rate:43pct", "11pct", "rate:86pct", "min", "rate:90pct", "rate:91pct", "30pct", "rate:56pct", "rate:27pct", "rate:44pct", "rate:39pct", "33pct", "rate:87pct", "61pct", "81pct", "68pct", "27pct", "66pct", "4pct", "7pct", "16pct", "75pct", "54pct", "93pct", "71pct", "92pct", "rate:5pct", "34pct", "rate:26pct", "rate:28pct", "70pct", "rate:63pct", "95pct", "rate:55pct", "94pct", "sum", "rate:47pct", "rate:37pct", "rate:42pct", "43pct", "91pct", "rate:73pct", "rate:min", "rate:29pct", "82pct", "5pct", "rate:12pct", "rate:78pct", "40pct", "rate:57pct", "rate:34pct", "98pct", "rate:40pct", "25pct", "rate:96pct", "60pct", "22pct", "26pct", "42pct", "39pct", "99pct", "rate:14pct", "rate:84pct", "83pct", "rate:74pct", "48pct", "15pct", "rate:81pct", "79pct", "37pct", "rate:99pct", "rate:max", "rate:15pct", "rate:80pct", "rate:41pct", "rate:36pct", "rate:13pct", "rate:79pct", "rate:32pct", "mean", "rate:95pct", "last", "38pct", "rate:85pct", "rate:16pct", "rate:9pct", "rate:50pct", "10pct", "74pct", "std", "53pct", "rate:67pct", "rate:60pct", "rate:88pct", "rate:4pct", "rate:71pct", "rate:72pct", "85pct", "rate:58pct", "first" ] }
Status¶
The overall status of the Gnocchi installation can be retrieved via an API call reporting values such as the number of new measures to process for each metric:
GET /v1/status HTTP/1.1 Content-Length: 0HTTP/1.1 200 OK Content-Length: 133 Content-Type: application/json { "metricd": { "processors": null, "statistics": {} }, "storage": { "measures_to_process": {}, "summary": { "measures": 0, "metrics": 0 } } }
Timestamp format¶
Timestamps used in Gnocchi are always returned using the ISO 8601 format. Gnocchi is able to understand a few formats of timestamp when querying or creating resources, for example
“2014-01-01 12:12:34” or “2014-05-20T10:00:45.856219”, ISO 8601 timestamps.
“10 minutes”, which means “10 minutes from now”.
“-2 days”, which means “2 days ago”.
1421767030, a Unix epoch based timestamp.