Page History
...
- Here we check for a correlation between umm-lib transformation time and length of concept metadata in characters, and fail to observe an obvious correlation – it is possible there may be a slight correlation for the median (blues) and 75th percentile (green) values, but the 'spikey' shape coinciding with decrease .
- Decrease in variability at the high ends suggests the possibility that those bins might simply occurs in bins that have too few events. (6 or even less)
- Three graphs are shown below, so that we may examine total time, generate step time, and parse step time separately. (With median, 75th, and 95th percentile shown for each) Recall that generate step + parse step = total time.
...
| UI Expand | ||
|---|---|---|
| ||
log_group="cmr-search-wl" "transform-with-strategy umm-lib:" "target-formats: [{:format :umm-json" earliest="06/05/2023:00:00:00" latest="06/05/2023:23:59:00" | rex "transform-with-strategy umm-lib: legacy/parse-concept time: (?<ms_parse>\d+) reduce w/ legacy/generate-metadata time: (?<ms_generate>\d+) concept-mime-type: application/(?<mime_type>[^ ]+)" | rex "parent request num-concepts: (?<num_concepts>\d+)" | rex "provider: (?<provider>[^ ]+) metadata length: (?<metadata_length>\d+)" | where mime_type="echo10+xml" | eval ms_total = ms_parse + ms_generate | bin metadata_length span=300 | chart median(ms_total), perc75(ms_total), perc95(ms_total) over metadata_length |
Generate Time
| UI Expand | ||
|---|---|---|
| ||
log_group="cmr-search-wl" "transform-with-strategy umm-lib:" "target-formats: [{:format :umm-json" earliest="06/05/2023:00:00:00" latest="06/05/2023:23:59:00" | rex "transform-with-strategy umm-lib: legacy/parse-concept time: (?<ms_parse>\d+) reduce w/ legacy/generate-metadata time: (?<ms_generate>\d+) concept-mime-type: application/(?<mime_type>[^ ]+)" | rex "parent request num-concepts: (?<num_concepts>\d+)" | rex "provider: (?<provider>[^ ]+) metadata length: (?<metadata_length>\d+)" | where mime_type="echo10+xml" | eval ms_total = ms_parse + ms_generate | bin metadata_length span=300 | chart median(ms_generate), perc75(ms_generate), perc95(ms_generate) over metadata_length |
Parse Time
| UI Expand | ||
|---|---|---|
| ||
log_group="cmr-search-wl" "transform-with-strategy umm-lib:" "target-formats: [{:format :umm-json" earliest="06/05/2023:00:00:00" latest="06/05/2023:23:59:00" | rex "transform-with-strategy umm-lib: legacy/parse-concept time: (?<ms_parse>\d+) reduce w/ legacy/generate-metadata time: (?<ms_generate>\d+) concept-mime-type: application/(?<mime_type>[^ ]+)" | rex "parent request num-concepts: (?<num_concepts>\d+)" | rex "provider: (?<provider>[^ ]+) metadata length: (?<metadata_length>\d+)" | where mime_type="echo10+xml" | eval ms_total = ms_parse + ms_generate | bin metadata_length span=300 | chart median(ms_parse), perc75(ms_parse), perc95(ms_parse) over metadata_length |
Overview
Content Tools