Concepts

Vitalsigns is a multi-tenant system. Each tenant within a Vitalsigns installation resides in a logical construct known as an application. Each application consists of a set organizational hierarchy, metrics, and data mappings that are used to populate the metrics and organization from the inputs to the system.

Organizational Hierarchy

The organizational hierarchy consists of two sets of definitions. The first set is meta data defining the base groups. Base groups is a flat list of the types of things tracked in the hierarchy. The meta data consists of the label for the group type, and data related to its processing. Note these definitions are not for the instances of a group type, but rather the general categories of things to track such as Agents, Campaigns, Call Centers, etc. The Actual instances (agent xyz) are discovered as part of the input processing based on the mapping definitions and the incoming data.

The second part to the organizational hierarchy is the actual arrangement of the base groups into a multi level hierarchy. Any given base type can appear multiple times in the hierarchy structure as needed.

Metrics

Metrics are the measurements derived from the input data. Metrics fall into three broad categories, static loaded, discovered loaded and calculated.

Static loaded metrics are directly mapped and derived from fields in the input. These are items like counts of the records meeting a condition, sums of value fields, or sums of durations.

Discovered loaded metrics are metrics where the input data contains or derives the name of the metric. As an example, call dispositions are generally discovered metrics where the disposition code is turned into meaningful text via reference data. That text is then used as the metric name, and a count of all calls with that disposition is maintained. As long as the reference data is properly maintained, new dispositions will automatically show up in Vitalsigns as new metrics with no additional work or mapping.

Calculated metrics are derived at run time using a formula with inputs being any other metric including other calculated metrics.

Detail Data

Detail Data is an optional ability to save the raw data from the inputs and make it available for searching within the system. The inputs can be selectively mapped such that not every field is saved.

Data Inputs

There are two basic types of input to Vitalsigns, reference data and activity data. Reference data is generally used to augment the activity data with additional information, whereas the activity data is used to populate the measurements in the system. As an example, the activity data in CCOps generally only includes an agents id. The CCOps application uses reference data to derive the name, supervisor and callcenter of the agent.

Data Inputs consist of files in either a basic JSON or delimited file format. The layout of inputs is defined to the system and these definitions may be shared across applications. The layout definition consists of:

  1. Basic information about the input such as format, delimiter for delimited files, records to skip (i.e. header records), etc.
  2. Data type and location information for each field in an input record. For delimited inputs location is the 1 based offset from the beginning of a row, for JSON it is the name of the field. Note only fields that will be used MUST be defined, although it is normal to define every field.
  3. Derived values. Derived values can be lookups into reference data, mathematical calculations, conditional expressions, date manipulations and other values derived from the base fields of a record.
  4. Conditional filter definitions used in the mapping process to apply conditional processing of records or fields.

Both types of inputs require a layout to be defined.

Data Mapping

The data mapping process is the step that ties everything together and produces a working application. Reference data inputs are mapped to the reference tables they populate and the activity data inputs are mapped to the groups,metrics, and detail data that they populate.

Reference data mapping is simply specifying the reference tables that an input populates, and for each, which fields in the input represent the key and return values.

Activity data mapping is a bit more complex as it involves mappings for base data group population, static metric calculation, discovered metric calculation and detail data population.

For each base group that applies to an input, a field is mapped from the input to the defined base group type. If the base group is identified by an external id, then a field to populate that is mapped as well.

For each statically mapped metric, there is a mapping that specifies the metric id, the input field for its value, and the type aggregation to be performed during processing (sum,max,min,etc).

Discovered metric mapping is similar to statically mapped metrics, except in place of the metric id, a field for the metric name is specified. The input system will then look up that name in the list of existing metrics to find the metric id, or create a new metric automatically if none are found.

Once mapping is complete, add the location to look for the inputs into the input directories, and load data.

Input Processing

The processing of activity inputs is done on a record by record basis. For each record in the input the following occurs:

  1. Base groups are resolved - for each group defined in the mapping, the field value is used to see if a group of the defined type with that name exists. If it does, then the data is associated to the existing group. If the group is new, a new group is created.
  2. Group hierarchy is validated - after all base groups are resolved, the set of base groups is then slotted into the group hierarchy and any changes/updates to the relationships are resolved.
  3. Detail data is saved to the mongodb database
  4. Metrics are aggregated - for each of the static or discovered metrics aggregates by the unique combination of groups are updated. Aggregates are calulcated to both the daily and internal levels.

Once all records in an input have been processed, the aggregates are saved to the mongodb database. Then if a subset of the base groups has been set to be aggregated further, that aggregation is done.

Summary

In summary, a Vitalsigns application consists of defining meta data for the Organizational Hierarchy, metrics, and saved transaction detail. Inputs are mapped into this structure, data is processed based on those definitions, and the result is an application that can be accessed through the Web application.