Service Experience Insights

Service Experience Insights (SEI) measures network quality as experienced from a particular place on the network. SEI agents probe targets and report what they see — latency, jitter, packet loss, and whether the target is reachable at all — so you can tell a genuinely degraded service from a complaint.

SEI has its own dashboard, and for day-to-day use that is where you work. This page is for the operator tasks that the dashboard does not cover, all of which are done from the command line.

How SEI fits together

SEI runs on the SPEKTRA Edge platform and reuses its services rather than duplicating them, which is the main thing to understand before running any command here: the resource you want may not live in the service you expect.

flowchart LR
    AG["SEI agent<br/><i>on a device or host</i>"] -->|"probes"| T["Target"]

    AG -->|"resources:<br/>Probe, ProbingTarget,<br/>ProbingDistribution"| W["watchdog.edgelq.com<br/><b>cuttle-watchdog</b>"]
    AG -->|"measurements:<br/>latency, jitter, loss"| M["monitoring.edgelq.com<br/><b>cuttle monitoring</b>"]
    IAM["iam.edgelq.com<br/><b>cuttle</b>"] -->|"users, roles,<br/>invitations"| DASH

    W --> DASH["SEI dashboard"]
    M --> DASH

The dashboard uses friendlier names than the underlying resources. When you move to the CLI, translate as follows.

The cuttle CLI

For the Service Experience Insight environment, you need to get the specialized version of the cuttle CLI from the following links:

• macOS
• Windows
• Linux

Manage Service Experience Insights

Because SEI shares components such as IAM and Monitoring, managing it means working across those services with cuttle.

Note that you need a separate cuttle tool to download: cuttle-watchdog. It uses the same configuration files as regular cuttle, so make sure you use proper context (environment and account) using regular cuttle.

Managing access privileges

Access is managed in IAM, using the mapping above. The one trap worth repeating: a dashboard “User” is an IAM role binding, not the IAM User resource of the same name.

Roles are also renamed for readability on the dashboard. For example, the privilege shown as “administrator” is services/watchdog.edgelq.com/roles/operator-admin.

Agents, targets, and probing assignments

These three resources live on the watchdog service, so they are managed with cuttle-watchdog. List them as follows.

# agents
cuttle-watchdog v1alpha2 list probes --project $PROJECT
# targets
cuttle-watchdog v1alpha2 list probing-targets --project $PROJECT
# probing assignments
cuttle-watchdog v1alpha2 list probing-distributions --project $PROJECT

Get metrics information

The metrics information (time series data) displayed on the SEI dashboard is stored on the Monitoring service, not on the Watchdog service. That is why the commands below use plain cuttle monitoring rather than cuttle-watchdog.

Below is a list of commonly used resources and metrics at SEI.

Resources

Metrics

Cheat sheet

Latency

cuttle monitoring query time-serie --project $PROJECT \
  --filter 'resource.type="watchdog.edgelq.com/probe" AND metric.type="watchdog.edgelq.com/probe/session/latency"' \
  --aggregation '{"alignmentPeriod": "5m", "perSeriesAligner": "ALIGN_MEAN", "crossSeriesReducer": "REDUCE_NONE"}' \
  --interval '{"startTime": "2023-01-01T00:00:00Z", "endTime": "2023-01-02T00:00:00Z"}'

Jitter

cuttle monitoring query time-serie --project $PROJECT \
  --filter 'resource.type="watchdog.edgelq.com/probe" AND metric.type="watchdog.edgelq.com/probe/session/jitter"' \
  --aggregation '{"alignmentPeriod": "5m", "perSeriesAligner": "ALIGN_MEAN", "crossSeriesReducer": "REDUCE_NONE"}' \
  --interval '{"startTime": "2023-01-01T00:00:00Z", "endTime": "2023-01-02T00:00:00Z"}'

Packet loss

cuttle monitoring query time-serie --project $PROJECT \
  --filter 'resource.type="watchdog.edgelq.com/probe" AND metric.type="watchdog.edgelq.com/probe/session/delivery"' \
  --aggregation '{"alignmentPeriod": "5m", "perSeriesAligner": "ALIGN_SUMMARY", "crossSeriesReducer": "REDUCE_NONE"}' \
  --interval '{"startTime": "2023-01-01T00:00:00Z", "endTime": "2023-01-02T00:00:00Z"}'

Obtaining path discovery information

Time series data for path discovery is obtained by get-hop-reports on the Watchdog service, not on the Monitoring service.

cuttle-watchdog v1alpha2 get-hop-reports probe \
  --probe projects/$PROJECT/regions/eu1/probes/$AGENT \
  --target projects/$PROJECT/probingTargets/$TARGET \
  --interval '{"startTime": "2023-01-01T00:00:00Z", "endTime": "2023-01-02T00:00:00Z"}'

Configure SEI agent delivery to SPEKTRA Edge devices

To distribute the SEI agent on an SPEKTRA Edge device, use the Distribution resource. This resource automatically creates a Pod resource for any SPEKTRA Edge device that matches the criteria; the Pod resource is the smallest unit that makes up the application and describes the container’s startup configuration.

Note that the creation of a Distribution resource requires the role applications-admin or equivalent, and cannot be created by services/watchdog.edgelq.com/roles/operator-admin alone.

Generate a shared token

Prior to the creation of a distribution resource, a token is generated to register the SEI agent on the controller. A shared token lets you register multiple agents at once. Shared tokens cannot be managed on the SEI dashboard, so the cuttle command must be used.

First, create a text file (shared_token.yaml) with the following contents. Refer to the Watchdog SDK API reference for the role of each field.

sharedToken:
  name: projects/$PROJECT/regions/us-west2/sharedTokens/test-token
  displayName: 'shared tokens for testing'
  deduplicationFieldPaths: [status.systemInfo.serialNumber]
  quota: -1
  probeTemplate:
    displayName: 'Newly registered agent ({{ probe.status.systemInfo.serialNumber }})'
    spec:
      agentType: HARDWARE_AGENT
      locationDiscovery:
        type: PREFER_PRIMARY
      contactInfo:
        email: $CONTACT_EMAIL

Next, use the following command to request the controller to create a resource

cuttle-watchdog v1alpha2 create shared-token -f shared_token.yaml

Now note the value of the SECRET field in the reply from the controller. This value is the token needed to activate the SEI agent.

Create a distribution

Next, create a Distribution to deliver the SEI agent. Save the following contents to a file (dist.yaml). Replace $TOKEN in the file with the appropriate value.

distribution:
  name: projects/$PROJECT/distributions/watchdog-agent
  displayName: Watchdog Agent
  spec:
    template:
      spec:
        compose: "version: \"3.6\"\nservices:\n watchdog:\n image: cloudwan/watchdogagent:latest\n environment:\n - API_SERVER_PORT=8889\n - REPORTING_INTERVAL=60s\n - ACTIVATION_TOKEN=$TOKEN\n - CONTROLLER_DOMAIN=stg01b.edgelq.com\n - LQD_DEVICE_CONFIG_PATH=/etc/watchdog/service-account.json\n volumes:\n # persist the watchdog data\n - /isodevice/data/watchdog:/etc/watchdog\n # mount src/modules for faster udp responder (xdp)\n - /usr/src:/usr/src\n - /lib/modules:/lib/modules\n network_mode: \"host\"\n privileged: true\n"

After saving the file, create the resource with the following command

cuttle applications create distribution -f dist.yaml

If an SPEKTRA Edge device is already registered in the Project and a Device resource exists, a Pod resource will be automatically generated when the Distribution is created. Confirm that the Pod resource has been created with the following command.

cuttle applications list pods --project $PROJECT

Related

• Monitoring — the metrics, logs, and alerting that SEI measurements feed into.
• cuttle CLI — installing and configuring the CLI these commands use.
• User accounts and access — roles and role bindings, which SEI manages through IAM.