Skip to main content
Swagger
All documentation pages

Working with Datasets

Decouple your test logic from your test data to keep your drift.yaml clean and maintainable.

Local and Remote Datasets

Datasets can be stored locally or fetched from a remote URI.

# drift.yaml
sources:
  - name: product-ds          # Source identifier (used in drift.yaml only)
    path: product.dataset.yaml # Local path
  
  - name: remote-data
    uri: https://data.internal.dev/product.dataset.yaml # Remote URI
    auth:
      username: admin
      secret: ${env:DATA_SECRET}

Best practice Using the same name for the source and the dataset can reduce confusion. For example, if your dataset is named product-data in the dataset file, you can name the source product-data as well.

Dataset Structure

Define your data objects within the data block of a drift-dataset-file.

# product.dataset.yaml
drift-dataset-file: V1
datasets:
  - name: product-data        # ← This is the ID you reference in operations
    data:
      products:
        existingProduct:
          id: 10
          type: "beverage"
          price: 10.99
          name: "cola"
          version: "1.0.0"
        newProduct:
          id: 25
          type: "food"
          price: 5.49
          name: "chips"
          version: "2.0.0"

Multiple Datasets in One File

A single dataset file can contain multiple named datasets:

drift-dataset-file: V1
datasets:
  - name: product-data
    data:
      products:
        product10:
          id: 10
          name: "cola"
  
  - name: user-data
    data:
      users:
        admin:
          id: 1
          email: "admin@example.com"

Understanding Dataset References

When referencing data in operations, use the dataset name from inside the dataset file, not the source name from drift.yaml.

# drift.yaml
sources:
  - name: my-source           # ← Source identifier (NOT used in data references)
    path: data.dataset.yaml

operations:
  createProduct:
    dataset: product-data     # ← Must match the 'name' inside data.dataset.yaml
    parameters:
      request:
        body: ${product-data:products.existingProduct} # ← Uses dataset name

If the dataset name doesn’t match, you’ll see an error like:

     ╭─[ drift.yaml:102:14 ]
     │
 102 │     dataset: product-data
     │              ──────┬─────
     │                    ╰─────── There is no dataset with name 'product-data'
─────╯

Injecting Data into Operations

Reference your data using the ${datasetName:path.to.key} syntax. Ensure you declare the dataset in the operation.

Single Object Reference

createProduct_Success:
  target: source-oas:createProduct
  dataset: product-data
  parameters:
    request:
      body: ${product-data:products.existingProduct}
  expected:
    response:
      statusCode: 201

Nested Field Reference

getProductByID:
  target: source-oas:getProductByID
  dataset: product-data
  parameters:
    path:
      id: ${product-data:products.existingProduct.id} # ← Access nested fields
  expected:
    response:
      statusCode: 200

Array References with Glob Syntax

Use the * glob to reference all items in a collection:

# Reference all product IDs
getProductByID_DoesNotExist:
  target: source-oas:getProductByID
  dataset: product-data
  parameters:
    path:
      id: ${product-data:notIn(products.*.id)} # ← Glob matches all product IDs
  expected:
    response:
      statusCode: 404

The * glob expands to all keys at that level:

  • ${product-data:products.*} → all product objects

  • ${product-data:products.*.id} → all product IDs

  • ${product-data:products.*.name} → all product names

Common Patterns

datasets:
  - name: product-data
    data:
      valid_products:
        - id: 10
          name: "cola"
        - id: 25
          name: "chips"

Reference using glob:

createProducts:
  dataset: product-data
  parameters:
    request:
      body: ${product-data:valid_products.*} # ← All items in array

Troubleshooting

Dataset Not Found

Error: There is no dataset with name 'product-data'

Solution: Ensure the dataset field in your operation matches the name field inside your dataset file, not the source name in drift.yaml.

Path Not Found

Error: ${product-data:products.nonExistent}: There is no entry with key 'nonExistent'

Solution: Verify the path exists in your dataset file using dot notation (e.g., products.existingProduct.id).

Was this page helpful?