Kiso: a foundation for complex, agentic, and reproducible experiments

Experimentation on distributed, heterogeneous computing environments—from edge devices to large-scale cloud platforms—demands orchestration technologies that are both flexible and extensible. Kiso is an open-source framework designed to provision resources and manage complex scientific workflows across the edge-to-cloud continuum. Its architecture unifies infrastructure provisioning, experiment configuration, and reproducible execution, enabling researchers to compose and monitor experiments that span geographically dispersed sites and variable network conditions. Although Kiso was conceived for workflow management—coordinating data-intensive tasks and ensuring reproducibility across dynamic infrastructures—its modular design makes it equally promising for providing reproducible environments for deploying and studying emerging agentic frameworks, where autonomous AI agents require consistent resource provisioning, cross-site communication, and result collection. We describe Kiso’s core capabilities for resource orchestration, experiment lifecycle management, and integration with containerized services, and we outline how these capabilities can support distributed multi-agent systems. In particular, we discuss how its declarative provisioning, extensible task abstractions, and built-in monitoring and output collection provide a natural foundation for experiments in which reasoning agents plan, negotiate, and adapt in real time. This study situates Kiso at the intersection of scientific workflow management and complex, agent-based computing, highlighting its potential to accelerate research on adaptive, self-organizing cyber-physical systems—an emerging frontier in complex systems science.

In recent years, there has been a proliferation of Internet of Things (IoT) devices that are essentially internet-connected computing devices located at or along the path to the network edge. They are used everywhere—in smart homes, hospitals, self-driving cars, and ecological sensors deployed in the environment (both oceanographic and terrestrial). Having computation close to where the data are produced is beneficial for a variety of reasons. It supports data privacy for applications such as federated machine learning and data reduction at source, where the edge device may have low network connectivity (can be intermittent, bandwidth-limited, or both). It is important to note that the edge is often resource- and power-limited, which makes complete end-to-end processing of data there infeasible. With the rollout of 5G mobile communications, it is now possible to obtain data from edge devices faster and with lower latency. At the same time, cloud computing adoption has dramatically increased in the past few years in both academia and industry. Cloud computing is attractive because of its benefits of increased availability, reliability, and scalability. Edge-to-cloud computing environments enable applications to leverage the edge for processing close to the data source and then offloading the main processing to the cloud and have been increasingly adopted. As a result, a significant amount of computer science research is being conducted into the various facets of this paradigm.

In order to conduct this research, especially in the area of resource provisioning, job scheduling, and fault tolerance, researchers have traditionally relied on simulation to evaluate their solutions. For more realistic experimentation and evaluation, use of a realistic testbed is important. In the past decade, researchers have gained access to a variety of national cyberinfrastructure testbeds such as Chameleon (Keahey et al., 2020) and FABRIC (Baldin et al., 2019). While FABRIC focuses on network-centric capabilities, Chameleon provides cloud resources and, more recently, edge devices known as “CHI@Edge”. However, the use of these testbeds often requires researchers to set up and configure the bare metal machines and to establish networking between the virtual machines, especially when seeking to mimic a distributed set of resources. This task is often best accomplished by a person with strong system skills as it requires good understanding of networking and general systems administration. Even then, it is not sufficient to only set up the machines and configure the networking manually between them. Such a process is often error-prone and time consuming.

A researcher, looking to explore job scheduling strategies and fault tolerance, may not necessarily possess these skills. Even if they do possess the requisite expertise, they need to spend a considerable amount of time familiarizing themselves with these testbeds and develop layers of shim code for their experiment setup, before even running a single experiment. Thus, resource provisioning and configuration need to be automated to enable easy and reproducible experimentation.

To address the above challenges, we set out to develop an experimentation platform called Kiso 1 that automates the setup and deployment of experiments and allows researchers to run complex scientific workflows across the edge-to-cloud continuum. Kiso builds on top of EnOSlib (Cherrueau et al., 2022), which is a library for building experimental frameworks on various platforms. Kiso takes care of the infrastructure provisioning, deployment, and configuration of a workflow system and the running of experiments where each experiment must execute a collection of workflows.

In this study, we describe the requirements and design goals for Kiso followed by a discussion of the high-level architecture, use of EnOSlib, and Kiso’s concepts. This is followed by a discussion of Kiso’s capabilities, including supported backend testbeds, supported deployment, support for Pegasus workflows (Deelman et al., 2015), and the experimentation layer. We then discuss the initial work for supporting agentic experiments, such as COLMENA (Lordan et al., 2024), and the challenges encountered in developing Kiso’s capabilities.

Background

The initial use-case for Kiso was to enable users to deploy edge-to-cloud workflows modeled and executed using the Pegasus Workflow Management System (Pegasus WMS) (Deelman et al., 2015) on target cyberinfrastructure (CI) testbeds—initially Chameleon and Chameleon Edge—and run experiments that involve executing a collection of workflows. Pegasus WMS allows researchers to develop, run, monitor, and debug large-scale scientific workflows. Pegasus also bridges the current CI by effectively coordinating multiple distributed resources. In Pegasus, users describe their workflows using a high-level resource-agnostic workflow description that is then mapped to target distributed resources. During this mapping process (often referred to as “planning”), Pegasus discovers input data, executables, and execution site layout to generate an executable workflow that can be executed on the target distributed resources. Pegasus WMS builds on top of HTCondor (Thain et al., 2005), specifically HTCondor DAGMan (Couvares et al., 2007), which is used to manage the workflow execution. Typically, the workflow submissions are coordinated from a workflow submission host, also known as a “workflow submit node”, that has Pegasus WMS and components of HTCondor installed.

One of the supported target execution environments for Pegasus is a HTCondor pool, whereby jobs run on condor workers are deployed on distributed compute resources managed by HTCondor. This deployment is illustrated in Figure 1. User jobs submitted by a WMS are then matched to an appropriate machine using a process called “matchmaking” (Raman et al., 1998). A key benefit of setting up a HTCondor pool across edge and cloud resources is that it does not require a shared filesystem (in contrast to traditional HPC schedulers). Additionally, HTCondor has been demonstrated to work well in distributed wide area networks (Pordes et al., 2007) and is also designed to handle network outages and job and resource failures gracefully.

Setting up any distributed compute environment in a testbed environment requires the provisioning of nodes and setting up and configuring each of them. The setup involves installing the software dependencies required for the job management system. Careful attention must be given to the networking setup to ensure that the provisioned nodes can connect to each other and also to the workflow submission node. This aspect can be particularly challenging, especially for researchers lacking systems experience.

Therefore, in the context of Pegasus workflows, deployment on a cloud testbed involves provisioning of the nodes, installing HTCondor on each of them, and configuring them to communicate with each other and the workflow submit node.

Design goals for Kiso

Based on our experience in the execution of edge-to-cloud workflows (Tanaka et al., 2022), we developed a set of design goals for experiment orchestration. Remote launch: Users should be able to install Kiso on their desktop/local system and coordinate and launch experiments on one or more of the supported cloud testbeds.

Automation of the provisioning of resources from cloud testbeds such as Chameleon and Chameleon Edge. This involves not only provisioning the virtual machines from the testbeds but also deploying the necessary software dependencies and configuration.

Reproducibility: a core design goal of Kiso. Experiments in Kiso should be transparent, repeatable, and independently reproducible. Users should describe the experimental setup programmatically, including workflow, resource roles, and execution parameters.

Layered architecture: clean separation of functionalities, with each building on top of the other. Kiso builds on top of EnOSlib (Cherrueau et al., 2022) that facilitates programmatic access to Cloud testbeds and run commands on the provisioned resources. EnOSlib in turn relies on Ansible (2025) to perform some predefined tasks (e.g., remote commands) (EnOSlib, 2025).

Support for different cloud testbeds: Kiso should be able to support multiple cloud testbeds and also support a local backend that enables users to do experiment prototyping locally using Kiso.

Support for real world edge use cases: Kiso should enable the provisioning of edge nodes/devices from the testbeds and incorporate them as worker nodes into the HTCondor pool. Kiso should support scenarios where edge devices have sensors, such as cameras that produce data that must be processed or filtered in some fashion. For example, if edge devices are scanning the environment to detect wildlife, then images where no wildlife is detected can be filtered out. The remaining images can then be transmitted to the cloud where further processing can be performed (to aggregate wildlife detection or classify animals).

Extensible experiment layer: We want to enable users to extend Kiso as they see fit. Kiso enables users to implement plugins to install new types of software, install new types of deployments, and define and run new types of experiment. To support the execution of experiments, Kiso needs to be able to

upload the files needed for an experiment;

run scripts to set up the experiments;

trigger an instance of the experiment and wait for it to finish;

run scripts to process the output generated during the experiment; and

download the files generated by the experiment.

Kiso architecture

Use of EnOSlib

During the initial design and exploration phase of Kiso, we evaluated existing frameworks for managing and automating experiments on distributed and bare-metal testbeds. We thus identified EnOSlib (Cherrueau et al., 2022), an open-source library that provides programmatic control over heterogeneous remote resources, including computers (e.g., physical servers and containers) and networking (e.g., IPv4 and IPv6 configurations).

Rather than developing bespoke resource management mechanisms, Kiso is built on top of EnOSlib, allowing us to leverage a mature and well-tested abstraction layer to orchestrate experiments. This design choice aligns with Kiso’s open-source philosophy of reusing established technologies to improve robustness, maintainability, and reproducibility. EnOSlib’s existing support for the Chameleon testbed backend was directly leveraged and extended as part of the Kiso architecture.

While EnOSlib provides a foundational abstraction for interacting with distributed resources, deploying and executing edge-to-cloud workflows using EnOSlib alone still requires researchers to write substantial amounts of custom code to configure and orchestrate experiments. EnOSlib is primarily a programmatic Python library, whereas Kiso adopts a declarative approach in which experiments are specified using YAML configuration files. This design significantly reduces the amount of user-written code, limiting it to experiment-specific logic rather than infrastructure setup and coordination.

In addition, Kiso automates the deployment of a complete workflow execution stack, including the workflow management system and its associated execution environment, provisioned from cloud-based testbeds. This automation enables the reproducible and repeatable execution of edge-to-cloud workflow experiments with minimal manual intervention. Table 1 summarizes how Kiso builds on EnOSlib and extends it with higher-level capabilities for defining, deploying, and executing experiments.

Contributions to EnOSlib

Based on our experiences developing and evaluating Kiso, we identified the limitation of being able to deploy only a single image across multiple nodes when using a single instance of EnOSlib’s Chameleon and Vagrant providers. This makes it tedious to set up a heterogeneous cluster as users would need to replicate largely identical configurations multiple times, once for each type of OS image used in the cluster. We contributed code to support provisioning nodes using different images by a single instance of EnOSlib’s Chameleon and Vagrant providers. The “Before” block of the example below illustrates that both the large and tiny VM would use the same box (OS image), whereas the “After” block of the example illustrates that users can specify the box (OS image) for each VM individually. Side-by-side comparison of YAML configuration labeled “Before” and “After.” The “Before” YAML lists one machine under “resources” with box bento/rockylinux-9, flavour large, and one machine with flavour tiny. The “After” YAML adds machine ordering, separating box, flavour, and number for both machines; the second machine’s box changes to ubuntu/trusty64.

EnOSlib also does not support provisioning nodes on the FABRIC testbed, a functionality that we needed for our work. Thus, we implemented an EnOSlib FABRIC provider to do the provisioning. We worked to support most of the FABRIC features, such as adding GPUs, NVMe storage, persistent storage, and different types of network services such as IPv4, IPv6, Layer 2, and Layer 3. The example below provisions a single VM at FABRIC’s UCSD site, with one TeslaT4 GPU, one NVMe storage mounted on / mnt/nvme, and a persistent storage mounted on / mnt/kiso-fabric-integration.

In the following sections, we describe Kiso, the underlying concepts, and its capabilities and challenges that we encountered.

Description of Kiso

To automate the setup of experiments for edge-to-cloud workflows and their deployment and execution, a user provides an experiment description file that describes the experiment they want to run. The experiment file is YAML-formatted and has been designed with extensibility in mind. The format is described later in Section 3.2.2.

Given an experiment description, Kiso does the following (also illustrated in Figure 2). Provisions resources from one or more multiple supported cloud backends.

Sets up and configures infrastructure deployment. For example, in the case of Pegasus workflows, it sets up the workflow submit node and a functional HTCondor pool overlaid atop the resources.

Runs the experiment. For workflow experiments, it usually refers to running a collection of edge-to-cloud workflows and monitoring them.

After the experiment has completed, Kiso preserves the output artifacts and downloads them from the Cloud testbed to a local directory on the user’s local system.

Deprovisions the resources requested during the setup phase.

Concepts: experiment definition file

An experiment definition file is the main file that the user provides to Kiso; it comprises four components: Sites: the infrastructure providers and locations from which compute resources are provisioned (e.g., cloud or testbed sites).

Deployment: a declarative specification of the execution environment topology, including node roles, grouping, and connectivity, and the workflow system software to be installed on the provisioned nodes defined by the deployment (e.g., HTCondor).

Software: the software stack and system capabilities to be installed on the provisioned nodes defined by the software (e.g., Docker, Apptainer, and Ollama).

Experiments: the collection of experiments to be run on the deployed and configured execution environment.

The above four components, where sites with assigned labels define the machines, and those same labels associated with deployment, software, and experiments bind those actions to specific machines. Together, these four components and their label-based binding form a single, reproducible Kiso experiment.

Sites

This section describes the sites upon which the execution environment needs to be deployed. The sites directly map to the supported backend resources (Chameleon, Chameleon Edge, Vagrant, and FABRIC). The user can specify one or more sites for their experiments. Kiso supports both virtual machines and containers. Each machine defined in the sites section is assigned one or more labels that denote the roles or services it is intended to run. These labels are used to associate machines with corresponding entries in the software section, determining which software components and configurations are installed on each machine. Screenshot of a YAML configuration file specifying three site sections: chameleon-edge with a Raspberry Pi 4–64 machine labeled execute-host-2; vagrant with a large machine labeled submit-host and execute-host-1; chameleon with a compute_zen3 machine labeled central-manager-host. Each machine section lists its labels, machine type or flavor, and the number as one.

For example, the above snippet illustrates how Kiso provisions machines from multiple supported backends and specifies the number of instances to allocate for each resource type.

Supported backends

Kiso can support a variety of popular Cloud testbeds, such as Chameleon, Chameleon Edge, and FABRIC. Because EnOSlib also supports Vagrant, Kiso can set up deployments in the local environment for easier prototyping and testing. These backends are described below. Vagrant: a command line tool for managing virtual machines on a single machine (HashiCorp, 2025). Vagrant is widely used to manage virtual machines during software development. EnOSlib and Kiso support Vagrant for the rapid development of experiments as local virtual machines can be spun up or down quickly.

Chameleon and Chameleon Edge: a large-scale, deeply reconfigurable experimental platform built to support computer science systems research, education, and emerging applications. Community projects range from developing new operating systems, virtualization methods, performance variability studies, and power management to software-defined networking, artificial intelligence, and resource management (Keahey et al., 2020). Chameleon provides bare-metal nodes and edge nodes such as Raspberry Pis and NVIDIA Jetson Nanos.

FABRIC is a national-scale, programmable research infrastructure that enables end-to-end experimentation across cloud, network, and edge environments (Baldin et al., 2019). FABRIC supports research in networking, cybersecurity, distributed computing, storage, virtual reality, 5G, machine learning, and data-intensive science. FABRIC’s core capabilities include programmable networking, heterogeneous accelerators (GPUs, FPGAs, and BlueField-3 DPUs), wide-area connectivity, and reproducible experimentation. The infrastructure spans 29 sites across the United States, each hosting extensive compute and storage resources interconnected by high-speed optical links. Selected sites provide P4-programmable Tofino switches for advanced networking research. FABRIC also extends internationally to Amsterdam, CERN, Bristol, and Tokyo, supporting global experimentation and connecting to external resources including research testbeds (CloudLab (Duplyakin et al., 2019), Chameleon (Keahey et al., 2020), COSMOS (Raychaudhuri et al., 2020), Powder (Breen et al., 2021)), national and international networks (ESnet (Energy Sciences Network, 2025), Internet2 (Internet2, 2025), AmLight, and CESNET), research universities, and scientific facilities such as NIST and CERN.

Deployment

The deployment section declaratively defines the workload management system as being instantiated on the provisioned resources and describes the logical roles that constitute its execution environment. In the case of HTCondor, the deployment specification identifies the required daemon types, their role-specific configurations, and their intended multiplicity. Daemon placement is not specified procedurally; instead, daemon roles are associated with machines through shared label keys that bind deployment roles to the machines defined in the sites section. Each daemon type may be instantiated on multiple machines, with all instances sharing the same role-specific configuration unless otherwise specified. This role-based, label-driven approach decouples workload-system topology from physical resource allocation, enabling flexible and reproducible deployments across heterogeneous testbeds. Screenshot of a YAML configuration displaying a deployment section for htcondor with three kinds: central-manager, submit, and execute. Each kind has an associated labels subsection defining roles such as central-manager-host, submit-host, and execute-host.

Kiso is designed to provide users with an execution environment in which edge devices can be readily incorporated alongside cloud and testbed resources. In this context, HTCondor is a natural choice compared to traditional HPC workload management systems such as SLURM as it does not require a shared file system and is designed to tolerate resource and job failures, network latency, and intermittent connectivity. At the same time, Kiso’s architecture is not limited to a single workload management system—the design of the deployment abstraction explicitly supports the integration of alternative systems. See Section 3.2.2 for how to add support for new workload management systems.

The combined sites and deployment examples above can be visualized as the configuration illustrated in Figure 3. In this scenario, Kiso declaratively specifies a HTCondor pool spanning heterogeneous resources, including Chameleon Cloud, a user desktop environment, and Chameleon Edge. Specifically, the configuration defines a virtual machine (virtual machine 1) provisioned on Chameleon Cloud that serves as the HTCondor central manager, hosting the daemons responsible for scheduling and pool state management;

a virtual machine (virtual machine 2) instantiated on a user desktop/laptop via Vagrant, assigned the roles of workflow submit node and HTCondor execute node, and provisioned with Pegasus and HTCondor; and

an edge device (e.g., a Raspberry Pi) provisioned through Chameleon Edge and configured as a HTCondor execute node.

Software

The software key allows users to specify which Kiso-supported software to install on the machines provisioned from the testbeds. Currently, Kiso supports the installation of Docker (Docker, 2025), Apptainer (Apptainer, 2025), and Ollama (Ollama, 2025). For Ollama, users can specify the models to be pulled after installation. YAML configuration snippet shows software categories: docker with label docker-host, apptainer with label apptainer-host, and ollama with label ollama-host and model gpt-oss:20b. Blue text highlights section headers.

Experiment types

The experiments key allows users to specify which experiments are to be executed. Kiso natively supports two types of experiment: shell and pegasus. New experiment types can be easily added (Section 3.2.2).

Shell experiment type

The shell experiment type is the most generic experiment type supported. With shell, users can sequentially run a series of scripts on the provisioned nodes. The nodes on which a script is run are determined by matching labels associated with the script and the machines defined in the sites section. Optionally, users can also specify what output files should return to the user’s desktop after the experiment. Screenshot of a YAML configuration file defining an experiment with shell scripts. It includes installation commands for Python three point thirteen and related packages, a script to run a Python file, and specifies agent-output.txt as output.

Pegasus experiment type

The pegasus experiment type supports the execution of Pegasus workflows by encapsulating common workflow management tasks, including monitoring workflow completion, invoking Pegasus analysis tools (e.g., pegasus-analyzer) upon failure, and collecting execution statistics at the end of a run. This abstraction eliminates the need for users to write custom control code. Figure 4 shows the deployment of a pegasus experiment on the testbeds.

The main key specifies the experiment script responsible for generating and submitting the workflow for execution. For Pegasus experiments, Kiso expects this script to construct the workflow and invoke the Pegasus planner with user-defined configuration options.

The poll_interval key defines the interval (in seconds) at which Kiso polls the status of a running experiment.

The timeout key specifies a maximum execution duration (in seconds), after which a running experiment is terminated.

The count key specifies the number of times an experiment is executed. This is particularly useful for running repeated trials in order to collect performance or variability metrics.

The submit_node_labels key specifies the labels identifying the nodes on which the experiment script defined by the main key is executed. Nodes referenced by submit_node_labels must be valid HTCondor submit nodes, meaning that they run an HTCondor submit daemon. The set of HTCondor daemons running on each node is defined in the htcondor specification of the deployment section. Any node running either a submit daemon (kind: submit) or a personal HTCondor instance (kind: personal) may serve as a submit node; a personal HTCondor instance corresponds to a single-node deployment running all HTCondor daemons.

The inputs key specifies files or directories to be transferred from the user’s desktop to provisioned resources prior to experiment execution. By default, the experiment directory—the directory containing Kiso’s experiment definition—is copied to all provisioned nodes. For each input specification, src denotes the file or directory on the user’s desktop, dst specifies the destination directory on the remote machine, and labels identifies the set of remote machines to which the input should be copied.

The outputs key specifies files or directories to be transferred from remote machines back to the user’s desktop after experiment execution. By default, the Pegasus workflow submit directory is copied from the remote submit node to the user’s desktop. For each output specification, src denotes the file or directory on the remote machine, dst specifies the destination directory on the user’s desktop, and labels identifies the set of remote machines from which outputs are retrieved.

The setup key specifies scripts to be executed in order to prepare the environment prior to running the Pegasus experiment. The associated labels key identifies the machines on which each setup script is executed.

The post_scripts key specifies the scripts to be executed after the experiment’s completion to prepare outputs before they are transferred back to the user’s desktop. The associated labels key identifies the machines on which each post-processing script is executed. Screenshot of a YAML configuration file defining an experiment with keys for kind, name, count, main, submit node labels, setup instructions, input and output file paths, post-execution scripts, and related commands.

Mapping software to machines

Across Kiso’s experiment definition, the labels key provides a uniform, declarative mechanism for binding actions—such as software installation, deployment configuration, and data movement—to the machines on which they are executed. In the example in Figure 5, the sites section specifies two machines to be provisioned: labeled large and tiny, respectively. The corresponding action specifications define two actions: installing Docker and configuring a personal HTCondor instance (i.e., a single-node HTCondor deployment). Kiso determines the placement of actions by matching label values assigned to machines with those specified in action definitions. In this case, the docker-host label assigned to the large machine matches the label specified in the Docker software definition, resulting in Docker being installed on the large machine. Similarly, the personal-htcondor label matches the HTCondor configuration action, causing Kiso to install and configure a personal HTCondor instance on the tiny machine.

Monitoring and output collection

Operational logging

A Kiso experiment, when run, goes through phases: provisioning, setup (installing and configuring software and deployments), a run phase to run the users’ experiments, and deprovisioning. The user invokes Kiso’s up, run, and down commands to go through the described phases. Each of the commands has a --debug flag, which captures the logs generated during the experiment phases. Apart from Kiso’s logs, this also captures logs generated by the testbed APIs used to provision resources. These logs are useful for debugging a variety of issues. For testbeds, they capture connectivity issues or intermittent internal server errors reported by the testbed APIs. At runtime, they capture issues occurring during the installation and configuration of software, such as insufficient disk space or the inability to connect to an external URL to fetch required files or packages.

Resources provisioned on Chameleon Edge require additional monitoring to track command execution and file upload/timeouts. Unlike other testbed sites that provision nodes accessible over SSH, the Chameleon Edge site provisions containers accessible only through their APIs. On virtual machines, Kiso can run commands, upload, and download files over SSH; however, to run commands, upload, and download files on Chameleon Edge containers, we had to use its APIs, the timeout behavior of which can cause the completion status to be reported incorrectly. We describe these challenges in Section 3.3.

Output collection

Running Kiso experiments may produce output artifacts, which need to be brought back from the remote provisioned nodes for preservation. Kiso supports bringing back outputs from the provisioned nodes using the outputs key. This key can be specified on both the Shell and Pegasus experiment types. Using the outputs key, the user specifies a collection of output artifacts that Kiso must bring back using the src and dst keys, and what node(s) to bring them back from using the labels key.

Pegasus workflow-specific outputs

In addition to user-specified output artifacts, a Pegasus experiment automatically retrieves Pegasus-specific artifacts from the remote submit node. These artifacts include the Pegasus submit directory, which holds the executable workflow generated from the abstract workflow by the Pegasus planner, along with associated execution metadata. Upon an experiment’s completion, Kiso automatically invokes two Pegasus tools. First, it runs pegasus-statistics to compute workflow performance metrics, including the wall time—the total elapsed time of the workflow as observed by the user—and the cumulative wall time—the sum of the individual run times of all jobs in the workflow. Second, it runs pegasus-analyzer to determine the outcome of the workflow execution. If the workflow succeeded, the analyzer confirms successful completion. If the workflow failed, the analyzer identifies which jobs failed, captures their standard output and standard error, and highlights relevant Pegasus files for further investigation. The results of both tools are written into the submit directory, which is brought back to the user’s desktop as part of the output collection process.

Security and allocation usage

Kiso provisions ephemeral environments specifically for experiment execution rather than maintaining persistent infrastructure, thereby reducing overall security exposure. Resources are instantiated according to user-defined specifications, and the responsibility for configuring and enforcing security policies remains with the user.

Kiso primarily targets academic environments that operate under allocation-based access models rather than direct monetary billing. It currently supports platforms such as FABRIC and Chameleon, both of which require users to specify a wall time for resource reservations. If no walltime is explicitly provided, Kiso defaults to a one-day allocation. When the walltime is exceeded, the testbed terminates the resources, though how quickly this happens would vary across testbeds. Consequently, resource usage is bounded and controlled by the walltime defined within the experiment specification itself.

Pegasus experiment examples

We briefly describe two edge-to-cloud Pegasus experiment examples in Kiso that leverage already developed Pegasus workflows. The first example runs a workflow that does plankton classification (Kyathanahally et al., 2021), while the other experiment runs the Orcasound workflow (Tanaka et al., 2022) that identifies the whistles of the Orca whales from a set of hydrophone data. Please refer to data availability in Section 4.3 for links to the various GitHub repositories containing the experiment descriptions.

Plankifier

Plankton are effective indicators of environmental change and ecosystem health in freshwater habitats, but collecting plankton data using manual microscopic methods is extremely labor-intensive and expensive. Automated plankton imaging offers a promising way forward to monitor plankton communities with high frequency and accuracy in real time. However, the manual annotation of millions of images poses a serious challenge to taxonomists. Deep learning classifiers have been successfully applied here to categorize marine plankton images. The Kiso experiment runs a plankton classification workflow (Kyathanahally et al., 2021).

Orcasound

This Orcasound workflow is based on an open-source software and hardware project that trains itself using the audio files generated through hydrophone sensors deployed in three locations in the state of Washington (San Juan Island, Point Bush, and Port Townsend) in order to automatically identify the whistles of the Orca whales in the Pacific Northwest. This workflow uses code and ideas available in the Orcasound GitHub actions workflow 2 and the Orca Hello real-time notification system 3.

The Pegasus version of the workflow (Tanaka et al., 2022) takes in the hydrophone data of one or more sensors in batches for each timestamp and converts them into a WAV format. The WAV output is converted into spectrogram images that are stored in the final output location. The workflow then performs inference against the Orcasound model to identify potential sounds produced by orcas. These predictions are finally merged into a JSON file for each sensor. The Kiso experiment is set up to run this Orcasound workflow.

Extending Kiso with new software, deployments, and experiments types

Kiso is designed to be extensible, allowing users to introduce new software, deployment, and experiment types through a plugin-based mechanism. Software plugins enable the integration of new software components; deployment plugins enable the integration of additional workload management systems, and experiment plugins facilitate the incorporation of new experiment types with different lifecycles.

If needed, Kiso’s architecture can be extended to support additional plugins that are invoked at specific stages of the Kiso lifecycle—for example, before and/or after provisioning, before and/or after software installation, before and/or after deployments, or before and/or after executing experiments.

All plugins operate within the Kiso environment and therefore have access to all provisioned nodes. This access enables them to perform a wide range of configuration and orchestration tasks as required. For example, the built-in HTCondor deployment plugin determines whether a cluster is being deployed on a single site or across multiple sites. For multi-site deployments, it identifies the nodes that require publicly accessible IP addresses, such as the central manager and submit nodes, and automatically provisions public IPs for those nodes to ensure correct cluster configuration and operation.

Extensions follow a common pattern across all three categories and are implemented by providing a Python module that exposes a small set of well-defined interfaces. Specifically, users must do the following. Specify a Python project entry point 4 that references the extension’s Python class. The entry point group name should be “kiso.software” for software plugins, “kiso.deployment” for deployment plugins, and “kiso.experiment” for experiment plugins. The class should define four to five attributes described below.

Define a schema class attribute containing a Python dictionary representing a JSON Schema used to validate the corresponding YAML block from the Kiso experiment definition.

Define a config_type class attribute that defines a native Python dataclass 5 to store the data from the corresponding YAML block from the Kiso experiment definition.

Define a check instance attribute that defines a Python method to check the configuration which cannot be checked with the JSON Schema.

Define a __call__ instance attribute defining a Python method that installs the software or deployment or runs the experiment.

Define a kind class attribute (only for experiment plugins) defining a Python string representing the name of the experiment type.

Challenges: Chameleon Edge

To install software and execute and manage experiments on provisioned resources, Kiso must be able to execute commands and transfer files to and from the provisioned resources. EnOSlib provides this functionality through Ansible, which relies on SSH to execute commands and copy files. However, resources provisioned through Chameleon Edge are deployed as containers without SSH access; interaction with these containers is possible only through the Chameleon Edge APIs.

To accommodate this heterogeneity, Kiso adapts its execution mechanism based on the resource backend associated with each action. When an action targets resources provisioned on Chameleon Edge, Kiso invokes the Chameleon Edge Python API to execute commands and manage data movement. For all other backends, Kiso leverages EnOSlib’s existing SSH-based utilities.

In addition, to the absence of SSH support, we encountered additional challenges when using Chameleon Edge’s API to run the actions on containers, which are described below.

Command execution timeouts

Chameleon Edge APIs have timeouts, which expire when operations take too long to complete. However, it does not expose a mechanism for overriding timeouts. When the API used to execute the command times out, the API call terminates, but the command invoked by the API continues to run on the remote container. Kiso addresses this case. When executing actions on Chameleon Edge, Kiso invokes commands on remote containers using the Chameleon Edge API, then periodically polls for their completion. Kiso does so by modifying the original command, adding a step to save the exit code in a temporary file at the end of the command. If the expected temporary file exists, the invoked command is deemed to be complete; otherwise, Kiso continues polling. Once the file is detected, Kiso retrieves the recorded exit code and uses it to display the status of the experiments.

File upload/download timeouts

To upload a file or directory, Chameleon Edge APIs first compresses it before uploading it to the remote container where it is uncompressed at the destination directory. Similarly, during the download operation, the source file or directory is compressed and copied to the user’s desktop, where it is uncompressed at the destination directory. The upload/download operations can time out when the directory being uploaded or downloaded is large. Kiso addresses this limitation by first attempting to transfer the entire source directory in single operation. If the transfer times out, Kiso falls back to iterating over the directory contents and transferring files individually, thereby avoiding repeated timeouts for large data transfers.

Challenges: FABRIC

Persistence of network configuration across reboots

FABRIC’s Python API provides comprehensive methods for provisioning and configuring FABRIC slices, including network configuration across provisioned nodes and IP address assignment. However, network settings configured programmatically via the FABRIC API do not persist across system reboots, requiring manual reconfiguration after each restart—a significant impediment to running long-duration experiments or experiments requiring node reboots.

To address this challenge, we developed automated configuration scripts that ensure that network settings persist across reboots. We developed complementary solutions for both RHEL-based systems using NetworkManager (NetworkManager Project, 2025) and Debian/Ubuntu systems using Netplan (Canonical Ltd, 2025). These scripts support FABRIC’s diverse network service types (FABRIC Testbed, 2025), such as FABNet v4 (a private network with IPv4) and Level 2 Site-to-Site (L2STS), and handle multi-homed configurations where nodes maintain multiple active network interfaces simultaneously.

The scripts are idempotent and generate persistent OS-level configurations that survive system reboots. This solution enables Kiso to fully automate FABRIC network configuration as part of the deployment process, eliminating manual post-reboot reconfiguration and supporting reliable long-running experiments across distributed infrastructure.

Use case: COLMENA experiment

Although initially developed to support Pegasus WMS experiments, Kiso is a flexible and extensible system that can support different types of experiments. Here, we describe how the agent-based framework COLMENA can leverage the experimental capabilities of Kiso.

Background

COLMENA is an open framework designed to simplify the development, deployment, and operation of hyper-distributed applications across the compute continuum (Lordan et al., 2024; 2026). It enables collections of heterogeneous devices to collaborate as a decentralized swarm of agents, presenting the infrastructure as a single, unified computing platform. Each device hosts a software component called a COLMENA Agent, which transforms the device into an autonomous, self-aware entity. Applications are expressed using the COLMENA programming model as a composite of infrastructure-agnostic, interacting roles that implement the different functionalities of the service.

Agents dynamically decide which roles to execute by evaluating their capabilities, resource availability and context against application requirements and user-defined key performance indicators (KPIs) for quality of service (QoS), quality of experience (QoE), and resource efficiency. To achieve the desired collective behavior, these decisions can be taken fully autonomously or coordinated among multiple devices—for instance, through consensus mechanisms. By enabling autonomous role allocation and adaptation, COLMENA improves scalability, resilience, and energy efficiency.

In this study, we present the deployment of a COLMENA experiment with Kiso in the context of modern power grid systems. The electrical grid is undergoing a fundamental transformation driven by the large-scale integration of renewable energy sources such as solar and wind. The application is defined by roles that react to contingencies (such as line disconnections, load changes, or the introduction/removal of generation sources) to monitor grid status. The application defines the normalized grid frequency as a KPI, which must be maintained within a tight range around its nominal value (± 0.001). Whenever the frequency falls outside this range, agents self-organize to select a device in each area to execute a frequency optimizer role. The self-organization is achieved using an extension of the consensus mechanism developed by Thareja et al. (2025).

As a first approximation, we are using ANDES (Cui et al., 2020), a widely used library/toolbox for the simulation of power systems. In particular, we simulate the NPCC 140-bus network, a reduced model for the power grid of the northeastern United States and Canada. We deploy this use case using FABRIC sites, hosting the (centralized) simulation in one of the nodes, and mapping each electrical grid area to a FABRIC site. COLMENA roles interact with the simulation by sending HTTP requests to the centralized node hosting the simulation. We define three COLMENA roles: the SimulationManager advances the timesteps of the simulation continuously, the Monitoring role fetches the generator frequencies and publishes them in COLMENA as metrics, and the Optimizer role reacts to contingencies, coordinates with other optimizers to find a global solution, and changes simulation parameters in ANDES.

Motivation for using Kiso with COLMENA

COLMENA separates the definition of agents and their communication and sharing mechanisms from the definition and deployment of services. This separation enables agents to dynamically assume different roles based on their capabilities and system requirements. Services are defined using the programming model and then packaged into a bundle containing Docker images files (for running the roles) and a service description in JSON format. Agents incorporate and monitor a service upon receiving its service description, evaluating relevant KPIs to decide whether to start or stop specific roles. When an agent decides to start a role, it pulls the corresponding Docker image and executes it.

The deployment of COLMENA and its services creates a highly distributed and dynamic environment in which agents autonomously execute and terminate roles according to the QoS of the deployed services. This setup also opens the opportunity of investigating complex decision-making policies that lead to sophisticated collective behaviors. However, deploying COLMENA involves multiple steps that must be executed and verified before proceeding to the next. When scaling to a large number of agents, it is essential to automate this process and use tools that can monitor and ensure correct deployment at each stage.

Kiso is a useful tool for COLMENA for several reasons. First, it automates resource allocation across collaborative testbeds such as FABRIC, including networking and configuration between sites. Second, it enables fully decentralized deployment, allowing devices to locally deploy agents and application-specific processes. Third, it provides mechanisms to ensure robust service execution, such as locally verifying that all deployment steps are completed before advancing. Finally, it can aggregate results from multiple devices into a centralized location for analysis.

Use of Kiso

We used Kiso to deploy COLMENA and the proposed use case across six FABRIC sites. Networking is configured using FABRIC’s FABNetv4, consisting of Layer 3 links between the sites. A detailed description of these FABRIC networking mechanisms is provided in Ruth et al. (2022). All nodes were included in the host file of each machine. COLMENA uses Zenoh (Eclipse Foundation, 2022) to establish a peer-to-peer communication platform within agents, and its topology can be adapted with a configuration file. Zenoh is used for exchanging data and metrics, which are used for KPI evaluation. In this experiment, each agent ran a router, and it was configured such that a single COLMENA agent per site acts as a backbone router, handling inter-site traffic, while the remaining agents connect to this local backbone. The backbone routers themselves are interconnected in a ring topology, although this structure can be easily modified through the Zenoh configuration file, which is part of COLMENA’s setup.

Listing 1 describes the node configuration for the UCSD site. The first node hosts the ANDES simulation (simulation) and the Redis database (redis), which is required for the consensus policy. The second node runs the COLMENA agent (colmena) and the consensus component (consensus). In addition, this node triggers the experiment by building role images and distributing the service code to its peers (trigger). It also collects KPI values and stores them in a log file (kpis). The remaining nine nodes run colmena and consensus.

The other five sites deploy the rest of the nodes, all labeled colmena and consensus.

Listing 1: FABRIC UCSD site

Each label is defined by a sequence of shell commands and includes scripts that execute the corresponding processes and ensure their correct operation by waiting for specific text to appear in Docker logs and terminal output.

Finally, outputs are collected from all agents, as shown in Listing 2. These include simulation plots, KPI values over time, and agent timestamps recorded at key events (e.g., when a KPI threshold is violated, when a role starts, and when the KPI is satisfied again).

Listing 2: output collection configuration

Benefits of using Kiso for COLMENA

Using Kiso to deploy the COLMENA experiment provides simplicity, as users are not required to learn the fablib API (provided by FABRIC). Experiments can be executed repeatedly to collect statistics, with execution times shown in Table 2 ranging from 30 min to 1 h 30 min, depending on the number of agents per site. In addition to experiment execution time, the table also reports the time required to allocate and configure resources. The number of agents per site was varied from three to five and ten, with all configurations deployed across six sites. Each experiment was repeated five times. The full process took up to 32 h 18 min. This makes Kiso particularly advantageous, as it enables experiments to be deployed and repeated automatically with minimal user intervention, making it practical to conduct large experimental campaigns unattended, including overnight runs.

The first execution of the experiment introduces additional overhead due to software downloads, including Git repositories and container images from Docker Hub. This initial overhead amounts to 16.1, 20.1, and 28 min for the respective configurations. Overall, the experiment allocates 18–60 nodes, illustrating a typical scenario that Kiso can handle automatically with minimal modifications. Additionally, Kiso supports combining nodes from different testbeds, which was not explored here but represents a potential advantage.

The analogous Jupyter notebook previously used for this experiment consists of 278 lines of Python code, encompassing resource allocation and configuration, experiment deployment, and result retrieval. Although parallel execution is supported by the fablib API, it must be explicitly managed by the user, typically requiring centralized control logic and iteration over the allocated nodes. In contrast, Kiso performs these steps in a decentralized manner on each machine. The Kiso experiment file contains 206 lines of YAML, providing a more compact and simpler specification. In both approaches, the same execution scripts are used, and therefore their lines are excluded from this comparison.

Kiso evaluation

In this section, we share results from experiments that used Kiso for provisioning and configuring a HTCondor pool, which is a preferred deployment for running edge-to-cloud workflows using Pegasus. In the first experiment, we provision a HTCondor pool from the FABRIC testbed using Kiso. The second experiment tries to mimic a real world Jupyter notebook that was developed by students as part of a summer internship program; it provisioned and ran the plankton classification workflow described earlier in Section 3.2.1 and qualitatively highlights the benefit of using Kiso if it had been available. Both experiments and resulting logs are available in the GitHub Repository 6.

Time to setup a HTCondor cluster of varying sizes

In this section, we describe the performance numbers collected using Kiso. We created four experiments that provision and configure 5-, 10-, 15-, and 30-node HTCondor clusters. Each cluster has one central manager node, one submit node, and the rest are execute nodes. In the experiment, nodes were provisioned on the FABRIC testbed across three sites: WASH, UTAH, and UCSD. The central manager node was provisioned on the WASH site, the submit node was provisioned on the UTAH site, and the worker nodes were provisioned on the UCSD site. We configured a FABRIC FABNet IPv4 network on all nodes to facilitate the communication needed by the HTCondor cluster. We ran these experiments ten times and from an Apple Mac Mini with M2 Pro, 12 cores, and 32 GB RAM running macOS Tahoe. In Table 3, we show the minimum, maximum, and average time to provision 5-, 10-, 15-, and 30-node HTCondor cluster.

The above numbers highlight that Kiso is able to correctly provision nodes from various FABRIC sites of increasing sizes and install/configure and launch the different HTCondor daemons to bring up a functioning HTCondor Pool. We verify in the experiment that the HTCondor pool is correctly set up by running the commands condor_q and condor_status to verify that the job queue is up, and the pool reports the designated number of worker nodes.

Benefits of using Kiso for reproducible workflow experimentation

We ran the Plankifier experiment described in Section 3.2.1 using a Jupyter notebook that was developed for the summer internship program as a reference notebook to test feasibility and correctness before students started their project and as a Kiso experiment. Similar Jupyter Notebooks were then created by students as part of a summer internship (Choudhary et al., 2022) over a span of 5 weeks.

The Plankifier experiment provisions and configures a two-node HTCondor cluster. A compute_zen3 node is provisioned on the Chameleon’s TACC site representing the cloud site, and a raspberrypi4-64 node is provisioned on Chameleon’s Edge site representing the edge site. The cloud node is configured to be the HTCondor central manager, submit, and execute node, while the edge node is configured to be an HTCondor execute node. The workflow was run ten times and the minimum, maximum, and average runtime in minutes are shown in the Tables 5, 6.

We ran the Kiso experiment by installing Kiso on an Apple Mac Mini with M2 Pro, 12 cores, and 32 GB RAM running macOS Tahoe, whereas we ran the Jupyter Notebook experiment from ChameleonCloud’s Jupyter interface.

Note: A difference between the experiment run using the Jupyter notebook and Kiso was the Docker container used to run the job. The container used by the Jupyter notebook is (pegasus/condor8-arm64-plankifier-worker) and the container used in the Kiso experiment is (pegasus/plankifier). The Table 4 explains the minor runtime differences in the time to set up the experiment. It is largely attributable to the container used, as the container used in the notebook has HTCondor installed whereas that used by Kiso does not; it is installed by Kiso at the time of provisioning.

To develop the notebook the students needed to learn Bash to develop scripts to install and configure software such as Pegasus and HTCondor;

learn Python to provision resources on Chameleon using their Python API and to develop the Pegasus workflow;

learn Pegasus WMS’ Python API to develop the Plankifier workflow;

learn to install, configure, and test a complex HTCondor cluster;

learn Chameleon APIs to provision Chameleon resources, run commands on the provisioned nodes to setup, and configure them to run the experiment.

If the same task were to be achieved with Kiso, the task of developing the experiment would have been greatly simplified as the students would have only needed to learn minimal Bash to trigger the workflow, and not for installing and configuring Pegasus, HTCondor, etc., which is just “./workflow.py”;

learn Pegasus WMS’ Python API to develop the Plankifier workflow;

learn basics on HTCondor to define the < 15 lines of Kiso’s HTCondor configuration. Students would not need to install, configure, and test the cluster.

If Kiso were available, the students would have developed this experiment much faster and with more ease as they would not have had to worry about learning how to provision resources against Chameleon and then to deploy and configure a HTCondor pool to run the workflow on. Moreover, it would have allowed the students to spend more time on higher level concepts and the science itself.

The Tables 5, 6 show the minimum, maximum, and average time to provision nodes on Chameleon and Chameleon Edge, and the minimum, maximum, and average time it took to run the Plankifier workflow.

Related work

Overall, the functionality that Kiso provides straddles three traditionally distinct classes of tools. Infrastructure as code (IaC), such as Terraform (2025) and Pulumi (2025).

Configuration management, such as Salt Project (2025) and Ansible (2025).

An experiment execution engine, such as Popper (Jimenez et al., 2017).

In the following section, we compare representative tools in each category to clarify Kiso’s design goals and positioning.

Infrastructure as code

Terraform (2025) is a widely adopted IaC tool that provisions resources across major commercial cloud providers. It is primarily designed for deploying long-running stateful or stateless production services. However, Terraform provides limited native support for academic testbeds. Such testbeds often expose their resources through Python SDKs or REST APIs and have not prioritized Terraform integration.

In contrast, Kiso prioritizes support for academic testbeds. Kiso uses EnOSlib for provisioning. Kiso’s provisioning model is optimized for reproducible, short-lived, and largely stateless experimental workflows rather than for persistent production services. Any configuration changes made to provisioned infrastructure are treated as part of the experiment lifecycle; once the experiment concludes, those changes are not preserved.

By comparison, infrastructure-as-code tools such as Terraform are primarily intended to provision and manage machines and services for production environments. As a result, they must ensure strong guarantees around state preservation and consistency during both provisioning and deprovisioning operations.

This distinction reflects a fundamental difference in purpose: Kiso prioritizes experimental reproducibility and lifecycle automation whereas tools such as Terraform focus on robust, long-term production infrastructure management.

Terraform is designed for long-term infrastructure management and therefore maintains a state file to track provisioned resources. The desired infrastructure is declared in HashiCorp Configuration Language (HCL), while the actual deployed state is recorded in a Terraform state file, which may be stored locally in Amazon S3 or other remote backends. During execution, Terraform compares the declared configuration with the recorded state to compute the minimal set of changes required to reconcile the actual infrastructure with the desired state.

In contrast, Kiso (via EnOSlib) provides only limited support for incremental infrastructure modification. For example, it can provision newly requested resources but may not fully account for resources removed from the site specification. In such cases, unused resources are not actively reconciled during execution; instead, they are released either when Kiso explicitly deprovisions the environment at the end of the experiment or when the underlying resource reservation expires, whichever occurs first.

Configuration management

Ansible (2025) and Salt Project (2025) are well-established configuration management systems aimed at automating the deployment, configuration, and maintenance of complex IT infrastructures. These tools focus on system administration and operational consistency at scale.

Kiso, by comparison, provides declarative experiment orchestration, role-based deployment, and repeatable execution semantics tailored to research workflows. It is built atop EnOSlib, which itself leverages Ansible’s Python libraries. Kiso supports file transfers and remote command execution on provisioned nodes through EnOSlib’s abstractions over Ansible’s core capabilities. These abstractions handle creating Ansible inventory, playbooks, and task files. Importantly, Kiso does not seek to replace configuration management systems; rather, it builds upon them to provide higher-level experiment-oriented abstractions.

Ansible can be used directly to configure Kiso provisioned resources, but it would require knowledge of IT systems, Linux, Ansible, and other factors. Users would also need to create required Ansible-specific YAML files, such as the inventory file describing all the hosts and groups and playbooks describing the tasks to execute.

Experiment execution

Kiso supports two experiment types: shell and Pegasus.

The shell experiment type bears some resemblance to Popper. However, important differences exist. Popper defines workflows in which each step is encapsulated within a Docker container and executed locally on the node where Popper is invoked. In contrast, Kiso’s shell experiments execute arbitrary commands (e.g., Bash and Python) directly on one or more provisioned remote resources. Execution therefore occurs across distributed infrastructure rather than on a single local machine. More broadly, Kiso enables the reproducible deployment and execution of distributed experiments by provisioning and orchestrating multi-site, edge-to-cloud infrastructures and managing full experiment lifecycles.

The Pegasus experiment type provides native support for Pegasus workflows. Coupled with Kiso’s support for HTCondor as a supported deployment environment, Kiso automates the challenging work of setting up a functioning HTCondor pool across testbeds for a user. Given a user-provided workflow generator, Kiso can automatically plan, execute, monitor, await completion, compute statistics, and analyze failures using Pegasus tooling—without requiring manual user intervention.

It should be noted that Kiso experiment description is not intended to describe the workflows that a user intends to run. Pegasus workflows are specified in a custom YAML-formatted description, a representation that predates but serves a similar purpose to the Common Workflow Language (CWL) (Crusoe et al., 2022): defining portable computational graphs that can execute across environments ranging from laptops to high-performance computing clusters.

Overall, Kiso’s experiment types are not intended to replace workflow languages or workflow management systems. Rather, Kiso simplifies and automates the provisioning, deployment, and execution processes required to run such workflows reproducibly on distributed experimental infrastructures.

Feature comparison

In Table 7, we compare features of Kiso with the combination of Terraform and a workflow management system, and Popper.

Conclusions

In this study, we presented Kiso, a framework designed to support research across the edge-to-cloud continuum by enabling the reproducible deployment of realistic edge-to-cloud workflows on heterogeneous testbeds. Kiso builds on EnOSlib and augments it with higher-level abstractions for experiment specification, deployment automation, and execution management. In addition, we contributed upstream enhancements to EnOSlib, including native support for the FABRIC testbed.

Reproducibility has been a core design objective from the outset. The framework is designed to make edge-to-cloud experiments transparent, repeatable, and independently reproducible. Kiso employs a declarative experiment specification that explicitly captures all aspects of an experiment, including workflow structure, resource roles, and execution parameters. This declarative representation enables experiment configurations to be version-controlled, portable across infrastructures, and re-executed without ambiguity.

All experiment nodes execute within containerized environments that define the complete software stack. Kiso installs a small set of additional tools on top of these containers, with both the base container images and tool configuration explicitly specified as part of the experiment definition. This approach minimizes hidden dependencies and ensures consistent and reproducible execution across sites.

Kiso experiments run on open, community-supported platforms. These platforms are broadly accessible, allowing other researchers to reproduce experiments using their own accounts and credentials without requiring specialized or private infrastructure.

Together, these features make Kiso experiments easy to publish, inspect, and reproduce. Complete experiment specifications and artifacts can be shared through public repositories, enabling independent validation and extension by the edge-to-cloud research community.

As the development of Kiso matured, we recognized that its capabilities extend beyond workflow deployment to support a broader class of experimental applications on cloud and testbed infrastructures. The COLMENA use case demonstrates how Kiso can be leveraged to deploy and manage distributed multi-agent systems, highlighting its applicability beyond traditional workflow-oriented experiments.

Overall, Kiso looks to address an important gap in computer science research, whereby there are not enough edge-to-cloud workflows available to researchers. Our hope is that Kiso will serve as a foundation for domain researchers to develop edge-to-cloud workflows that can be made available for computer science research.

Future work

Future research will extend Kiso along several complementary directions. First, we plan to add support for additional testbeds, such as the Platforms for Advanced Wireless Research (PAWR), enabling experiments to incorporate a broader range of edge devices and to span wider geographic regions. Second, we aim to integrate additional workload management systems, including SLURM, to broaden the class of execution environments supported by Kiso.

Finally, we plan to introduce additional experiment types that support the systematic deployment and evaluation of agent-based and autonomous systems, such as COLMENA. Rather than defining new agentic frameworks, these experiment types will focus on orchestrating and comparing experiments across heterogeneous infrastructures—for example, by deploying agents with different configurations or models, coordinating controlled inputs, and collecting comparable performance and behavior metrics across runs.