kolena.workflow

Legacy Warning

Content in this section reflects outdated practices or deprecated features. It's recommended to avoid using these in new developments.

While existing implementations using these features will continue to receive support, we strongly advise adopting the latest standards and tools for new projects to ensure optimal performance and compatibility. For more information and up-to-date practices, please refer to our newest documentation at docs.kolena.io.

• Developer Guide: Building a Workflow ↗
• Examples: kolena/examples ↗

kolena.workflow contains the definitions to build a workflow:

1. Design data types, including any annotations or assets:

   Defining a workflow

   TestSample, GroundTruth, and Inference can be thought of as the data model, or schema, for a workflow.

   Learn more ↗

   • TestSample: model inputs, e.g. images, videos, documents
   • GroundTruth: expected model outputs
   • Inference: real model outputs

2. Define metrics and how they are computed:

   • Evaluator: metrics computation engine

3. Create tests:

   Managing tests

   See the test case and test suite developer guide for an introduction to the test case and test suite concept.

   • TestCase: a test dataset, or a slice thereof
   • TestSuite: a collection of test cases

4. Test models:

   • Model: descriptor for a model
   • test: interface to run tests