kolena.workflow.GroundTruth

Legacy Warning

Content in this section reflects outdated practices or deprecated features. It's recommended to avoid using these in new developments.

While existing implementations using these features will continue to receive support, we strongly advise adopting the latest standards and tools for new projects to ensure optimal performance and compatibility. For more information and up-to-date practices, please refer to our newest documentation at docs.kolena.io.

The ground truth associated with a TestSample. Typically, a ground truth will represent the expected output of a model when given a test sample and will be manually annotated by a human.

from dataclasses import dataclass
from typing import List

from kolena.workflow import GroundTruth
from kolena.workflow.annotation import Polyline, SegmentationMask

@dataclass(frozen=True)
class AvGroundTruth(GroundTruth):
    road_area: SegmentationMask
    lane_boundaries: List[Polyline]
    visibility_score: int

A TestCase holds a list of test samples (model inputs) paired with ground truths (expected outputs).

GroundTruth

Bases: DataObject

The ground truth against which a model is evaluated.

A test case contains one or more TestSample objects each paired with a ground truth object. During evaluation, these test samples, ground truths, and your model's inferences are provided to the Evaluator implementation.

This object may contain any combination of scalars (e.g. str, float), Annotation objects, or lists of these objects.

For Composite, each object can contain multiple basic test sample elements. To associate a set of attributes and/or annotations as the ground truth to a target test sample element, declare annotations by extending DataObject and use the same attribute name as used in the Composite test sample.

Continue with the example given in Composite, where the FacePairSample test sample type is defined using a pair of images under the source and target members, we can design a corresponding ground truth type with image-level annotations defined in the FaceRegion object:

from dataclasses import dataclass

from kolena.workflow import DataObject, GroundTruth
from kolena.workflow.annotation import BoundingBox, Keypoints

@dataclass(frozen=True)
class FaceRegion(DataObject):
    bounding_box: BoundingBox
    keypoints: Keypoints

@dataclass(frozen=True)
class FacePair(GroundTruth):
    source: FaceRegion
    target: FaceRegion
    is_same_person: bool

This way, it is clear which bounding boxes and keypoints are associated to which image in the test sample.