Skip to content

loaders

canari_ml.data.loaders

canari_ml.data.loaders.CanariMLDataLoaderFactory()

A factory class for managing a map of loader names and their corresponding implementation classes. Based on IceNet v0.4.0_dev

Attributes:

Name Type Description
_loader_map

A dictionary holding loader names against their implementation classes.
Source code in src/canari_ml/data/loaders/__init__.py 
21
22
23
24
25
26
27
28
def __init__(self):
    """Initialises the CanariMLDataLoaderFactory instance and sets up the initial
        loader map.
    """
    self._loader_map = dict(
        serial=SerialLoader,
        # dask=DaskMultiWorkerLoader, # TODO: Implement this
    )

canari_ml.data.loaders.CanariMLDataLoaderFactory.loader_map property

The loader map dictionary.

canari_ml.data.loaders.CanariMLDataLoaderFactory.add_data_loader(loader_name, loader_impl)

Adds a new loader to the loader map with the given name and implementation class.

Parameters:

Name Type Description Default
loader_name str

The name of the loader.

 required
loader_impl object

The implementation class of the loader.

 required

Returns:

Type Description
None

None. Updates _loader_map attribute in CanariMLDataLoaderFactory with specified loader name and implementation.

Raises:

Type Description
RuntimeError

If the loader name already exists or if the implementation class is not a descendant of CanariMLBaseDataLoader.
Source code in src/canari_ml/data/loaders/__init__.py 
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
def add_data_loader(self, loader_name: str, loader_impl: object) -> None:
    """Adds a new loader to the loader map with the given name and implementation
        class.

    Args:
        loader_name: The name of the loader.
        loader_impl: The implementation class of the loader.

    Returns:
        None. Updates `_loader_map` attribute in CanariMLDataLoaderFactory with
            specified loader name and implementation.

    Raises:
        RuntimeError: If the loader name already exists or if the implementation
            class is not a descendant of CanariMLBaseDataLoader.
    """
    if loader_name not in self._loader_map:
        if CanariMLBaseDataLoader in inspect.getmro(loader_impl):
            self._loader_map[loader_name] = loader_impl
        else:
            raise RuntimeError(
                "{} is not descended from CanariMLBaseDataLoader".format(
                    loader_impl.__name__
                )
            )
    else:
        raise RuntimeError(
            "Cannot add {} as already in loader map".format(loader_name)
        )

canari_ml.data.loaders.CanariMLDataLoaderFactory.create_data_loader(loader_name, *args, **kwargs)

Creates an instance of a loader based on specified name from the _loader_map dict attribute.

Parameters:

Name Type Description Default
loader_name str

The name of the loader.

 required
*args Unpack

Additional positional arguments, is passed to the loader constructor.

 ()
**kwargs Unpack

Additional keyword arguments, is passed to the loader constructor.

 {}

Returns:

Type Description
object

An instance of the loader class.

Raises:

Type Description
KeyError

If the loader name does not exist in _loader_map.
Source code in src/canari_ml/data/loaders/__init__.py 
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
def create_data_loader(self, loader_name: str, *args: Unpack, **kwargs: Unpack) -> object:
    """Creates an instance of a loader based on specified name from the
       `_loader_map` dict attribute.

    Args:
        loader_name: The name of the loader.
        *args: Additional positional arguments, is passed to the loader constructor.
        **kwargs: Additional keyword arguments, is passed to the loader constructor.

    Returns:
        An instance of the loader class.

    Raises:
        KeyError: If the loader name does not exist in `_loader_map`.
    """
    return self._loader_map[loader_name](*args, **kwargs)