Skip to main content

Fabrics

Machine learning pipelines might require complex environment setups such as specific software (python libraries, conda environments etc.) and hardware (GPU, CPU etc) configurations. At Layer, We call these environments fabrics.

Fabrics enable you to define the following configurations that you need for your ML pipeline.

  • GPU count
  • CPU count
  • Memory size

Predefined fabrics

Layer provides predefined fabrics with general purpose resource sets. These fabrics are:

Fabric nameCPUMemoryGPU
f-xxsmall0.52GB
f-xsmall0.54GB
f-small24GB
f-medium314GB
f-gpu-small348GB1

Parameters

Fabric parameters are defined below.

Fabric name

Name of the fabric. You can use this name in the @fabric decorator to use this fabric as your computation environment.

CPU

Show the CPU count of the fabric.

Memory

Show the memory of the fabric in GB.

GPU

Count of the GPUs. GPUs are only available in fabrics which have "gpu" in their names. We provide NVIDIA K80 GPUs currently.

GPU fabrics

GPU fabrics can only be used to train models, but not to build datasets. You can use them in combination with @model decorator, but not alongside @dataset.

GPU fabrics provide environments with NVIDIA packages allowing you to utilize GPU, including:

  • CUDA 11.2
  • cuDNN 8.1.0.77
  • TensorRT 7.2.2

GPU fabrics are tested with Tensorflow and PyTorch, which are also preinstalled.

Preinstalled Libraries

Following libraries are already installed in all of the fabrics. You can use them directly without the need of installation.

pandas==1.3.5
scipy==1.7.3
Keras==2.6.0
scikit-learn==1.0.2
pytorch==1.9.0
tensorflow-cpu==2.6.0
xgboost==1.5.1
nltk==3.6.7
matplotlib==3.5.1
grpcio==1.39.0
pyspark==3.0.1
tensorflow-datasets

How to use fabrics

You can add a fabric decorator with the @dataset or @model decorators to your training function. For more information and code examples, refer to the @fabric SDK documentation.

Example: The following churn-model will be trained on a f-medium fabric with 3 CPUs and 14GB memory.

from layer.decorators import model, fabric
import layer

@model("churn-model")
@fabric("f-medium")
def train_model():
...

layer.login()
layer.run([train_model])