Skip to content

🤗 Transformers

PostgresML integrates 🤗 Hugging Face Transformers to bring state-of-the-art models into the data layer. There are tens of thousands of pre-trained models with pipelines to turn raw inputs into useful results. Many state of the art deep learning architectures have been published and made available for download. You will want to browse all the models available to find the perfect solution for your dataset and task.

Setup

Install the machine learning depedencies on the database for the transformers you would like to use:

See the Pytorch docs for more information.

$ sudo pip3 install torch

See the Tensorflow docs for more information.

$ sudo pip3 install tensorflow

See the Flax docs for more information.

$ sudo pip3 install flax

Models will be downloaded and cached on the database for repeated usage. View the Transformers installation docs for cache management details and offline deployments.

You may also want to install GPU support when working with larger models.

Standard Datasets

Many datasets have been published to stimulate research and benchmark architectures, but also to help demonstrate API usage in the tutorials. The Datasets package provides a way to load published datasets into Postgres:

$ sudo pip3 install datasets

Audio Processing

Torch Audio is required for many models that process audio data. You can install the additional dependencies with:

$ sudo pip3 install torchaudio

Comments