metadata
language:
- en
license: apache-2.0
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:36
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
base_model: Snowflake/snowflake-arctic-embed-m-v1.5
widget:
- source_sentence: How can I connect a GCP Image Builder to resources using ZenML?
sentences:
- |-
_run.steps[step_name]
whylogs_step.visualize()if __name__ == "__main__":
visualize_statistics("data_loader")
visualize_statistics("train_data_profiler", "test_data_profiler")
PreviousEvidentlyNextDevelop a custom data validator
Last updated 1 month ago
- |-
Implement a custom integration
Creating an external integration and contributing to ZenML
PreviousContribute to ZenMLNextOverview
Last updated 4 months ago
- >-
--connector <CONNECTOR_ID>
Example Command Output$ zenml image-builder connect gcp-image-builder
--connector gcp-generic
Successfully connected image builder `gcp-image-builder` to the
following resources:
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┓
┃ CONNECTOR ID │ CONNECTOR NAME │ CONNECTOR TYPE
│ RESOURCE TYPE │ RESOURCE NAMES ┃
┠──────────────────────────────────────┼────────────────┼────────────────┼────────────────┼────────────────┨
┃ bfdb657d-d808-47e7-9974-9ba6e4919d83 │ gcp-generic │ 🔵 gcp
│ 🔵 gcp-generic │ zenml-core ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┛
As a final step, you can use the GCP Image Builder in a ZenML Stack:
# Register and set a stack with the new image builder
zenml stack register <STACK_NAME> -i <IMAGE_BUILDER_NAME> ... --set
When you register the GCP Image Builder, you can generate a GCP Service
Account Key, save it to a local file and then reference it in the Image
Builder configuration.
This method has the advantage that you don't need to install and
configure the GCP CLI on your host, but it's still not as secure as
using a GCP Service Connector and the stack component configuration is
not portable to other hosts.
For this method, you need to create a user-managed GCP service account,
and grant it privileges to access the Cloud Build API and to run Cloud
Builder jobs (e.g. the Cloud Build Editor IAM role.
With the service account key downloaded to a local file, you can
register the GCP Image Builder as follows:
zenml image-builder register <IMAGE_BUILDER_NAME> \
--flavor=gcp \
--project=<GCP_PROJECT_ID> \
--service_account_path=<PATH_TO_SERVICE_ACCOUNT_KEY> \
--cloud_builder_image=<BUILDER_IMAGE_NAME> \
--network=<DOCKER_NETWORK> \
--build_timeout=<BUILD_TIMEOUT_IN_SECONDS>
- source_sentence: >-
How do I register and activate a ZenML stack with a new GCP Image Builder
while ensuring proper authentication?
sentences:
- >-
oad the returned whylogs profile to WhyLabs, e.g.:import pandas as pd
from whylogs.core import DatasetProfileView
import whylogs as why
from zenml import step
from zenml.integrations.whylogs.flavors.whylogs_data_validator_flavor
import (
WhylogsDataValidatorSettings,
)
whylogs_settings = WhylogsDataValidatorSettings(
enable_whylabs=True, dataset_id="<WHYLABS_DATASET_ID>"
)
@step(
settings={
"data_validator": whylogs_settings
}
)
def data_profiler(
dataset: pd.DataFrame,
) -> DatasetProfileView:
"""Custom data profiler step with whylogs
Args:
dataset: a Pandas DataFrame
Returns:
Whylogs Profile generated for the dataset
"""
# validation pre-processing (e.g. dataset preparation) can take place
here
results = why.log(dataset)
profile = results.profile()
# validation post-processing (e.g. interpret results, take actions) can
happen here
return profile.view()
Visualizing whylogs Profiles
You can view visualizations of the whylogs profiles generated by your
pipeline steps directly in the ZenML dashboard by clicking on the
respective artifact in the pipeline run DAG.
Alternatively, if you are running inside a Jupyter notebook, you can
load and render the whylogs profiles using the artifact.visualize()
method, e.g.:
from zenml.client import Client
def visualize_statistics(
step_name: str, reference_step_name: Optional[str] = None
) -> None:
"""Helper function to visualize whylogs statistics from step artifacts.
Args:
step_name: step that generated and returned a whylogs profile
reference_step_name: an optional second step that generated a whylogs
profile to use for data drift visualization where two whylogs
profiles are required.
"""
pipe = Client().get_pipeline(pipeline="data_profiling_pipeline")
whylogs_step = pipe.last_run.steps[step_name]
whylogs_step.visualize()
- >-
ogsDataValidatorSettings,
)
from zenml import step@step(
settings={
"data_validator": WhylogsDataValidatorSettings(
enable_whylabs=True, dataset_id="model-1"
)
}
)
def data_loader() -> Tuple[
Annotated[pd.DataFrame, "data"],
Annotated[DatasetProfileView, "profile"]
]:
"""Load the diabetes dataset."""
X, y = datasets.load_diabetes(return_X_y=True, as_frame=True)
# merge X and y together
df = pd.merge(X, y, left_index=True, right_index=True)
profile = why.log(pandas=df).profile().view()
return df, profile
How do you use it?
Whylogs's profiling functions take in a pandas.DataFrame dataset
generate a DatasetProfileView object containing all the relevant
information extracted from the dataset.
There are three ways you can use whylogs in your ZenML pipelines that
allow different levels of flexibility:
instantiate, configure and insert the standard WhylogsProfilerStep
shipped with ZenML into your pipelines. This is the easiest way and the
recommended approach, but can only be customized through the supported
step configuration parameters.
call the data validation methods provided by the whylogs Data Validator
in your custom step implementation. This method allows for more
flexibility concerning what can happen in the pipeline step, but you are
still limited to the functionality implemented in the Data Validator.
use the whylogs library directly in your custom step implementation.
This gives you complete freedom in how you are using whylogs's features.
You can visualize whylogs profiles in Jupyter notebooks or view them
directly in the ZenML dashboard.
The whylogs standard step
- >2-
build to finish. More information: Build Timeout.We can register the image builder and use it in our active stack:
zenml image-builder register <IMAGE_BUILDER_NAME> \
--flavor=gcp \
--cloud_builder_image=<BUILDER_IMAGE_NAME> \
--network=<DOCKER_NETWORK> \
--build_timeout=<BUILD_TIMEOUT_IN_SECONDS>
# Register and activate a stack with the new image builder
zenml stack register <STACK_NAME> -i <IMAGE_BUILDER_NAME> ... --set
You also need to set up authentication required to access the Cloud
Build GCP services.
Authentication Methods
Integrating and using a GCP Image Builder in your pipelines is not
possible without employing some form of authentication. If you're
looking for a quick way to get started locally, you can use the Local
Authentication method. However, the recommended way to authenticate to
the GCP cloud platform is through a GCP Service Connector. This is
particularly useful if you are configuring ZenML stacks that combine the
GCP Image Builder with other remote stack components also running in
GCP.
This method uses the implicit GCP authentication available in the
environment where the ZenML code is running. On your local machine, this
is the quickest way to configure a GCP Image Builder. You don't need to
supply credentials explicitly when you register the GCP Image Builder,
as it leverages the local credentials and configuration that the Google
Cloud CLI stores on your local machine. However, you will need to
install and set up the Google Cloud CLI on your machine as a
prerequisite, as covered in the Google Cloud documentation , before you
register the GCP Image Builder.
Stacks using the GCP Image Builder set up with local authentication are
not portable across environments. To make ZenML pipelines fully
portable, it is recommended to use a GCP Service Connector to
authenticate your GCP Image Builder to the GCP cloud platform.
- source_sentence: How can I register and set a stack with a new image builder using ZenML?
sentences:
- |-
ZenML - Bridging the gap between ML & Ops
Legacy Docs
Bleeding EdgeLegacy Docs0.67.0
🧙♂️Find older version our docs
Powered by GitBook
- >-
> \
--build_timeout=<BUILD_TIMEOUT_IN_SECONDS># Register and set a stack with the new image builder
zenml stack register <STACK_NAME> -i <IMAGE_BUILDER_NAME> ... --set
Caveats
As described in this Google Cloud Build documentation page, Google Cloud
Build uses containers to execute the build steps which are automatically
attached to a network called cloudbuild that provides some Application
Default Credentials (ADC), that allow the container to be authenticated
and therefore use other GCP services.
By default, the GCP Image Builder is executing the build command of the
ZenML Pipeline Docker image with the option --network=cloudbuild, so the
ADC provided by the cloudbuild network can also be used in the build.
This is useful if you want to install a private dependency from a GCP
Artifact Registry, but you will also need to use a custom base parent
image with the keyrings.google-artifactregistry-auth installed, so pip
can connect and authenticate in the private artifact registry to
download the dependency.
FROM zenmldocker/zenml:latest
RUN pip install keyrings.google-artifactregistry-auth
The above Dockerfile uses zenmldocker/zenml:latest as a base image, but
is recommended to change the tag to specify the ZenML version and Python
version like 0.33.0-py3.10.
PreviousKaniko Image BuilderNextDevelop a Custom Image Builder
Last updated 21 days ago
- >-
res Spark to handle the resource configuration."""def
_backend_configuration(
self,
spark_config: SparkConf,
step_config: "StepConfiguration",
) -> None:
"""Configures Spark to handle backends like YARN, Mesos or Kubernetes."""
def _io_configuration(
self,
spark_config: SparkConf
) -> None:
"""Configures Spark to handle different input/output sources."""
def _additional_configuration(
self,
spark_config: SparkConf
) -> None:
"""Appends the user-defined configuration parameters."""
def _launch_spark_job(
self,
spark_config: SparkConf,
entrypoint_command: List[str]
) -> None:
"""Generates and executes a spark-submit command."""
def launch(
self,
info: "StepRunInfo",
entrypoint_command: List[str],
) -> None:
"""Launches the step on Spark."""
Under the base configuration, you will see the main configuration
parameters:
master is the master URL for the cluster where Spark will run. You might
see different schemes for this URL with varying cluster managers such as
Mesos, YARN, or Kubernetes.
deploy_mode can either be 'cluster' (default) or 'client' and it decides
where the driver node of the application will run.
submit_args is the JSON string of a dictionary, which will be used to
define additional parameters if required ( Spark has a wide variety of
parameters, thus including them all in a single class was deemed
unnecessary.).
In addition to this configuration, the launch method of the step
operator gets additional configuration parameters from the
DockerSettings and ResourceSettings. As a result, the overall
configuration happens in 4 base methods:
_resource_configuration translates the ZenML ResourceSettings object to
Spark's own resource configuration.
_backend_configuration is responsible for cluster-manager-specific
configuration.
- source_sentence: >-
How can I install ZenML with support for a local dashboard, and what
precautions should I take when installing on a Mac with Apple Silicon?
sentences:
- >2-
visit our PyPi package page.
Running with Dockerzenml is also available as a Docker image hosted
publicly on DockerHub. Use the following command to get started in a
bash environment with zenml available:
docker run -it zenmldocker/zenml /bin/bash
If you would like to run the ZenML server with Docker:
docker run -it -d -p 8080:8080 zenmldocker/zenml-server
Deploying the server
Though ZenML can run entirely as a pip package on a local system,
complete with the dashboard. You can do this easily:
pip install "zenml[server]"
zenml up # opens the dashboard locally
However, advanced ZenML features are dependent on a centrally-deployed
ZenML server accessible to other MLOps stack components. You can read
more about it here.
For the deployment of ZenML, you have the option to either self-host it
or register for a free ZenML Pro account.
PreviousIntroductionNextCore concepts
Last updated 20 days ago
- |-
Evaluation and metrics
Track how your RAG pipeline improves using evaluation and metrics.
PreviousBasic RAG inference pipelineNextEvaluation in 65 lines of code
Last updated 4 months ago
- >-
🧙Installation
Installing ZenML and getting started.
ZenML is a Python package that can be installed directly via pip:
pip install zenml
Note that ZenML currently supports Python 3.8, 3.9, 3.10, and 3.11.
Please make sure that you are using a supported Python version.
Install with the dashboard
ZenML comes bundled with a web dashboard that lives inside a sister
repository. In order to get access to the dashboard locally, you need to
launch the ZenML Server and Dashboard locally. For this, you need to
install the optional dependencies for the ZenML Server:
pip install "zenml[server]"
We highly encourage you to install ZenML in a virtual environment. At
ZenML, We like to use virtualenvwrapper or pyenv-virtualenv to manage
our Python virtual environments.
Installing onto MacOS with Apple Silicon (M1, M2)
A change in how forking works on Macs running on Apple Silicon means
that you should set the following environment variable which will ensure
that your connections to the server remain unbroken:
export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
You can read more about this here. This environment variable is needed
if you are working with a local server on your Mac, but if you're just
using ZenML as a client / CLI and connecting to a deployed server then
you don't need to set it.
Nightly builds
ZenML also publishes nightly builds under the zenml-nightly package
name. These are built from the latest develop branch (to which work
ready for release is published) and are not guaranteed to be stable. To
install the nightly build, run:
pip install zenml-nightly
Verifying installations
Once the installation is completed, you can check whether the
installation was successful either through Bash:
zenml version
or through Python:
import zenml
print(zenml.__version__)
If you would like to learn more about the current release, please visit
our PyPi package page.
Running with Docker
- source_sentence: >-
How does the KubernetesSparkStepOperator utilize the
PipelineDockerImageBuilder class to manage Docker images for Spark jobs on
Kubernetes?
sentences:
- |-
ZenML - Bridging the gap between ML & Ops
Legacy Docs
Bleeding EdgeLegacy Docs0.67.0
🧙♂️Find older version our docs
Powered by GitBook
- >-
nsible for cluster-manager-specific configuration._io_configuration is a
critical method. Even though we have materializers, Spark might require
additional packages and configuration to work with a specific
filesystem. This method is used as an interface to provide this
configuration.
_additional_configuration takes the submit_args, converts, and appends
them to the overall configuration.
Once the configuration is completed, _launch_spark_job comes into play.
This takes the completed configuration and runs a Spark job on the given
master URL with the specified deploy_mode. By default, this is achieved
by creating and executing a spark-submit command.
Warning
In its first iteration, the pre-configuration with _io_configuration
method is only effective when it is paired with an S3ArtifactStore
(which has an authentication secret). When used with other artifact
store flavors, you might be required to provide additional configuration
through the submit_args.
Stack Component: KubernetesSparkStepOperator
The KubernetesSparkStepOperator is implemented by subclassing the base
SparkStepOperator and uses the PipelineDockerImageBuilder class to build
and push the required Docker images.
from typing import Optional
from zenml.integrations.spark.step_operators.spark_step_operator import
(
SparkStepOperatorConfig
)
class KubernetesSparkStepOperatorConfig(SparkStepOperatorConfig):
"""Config for the Kubernetes Spark step operator."""
namespace: Optional[str] = None
service_account: Optional[str] = None
from pyspark.conf import SparkConf
from zenml.utils.pipeline_docker_image_builder import
PipelineDockerImageBuilder
from zenml.integrations.spark.step_operators.spark_step_operator import
(
SparkStepOperator
)
class KubernetesSparkStepOperator(SparkStepOperator):
"""Step operator which runs Steps with Spark on Kubernetes."""
- "ngs/python/Dockerfile -u 0 build\n\nConfiguring RBACAdditionally, you may need to create the several resources in Kubernetes in order to give Spark access to edit/manage your driver executor pods.\n\nTo do so, create a file called rbac.yaml with the following content:\n\napiVersion: v1\nkind: Namespace\nmetadata:\n name: spark-namespace\n---\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n name: spark-service-account\n namespace: spark-namespace\n---\napiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRoleBinding\nmetadata:\n name: spark-role\n namespace: spark-namespace\nsubjects:\n - kind: ServiceAccount\n name: spark-service-account\n namespace: spark-namespace\nroleRef:\n kind: ClusterRole\n name: edit\n apiGroup: rbac.authorization.k8s.io\n---\n\nAnd then execute the following command to create the resources:\n\naws eks --region=$REGION update-kubeconfig --name=$EKS_CLUSTER_NAME\n\nkubectl create -f rbac.yaml\n\nLastly, note down the namespace and the name of the service account since you will need them when registering the stack component in the next step.\n\nHow to use it\n\nTo use the KubernetesSparkStepOperator, you need:\n\nthe ZenML spark integration. If you haven't installed it already, run\n\nzenml integration install spark\n\nDocker installed and running.\n\nA remote artifact store as part of your stack.\n\nA remote container registry as part of your stack.\n\nA Kubernetes cluster deployed.\n\nWe can then register the step operator and use it in our active stack:\n\nzenml step-operator register spark_step_operator \\\n\t--flavor=spark-kubernetes \\\n\t--master=k8s://$EKS_API_SERVER_ENDPOINT \\\n\t--namespace=<SPARK_KUBERNETES_NAMESPACE> \\\n\t--service_account=<SPARK_KUBERNETES_SERVICE_ACCOUNT>\n\n# Register the stack\nzenml stack register spark_stack \\\n -o default \\\n -s spark_step_operator \\\n -a spark_artifact_store \\\n -c spark_container_registry \\\n -i local_builder \\\n --set"
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
model-index:
- name: zenml/finetuned-snowflake-arctic-embed-m-v1.5
results:
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 384
type: dim_384
metrics:
- type: cosine_accuracy@1
value: 0.75
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 1
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 1
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 1
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.75
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.3333333333333333
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.2
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.1
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.75
name: Cosine Recall@1
- type: cosine_recall@3
value: 1
name: Cosine Recall@3
- type: cosine_recall@5
value: 1
name: Cosine Recall@5
- type: cosine_recall@10
value: 1
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.875
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.8333333333333334
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.8333333333333334
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 256
type: dim_256
metrics:
- type: cosine_accuracy@1
value: 0.75
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.75
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 1
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 1
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.75
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.25
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.2
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.1
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.75
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.75
name: Cosine Recall@3
- type: cosine_recall@5
value: 1
name: Cosine Recall@5
- type: cosine_recall@10
value: 1
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.8576691395183482
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.8125
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.8125
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 128
type: dim_128
metrics:
- type: cosine_accuracy@1
value: 0.75
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.75
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 1
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 1
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.75
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.25
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.2
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.1
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.75
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.75
name: Cosine Recall@3
- type: cosine_recall@5
value: 1
name: Cosine Recall@5
- type: cosine_recall@10
value: 1
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.8576691395183482
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.8125
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.8125
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 64
type: dim_64
metrics:
- type: cosine_accuracy@1
value: 0.75
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 1
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 1
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 1
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.75
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.3333333333333333
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.2
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.1
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.75
name: Cosine Recall@1
- type: cosine_recall@3
value: 1
name: Cosine Recall@3
- type: cosine_recall@5
value: 1
name: Cosine Recall@5
- type: cosine_recall@10
value: 1
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.875
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.8333333333333334
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.8333333333333334
name: Cosine Map@100
zenml/finetuned-snowflake-arctic-embed-m-v1.5
This is a sentence-transformers model finetuned from Snowflake/snowflake-arctic-embed-m-v1.5 on the json dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
Model Details
Model Description
- Model Type: Sentence Transformer
- Base model: Snowflake/snowflake-arctic-embed-m-v1.5
- Maximum Sequence Length: 512 tokens
- Output Dimensionality: 768 dimensions
- Similarity Function: Cosine Similarity
- Training Dataset:
- json
- Language: en
- License: apache-2.0
Model Sources
- Documentation: Sentence Transformers Documentation
- Repository: Sentence Transformers on GitHub
- Hugging Face: Sentence Transformers on Hugging Face
Full Model Architecture
SentenceTransformer(
(0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)
Usage
Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("zenml/finetuned-snowflake-arctic-embed-m-v1.5")
# Run inference
sentences = [
'How does the KubernetesSparkStepOperator utilize the PipelineDockerImageBuilder class to manage Docker images for Spark jobs on Kubernetes?',
'nsible for cluster-manager-specific configuration._io_configuration is a critical method. Even though we have materializers, Spark might require additional packages and configuration to work with a specific filesystem. This method is used as an interface to provide this configuration.\n\n_additional_configuration takes the submit_args, converts, and appends them to the overall configuration.\n\nOnce the configuration is completed, _launch_spark_job comes into play. This takes the completed configuration and runs a Spark job on the given master URL with the specified deploy_mode. By default, this is achieved by creating and executing a spark-submit command.\n\nWarning\n\nIn its first iteration, the pre-configuration with _io_configuration method is only effective when it is paired with an S3ArtifactStore (which has an authentication secret). When used with other artifact store flavors, you might be required to provide additional configuration through the submit_args.\n\nStack Component: KubernetesSparkStepOperator\n\nThe KubernetesSparkStepOperator is implemented by subclassing the base SparkStepOperator and uses the PipelineDockerImageBuilder class to build and push the required Docker images.\n\nfrom typing import Optional\n\nfrom zenml.integrations.spark.step_operators.spark_step_operator import (\n SparkStepOperatorConfig\n)\n\nclass KubernetesSparkStepOperatorConfig(SparkStepOperatorConfig):\n """Config for the Kubernetes Spark step operator."""\n\nnamespace: Optional[str] = None\n service_account: Optional[str] = None\n\nfrom pyspark.conf import SparkConf\n\nfrom zenml.utils.pipeline_docker_image_builder import PipelineDockerImageBuilder\nfrom zenml.integrations.spark.step_operators.spark_step_operator import (\n SparkStepOperator\n)\n\nclass KubernetesSparkStepOperator(SparkStepOperator):\n """Step operator which runs Steps with Spark on Kubernetes."""',
"ngs/python/Dockerfile -u 0 build\n\nConfiguring RBACAdditionally, you may need to create the several resources in Kubernetes in order to give Spark access to edit/manage your driver executor pods.\n\nTo do so, create a file called rbac.yaml with the following content:\n\napiVersion: v1\nkind: Namespace\nmetadata:\n name: spark-namespace\n---\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n name: spark-service-account\n namespace: spark-namespace\n---\napiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRoleBinding\nmetadata:\n name: spark-role\n namespace: spark-namespace\nsubjects:\n - kind: ServiceAccount\n name: spark-service-account\n namespace: spark-namespace\nroleRef:\n kind: ClusterRole\n name: edit\n apiGroup: rbac.authorization.k8s.io\n---\n\nAnd then execute the following command to create the resources:\n\naws eks --region=$REGION update-kubeconfig --name=$EKS_CLUSTER_NAME\n\nkubectl create -f rbac.yaml\n\nLastly, note down the namespace and the name of the service account since you will need them when registering the stack component in the next step.\n\nHow to use it\n\nTo use the KubernetesSparkStepOperator, you need:\n\nthe ZenML spark integration. If you haven't installed it already, run\n\nzenml integration install spark\n\nDocker installed and running.\n\nA remote artifact store as part of your stack.\n\nA remote container registry as part of your stack.\n\nA Kubernetes cluster deployed.\n\nWe can then register the step operator and use it in our active stack:\n\nzenml step-operator register spark_step_operator \\\n\t--flavor=spark-kubernetes \\\n\t--master=k8s://$EKS_API_SERVER_ENDPOINT \\\n\t--namespace=<SPARK_KUBERNETES_NAMESPACE> \\\n\t--service_account=<SPARK_KUBERNETES_SERVICE_ACCOUNT>\n\n# Register the stack\nzenml stack register spark_stack \\\n -o default \\\n -s spark_step_operator \\\n -a spark_artifact_store \\\n -c spark_container_registry \\\n -i local_builder \\\n --set",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
Evaluation
Metrics
Information Retrieval
- Datasets:
dim_384,dim_256,dim_128anddim_64 - Evaluated with
InformationRetrievalEvaluator
| Metric | dim_384 | dim_256 | dim_128 | dim_64 |
|---|---|---|---|---|
| cosine_accuracy@1 | 0.75 | 0.75 | 0.75 | 0.75 |
| cosine_accuracy@3 | 1.0 | 0.75 | 0.75 | 1.0 |
| cosine_accuracy@5 | 1.0 | 1.0 | 1.0 | 1.0 |
| cosine_accuracy@10 | 1.0 | 1.0 | 1.0 | 1.0 |
| cosine_precision@1 | 0.75 | 0.75 | 0.75 | 0.75 |
| cosine_precision@3 | 0.3333 | 0.25 | 0.25 | 0.3333 |
| cosine_precision@5 | 0.2 | 0.2 | 0.2 | 0.2 |
| cosine_precision@10 | 0.1 | 0.1 | 0.1 | 0.1 |
| cosine_recall@1 | 0.75 | 0.75 | 0.75 | 0.75 |
| cosine_recall@3 | 1.0 | 0.75 | 0.75 | 1.0 |
| cosine_recall@5 | 1.0 | 1.0 | 1.0 | 1.0 |
| cosine_recall@10 | 1.0 | 1.0 | 1.0 | 1.0 |
| cosine_ndcg@10 | 0.875 | 0.8577 | 0.8577 | 0.875 |
| cosine_mrr@10 | 0.8333 | 0.8125 | 0.8125 | 0.8333 |
| cosine_map@100 | 0.8333 | 0.8125 | 0.8125 | 0.8333 |
Training Details
Training Dataset
json
- Dataset: json
- Size: 36 training samples
- Columns:
positiveandanchor - Approximate statistics based on the first 36 samples:
positive anchor type string string details - min: 13 tokens
- mean: 23.11 tokens
- max: 38 tokens
- min: 31 tokens
- mean: 299.64 tokens
- max: 512 tokens
- Samples:
positive anchor How does the ZenML BaseService registry manage serialization and re-creation of configurations for BaseService instances as part of the remote model server setup?e details of the deployment process from the user.It needs to act as a ZenML BaseService registry, where every BaseService instance is used as an internal representation of a remote model server (see the find_model_server abstract method). To achieve this, it must be able to re-create the configuration of a BaseService from information that is persisted externally, alongside, or even as part of the remote model server configuration itself. For example, for model servers that are implemented as Kubernetes resources, the BaseService instances can be serialized and saved as Kubernetes resource annotations. This allows the model deployer to keep track of all externally running model servers and to re-create their corresponding BaseService instance representations at any given time. The model deployer also defines methods that implement basic life-cycle management on remote model servers outside the coverage of a pipeline (see stop_model_server , start_model_server and delete_model_server)....How do you ensure the MyExperimentTrackerFlavor is properly registered and available in ZenML?gister flavors.my_flavor.MyExperimentTrackerFlavorZenML resolves the flavor class by taking the path where you initialized zenml (via zenml init) as the starting point of resolution. Therefore, please ensure you follow the best practice of initializing zenml at the root of your repository.
If ZenML does not find an initialized ZenML repository in any parent directory, it will default to the current working directory, but usually, it's better to not have to rely on this mechanism and initialize zenml at the root.
Afterward, you should see the new flavor in the list of available flavors:
zenml experiment-tracker flavor list
It is important to draw attention to when and how these base abstractions are coming into play in a ZenML workflow.
The CustomExperimentTrackerFlavor class is imported and utilized upon the creation of the custom flavor through the CLI.
The CustomExperimentTrackerConfig class is imported when someone tries to register/update a stack component with this custom fl...How do you load and profile a dataset using the Whylogs data validator in ZenML?ogsDataValidatorSettings,
)
from zenml import step@step(
settings={
"data_validator": WhylogsDataValidatorSettings(
enable_whylabs=True, dataset_id="model-1"
)
}
)
def data_loader() -> Tuple[
Annotated[pd.DataFrame, "data"],
Annotated[DatasetProfileView, "profile"]
]:
"""Load the diabetes dataset."""
X, y = datasets.load_diabetes(return_X_y=True, as_frame=True)
# merge X and y together
df = pd.merge(X, y, left_index=True, right_index=True)
profile = why.log(pandas=df).profile().view()
return df, profile
How do you use it?
Whylogs's profiling functions take in a pandas.DataFrame dataset generate a DatasetProfileView object containing all the relevant information extracted from the dataset.
There are three ways you can use whylogs in your ZenML pipelines that allow different levels of flexibility:
instantiate, configure and insert the standard WhylogsProfilerStep shipped with ZenML into your pipelines. This is the easiest ... - Loss:
MatryoshkaLosswith these parameters:{ "loss": "MultipleNegativesRankingLoss", "matryoshka_dims": [ 384, 256, 128, 64 ], "matryoshka_weights": [ 1, 1, 1, 1 ], "n_dims_per_step": -1 }
Training Hyperparameters
Non-Default Hyperparameters
eval_strategy: epochper_device_train_batch_size: 4per_device_eval_batch_size: 16gradient_accumulation_steps: 16learning_rate: 2e-05num_train_epochs: 4lr_scheduler_type: cosinewarmup_ratio: 0.1tf32: Falseload_best_model_at_end: Trueoptim: adamw_torch_fusedbatch_sampler: no_duplicates
All Hyperparameters
Click to expand
overwrite_output_dir: Falsedo_predict: Falseeval_strategy: epochprediction_loss_only: Trueper_device_train_batch_size: 4per_device_eval_batch_size: 16per_gpu_train_batch_size: Noneper_gpu_eval_batch_size: Nonegradient_accumulation_steps: 16eval_accumulation_steps: Nonetorch_empty_cache_steps: Nonelearning_rate: 2e-05weight_decay: 0.0adam_beta1: 0.9adam_beta2: 0.999adam_epsilon: 1e-08max_grad_norm: 1.0num_train_epochs: 4max_steps: -1lr_scheduler_type: cosinelr_scheduler_kwargs: {}warmup_ratio: 0.1warmup_steps: 0log_level: passivelog_level_replica: warninglog_on_each_node: Truelogging_nan_inf_filter: Truesave_safetensors: Truesave_on_each_node: Falsesave_only_model: Falserestore_callback_states_from_checkpoint: Falseno_cuda: Falseuse_cpu: Falseuse_mps_device: Falseseed: 42data_seed: Nonejit_mode_eval: Falseuse_ipex: Falsebf16: Falsefp16: Falsefp16_opt_level: O1half_precision_backend: autobf16_full_eval: Falsefp16_full_eval: Falsetf32: Falselocal_rank: 0ddp_backend: Nonetpu_num_cores: Nonetpu_metrics_debug: Falsedebug: []dataloader_drop_last: Falsedataloader_num_workers: 0dataloader_prefetch_factor: Nonepast_index: -1disable_tqdm: Falseremove_unused_columns: Truelabel_names: Noneload_best_model_at_end: Trueignore_data_skip: Falsefsdp: []fsdp_min_num_params: 0fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}fsdp_transformer_layer_cls_to_wrap: Noneaccelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}deepspeed: Nonelabel_smoothing_factor: 0.0optim: adamw_torch_fusedoptim_args: Noneadafactor: Falsegroup_by_length: Falselength_column_name: lengthddp_find_unused_parameters: Noneddp_bucket_cap_mb: Noneddp_broadcast_buffers: Falsedataloader_pin_memory: Truedataloader_persistent_workers: Falseskip_memory_metrics: Trueuse_legacy_prediction_loop: Falsepush_to_hub: Falseresume_from_checkpoint: Nonehub_model_id: Nonehub_strategy: every_savehub_private_repo: Falsehub_always_push: Falsegradient_checkpointing: Falsegradient_checkpointing_kwargs: Noneinclude_inputs_for_metrics: Falseeval_do_concat_batches: Truefp16_backend: autopush_to_hub_model_id: Nonepush_to_hub_organization: Nonemp_parameters:auto_find_batch_size: Falsefull_determinism: Falsetorchdynamo: Noneray_scope: lastddp_timeout: 1800torch_compile: Falsetorch_compile_backend: Nonetorch_compile_mode: Nonedispatch_batches: Nonesplit_batches: Noneinclude_tokens_per_second: Falseinclude_num_input_tokens_seen: Falseneftune_noise_alpha: Noneoptim_target_modules: Nonebatch_eval_metrics: Falseeval_on_start: Falseeval_use_gather_object: Falseprompts: Nonebatch_sampler: no_duplicatesmulti_dataset_batch_sampler: proportional
Training Logs
| Epoch | Step | dim_384_cosine_ndcg@10 | dim_256_cosine_ndcg@10 | dim_128_cosine_ndcg@10 | dim_64_cosine_ndcg@10 |
|---|---|---|---|---|---|
| 1.0 | 1 | 0.875 | 0.875 | 0.8577 | 0.875 |
| 2.0 | 3 | 0.875 | 0.8577 | 0.8577 | 0.875 |
| 3.0 | 4 | 0.875 | 0.8577 | 0.8577 | 0.875 |
- The bold row denotes the saved checkpoint.
Framework Versions
- Python: 3.11.11
- Sentence Transformers: 3.3.1
- Transformers: 4.43.1
- PyTorch: 2.5.1+cu124
- Accelerate: 1.2.0
- Datasets: 3.2.0
- Tokenizers: 0.19.1
Citation
BibTeX
Sentence Transformers
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
MatryoshkaLoss
@misc{kusupati2024matryoshka,
title={Matryoshka Representation Learning},
author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
year={2024},
eprint={2205.13147},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
MultipleNegativesRankingLoss
@misc{henderson2017efficient,
title={Efficient Natural Language Response Suggestion for Smart Reply},
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
year={2017},
eprint={1705.00652},
archivePrefix={arXiv},
primaryClass={cs.CL}
}