Atalaya Deployment is a basic deployable unit of computing that can be created and managed in Atalaya product. It describes a desired state of what model artifacts are required and how model-serving instances are orchestrated.
Once you have a model uploaded to Atalaya, you can then create model-serving deployment using the model.
The simplest and most common deployment is single model deployment. You can use
atalaya deploy command to create such deployment by specifying the target
model. It will create a default endpoint for exposing target model as a real-time
atalaya deploy mnist:1.0.0 --replica 3 Deployment mnist created
You can now use
atalaya get deployment command to query the status of the
deployment. The default deployment name is the same as the model respository
Atalaya users can create deployment configuration YAML file to describe the desired state of the deployment, including all the models required for the deployment, how much and what type of resources to request, and what are the endpoints desired.
For example, to create a simple deployment same as
atalaya deploy command
above, using a configuration file:
atalaya apply -f mnist-deployment.yaml
# mnist-deployment.yaml apiVersion: atalaya.io/v1alpha kind: atalayaDeployment metadata: name: mnist spec: operators: - name: default modelRepo: mnist tag: 1.0.0 replicas: 3 endpoints: - name: default operator: default
To create complex deployment that involves multiple models and multiple endpoints, we recommend to use the deployment configuration files above.
Here is an example deployment configuration, that allow users to deploy a production model and a 'dev' model together as three endpoints:
# my-super-cool-app.yaml apiVersion: atalaya.io/v1alpha kind: atalayaDeployment metadata: name: my-super-cool-app spec: operators: - name: mnist-prod modelRepo: mnist tag: 1.0.0 replicas: 3 resources: request: nvidia.com/gpu: 1 - name: mnist-dev modelRepo: mnist tag: 1.0.2 replicas: 1 endpoints: - name: 'prod' operator: mnist-prod - name: 'experimental' operator: mnist-dev - name: 'shadow' type: combiner operators: [ mnist, mnist-dev ]
atalaya apply -f command to create a deployment of this configuration:
atalaya apply -f ./my-super-cool-app.yaml
We recommend to always check-in your deployment configuration file to your Git(or other VCS) repository alone with your training code. This way we ensures the production deployment can be easily reproduced in case of an outage.
You can find more about advanced usage of deployment configuration here.
An operator represents the underlying computing unit in a deployment, typically it is one or multiple model-serving instances and their configurations. Users can also specify the amount of resoruce and type of resource required for each instance for each operator. e.g.:
operators: - name: inception-prod modelRepo: inception tag: v1 replicas: 3 instanceType: c1 autoscale: min: 1 max: 5
Other types of operators, such as feature extraction operator and feature store operator are in experimental stage, contact our sales representative for more details.
A deployment can expose multiple endpoints. There are two types of endpoitns:
- router - route API requests to operators or other routers
- combiner - combine the responses from multiple router or operators
When type is not specified, the default endpoint type is router pointing to a single operator.
endpoints: - name: 'prod' operator: inception-prod - name: 'shadow' type: combiner operators: [ inception, inception-dev ] - name: 'a-b-testing' type: router operators: - name: inception-prod weight: 80 - name: inception-next weight: 20
atalaya get deployments command to list all your active deployments.
atalaya get deployment DEPLOYMENT_TAG to view current status of the
Start / Stop Deployment
Atalaya provides the ability for you to pause and restart a deployment:
atalaya stop my-super-cool-app Deployment my-super-cool-app deactivated
atalaya start my-super-cool-app Deployment my-super-cool-app activiated
atalaya apply command will also start the deployment if it is not already
atalaya apply -f updated-deployment.yaml Deployment my-super-cool-app reconfigured
atalaya delete deployment TAG command to delete a deployment. Note that you
must deactivate a deployment with
atalaya stop command before removing it.
atalaya delete deployment my-super-cool-app Deployment my-super-cool-app deleted
In our Web UI, under the deployment detail page, the monitoring tab, you will find a dashboard containing all the metrics from the running instances of the deployment.
Currently, prediction logs are only available in Web UI, go to the deployment detail page and find the logging tab. We also offer enterprise customers to delivery the raw prediction log files to their cloud storage.