We use analytics and cookies to understand site traffic. Information about your use of our site is shared with Google for that purpose.You can read our privacy policies and terms of use etc by clicking here.
Image Data Outlier Detection
This demo is based on VAE outlier detection in the alibi detect project. Here we will :
- Launch an image classifier model trianed on the CIFAR10 dataset
- Setup an outlier detector for this particular model
- Send a request to get a image classification
- Send a perturbed request to get a outlier detection
Note
This demo requires Knative installation on the cluster as the outlier detector will be installed as a kservice.Create Model
Create the model in its own dedicated namespace (needed for outlier detection). It should be a knative eventing namespace (labelled knative-eventing-injection=enabled
).
Use the following model url with tensorflow
runtime. Set the protocol to ‘tensorflow’ and the predictor log url as “http://default-broker":
gs://seldon-models/tfserving/cifar10/resnet32
Setup Request Logger
Go to ‘Setup Request Logger’ and enter:
Logger Name: seldon-request-logger
Logger Image URI: docker.io/seldonio/seldon-request-logger:0.3.1
Environment Variables
ELASTICSEARCH_HOST: elasticsearch-master.seldon-logs.svc.cluster.local
ELASTICSEARCH_PORT: 9200
ELASTICSEARCH_PROTOCOL: http
Setup Outlier detector
Setup an outlier detector with model name cifar10
using the default settings (which sets Reply URL as seldon-request-logger in current namespace) and storage URI as follows:
gs://seldon-models/alibi-detect/od/OutlierVAE/cifar10
Make Predictions
Run a single prediction using the tensorflow payload format of an image truck. Also a perturbed image of the truck in the same format at outlier truck image. Make a couple of these requests at random using the predict tool in the UI.
Monitor outliers on the Requests Screen
Go to the requests screen to view all the historical requests. You can see the outlier value on each instance. Also you can highlight outliers based on this score and also use the filter to see only the outliers as needed.
Delete Deployment
Finally, delete the deployment. This will also remove the outlier detector created for the deployment.
Notes
A dedicated namespace is recommended as knative eventing sends all requets to the outlier detector as well as the logger. In a shared namespace requests from other models would also go to the outlier detector, which is for cifar10.
This demo can also be run for kfserving/inferenceservices using otherwise the same steps (see https://github.com/kubeflow/kfserving/tree/master/docs/samples/outlier-detection/alibi-detect/cifar10). The namespace must be labelled serving.kubeflow.org/inferenceservice=enabled
.