Setting Up Medoids for Anomaly Detection

Preparatory workflow to set cluster centres and cluster threshold scores, so anomalies can be detected based on these thresholds

Here, we're using two approaches to set up these centres: the upper branch is the "distance matrix approach", and the lower is the "multimodal embedding model approach".

Once again, variables for Qdrant: cluster URL and a collection we're working with

Which point in the cluster we're using to draw threshold on: the furthest one from center, or the 2nd, ... Xth furthest one;

Here we are getting facet counts: information which unique values are there behind "crop_name" payload and how many points have these values (for example, we have 31 "cucumber" and 29 "cotton")

Which point in the cluster we're using to draw threshold on: the furthest one from center, or the 2nd, ... Xth furthest one; <this is the 2nd approach>

We need to get the total amount of points in Qdrant collection to use it as a limit in the "Crop Counts" node, so we won't lose any information;

Here we're extracting and gathering all the information about crop clusters, so we can call Qdrant distance matrix API for each cluster. We're propagating the biggest cluster size (of labeled data, in our case all data is labeled; for real use cases don't call distance matrix API if your labeled data is more than a couple of hundreds), the number of unique crop values and unique crop values themselves. We will run the algorithm once per unique crop cluster (to find it's center and threshold).

Hardcoded descriptions on how each crop usually looks; They were generated with chatGPT, and that can be technically done directly in n8n based on the crop name or a crop picture (we need a good description of how the most normal specimen of a crop looks like)

Calling distance matrix API once per cluster.

sample - how many points we are sampling (here filtered by crop_name field, so we are sampling within each cluster, and since we are passing the biggest cluster size to sample, we will get all points from each cluster.

limit is the number of neighbours distance to which we will see calculated. Since we want all pairwise distances between the points within a cluster, here we're once again setting an upper limit equal to the biggest cluster size;

Splitting out into each unique crop cluster

Using distance matrix generated by Qdrant and coo_array from scipy, we're finding a representative for each cluster (point which is the most similar to all other points within a cluster, based on the Cosine distance)

To find a representative with this approach, we:

1. Embed descriptions of crops with the same Voyage model we used for images (we can do so, since model is multimodal)
2. For each (crop) cluster, find an image the closest by Cosine similarity metric to this embedded description. We will consider it a perfect representative of the cluster

Embedding descriptions with Voyage model [Note] mind input_type, it's "query"

Find the closest image to the description embeddings (done per cluster) [Note] Mind exact parameter [Note] limit is 1 because vector database always returns points sorted by distance from the most similar one to the least [Note] using parameter is here because our vectors uploaded in the previous pipeline are named "voyage".

Fetching vectors of centres by their IDs

Set in Qdrant "is_medoid" payloads for points which were defined as centres by "distance matrix approach"

Here, we don't have to fetch a vector by point id as in the "distance matrix approach", since an API call in the previous node is able to return vectors stored in Qdrant as a response, while the distance matrix API returns only points IDs.

Set in Qdrant "is_text_anchor_medoid" payloads for points which were defined as centres by "multimodal embedding model approach".

Starting from here, this and the three following nodes are analogous for both methods, with a difference only in variable names. The goal is to find a class (cluster) threshold score so we can use it for anomaly detection (for each class).

Finding the most dissimilar point to a centre vector (within each class) is equivalent to finding the most similar point to the opposite of a centre vector, aka the centre vector with all coordinates multiplied by -1. It is always true with Cosine vector similarity metric (that we're using).

So here, we found the most dissimilar point within the crop class to the class centre (or the Xth dissimilar point, depending on a variable set in the beginning of this pipeline). Our threshold score is the similarity score between this point and the class centre. Now we're saving it as meta information of each class centre point. All preparatory work for anomaly detection is done.

For anomaly detection

1. The first pipeline is uploading (crops) dataset to Qdrant's collection.
2. This is the second pipeline, to set up cluster (class) centres in this Qdrant collection & cluster (class) threshold scores.
3. The third one is the anomaly detection tool, which takes any image as input and uses all preparatory work done with Qdrant (crops) collection.

To recreate it

You'll have to upload crops dataset from Kaggle to your own Google Storage bucket, and re-create APIs/connections to Qdrant Cloud (you can use Free Tier cluster), Voyage AI API & Google Cloud Storage

In general, pipelines are adaptable to any dataset of images

[2/3] Set up medoids (2 types) for anomaly detection (crops dataset)

Purpose and structure

This folder contains the [2/3] Set up medoids (2 types) for anomaly detection (crops dataset) workflow. Its executable nodes use code, httpRequest, manualTrigger, merge, set, splitOut. The table and connection map below are derived from the JSON, not an inferred guarantee of behavior.

Setup

Import inactive. Configure your own credentials and replace REPLACE_WITH_* values. Review external destinations and Code nodes before running.

Entry points

When clicking ‘Test workflow’

Verification

Sanitized source candidate; not runtime-tested. This overview is generated from node metadata. Read the folder README for the connection map, credential types, and license.

Attribution

https://github.com/Zie619/n8n-workflows/blob/94007c1445d9258a7da116646b79473e7c7c3282/workflows/Splitout/1915_Splitout_Code_Automation_Webhook.json MIT; preserve source notices.