AYAZ AHMED / PORTFOLIO
← ALL PROJECTSSELECTED WORK / 01

BIOINFORMATICS / RESEARCH SOFTWARE

AQUADEX

An eDNA analysis pipeline combining Kraken2 taxonomy, DNABERT-S sequence embeddings, exact FAISS L2 retrieval and density-based clustering.

Recorded interface walkthrough · demonstration dataSCROLL TO EXPAND ↓
01 / THE PROJECT

Reference classification meets representation learning.

Environmental DNA contains mixed, noisy sequence reads whose organisms may be poorly represented in reference databases. Aquadex couples conventional taxonomic assignment with a second, representation-based retrieval path: quality-filtered and dereplicated sequences are embedded, anchored to reference vectors and ranked by distance-derived novelty. Per-run HDF5, TSV, Newick and JSON artifacts connect the analysis stages to a dashboard for quality control, diversity and candidate inspection.

k = 5

Reference neighbours

Default exact-search neighbourhood in FAISSAnchoring.

float32

Vector storage

Reference embeddings are stacked into a dense FAISS matrix.

1 − e⁻ᵈ

Novelty transform

Applied to the nearest squared-L2 reference distance.

Nearest-reference novelty

d∗=min⁡j∥z−rj∥22,N(z)=1−e−d∗d_* = \min_j\lVert z-r_j\rVert_2^2,\qquad N(z)=1-e^{-d_*}

z is a query sequence embedding and rⱼ a reference embedding. IndexFlatL2 computes squared Euclidean distance; the minimum distance feeds directly into the NoveltyScorer distance metric.

Reference anchoring confidence

A(z)=exp⁡ ⁣(−1k∑j=1kdj),k=5A(z)=\exp\!\left(-\frac{1}{k}\sum_{j=1}^{k}d_j\right),\qquad k=5

FAISSAnchoring averages the retrieved neighbour distances before exponentiation. This distinguishes a close single neighbour from the broader local reference neighbourhood. Both formulas are implemented in the pipeline source.

02 / THROUGH THE SYSTEM

INPUT → OUTPUT
  1. 01

    Prepare

    FASTQ input passes through fastp quality control, then sequence conversion and vsearch dereplication.

  2. 02

    Classify

    Kraken2 compares reads with a reference database to attach taxonomic assignments.

  3. 03

    Embed

    Locally loaded DNABERT-S converts sequences into vectors; FAISS searches their nearest reference neighbours.

  4. 04

    Explore

    Distance-based novelty scores, dimensionality reduction and clustering organise unfamiliar sequences for inspection.

  5. 05

    Inspect

    The dashboard connects quality, diversity, taxonomy and embedding views with downloadable run artifacts.

01 / ENGINEERING

FASTQ preprocessing and artifact isolation

The tool-backed path runs fastp for read filtering, seqtk for sequence conversion and vsearch for dereplication before Kraken2 assignment. Run-specific directories separate intermediate files from final reports. A background worker advances the pipeline while status endpoints expose progress, streaming completed artifacts directly to the client.

02 / ENGINEERING

DNABERT-S → float32 vectors → IndexFlatL2

DNABERT-S generates sequence representations locally. Reference embeddings are loaded from HDF5, stacked as float32 arrays and inserted into FAISS IndexFlatL2. Each query returns five nearest reference identifiers and squared Euclidean distances. Exact flat search avoids an approximate-index recall parameter; the search work scales with the number and dimension of reference vectors.

03 / ENGINEERING

Novelty scoring and latent-space structure

The novelty scorer maps the minimum reference distance through 1 − exp(−d). A separate anchoring score uses exp(−mean(d)) across retrieved neighbours. UMAP or PCA projects vectors for inspection and HDBSCAN identifies density-connected groups, prioritizing candidate sequences with the highest reference divergence.

04 / ENGINEERING

Diversity aggregation and reproducible inspection

The results layer computes community summaries from abundance data, including Shannon-derived effective diversity and Hill-number views. HDF5 retains embeddings, TSV preserves sequence-level outputs and Newick represents the generated tree. The interface links quality, taxonomy, diversity and sequence projections with direct artifact downloads.

04 / BUILT WITH

TOOLS & THEIR PLACE
  • 01Python 3.11Pipeline orchestration & analysis
  • 02DNABERT-SSequence representation learning
  • 03FAISS IndexFlatL2Exact vector retrieval
  • 04Kraken2Reference taxonomic assignment
  • 05fastp / vsearchQC & dereplication
  • 06UMAP / PCADimensionality reduction
  • 07HDBSCANDensity-based clustering
  • 08HDF5 / h5pyEmbedding artifacts
AYAZ AHMED / PORTFOLIOBACK TO COLLECTION ↑