Skip to content

CLAP-based AI plugin providing (1) semantic similarity scores and (2) nearest-examples retrieval, plus optional caption proposals.

Notifications You must be signed in to change notification settings

orcasound/orca-clap

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 
 
 

Repository files navigation

orca-clap

Contrastive Language-Audio Pretraining (CLAP) model that recognizes similarities between audio-text pairs, ingesting Orcasound community reports and Orcahello moderator annotations as training data. Primary applications include natural language search of audio files, and annotation assistance.

Proposed layout:

  • python/ (inference server—tiny FastAPI with /embed, /score, /nearest)
  • tools/
    • node-text-audio-pairs/ (Node utility for generating pairs)
    • shared CSV schema samples
  • models/ (download/readme scripts, no weights checked in)
  • docs/ (usage + examples)

APIs this repo should expose:

  • POST /score → { audio_url | wav:bytes, prompts: [text...] } -> { scores: [{prompt, sim}], version }
  • POST /nearest → { audio_url, k } -> { neighbors: [{audio_id, sim, meta}], version }
  • POST /caption (optional/MVP later) → { audio_url } -> { caption, evidence_neighbors }

About

CLAP-based AI plugin providing (1) semantic similarity scores and (2) nearest-examples retrieval, plus optional caption proposals.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published