Since 18 July 2026 · one backyard · Salt Lake City
An AI to talk to crows.
This started when I began feeding a murder of local crows. They were obviously
smart, obviously talking to each other, and I wanted to know what about.
CrowGPT is the set of tools that came out of it: CawDex
reads crow calls by type, the CrowGPT model says them back, and the
goal is a translator that runs in both directions.
Thrasher, one of the two parents, strutting the lawn.
Corvid species with a call-type head
6
Each detection routed to its own species' model
Alarm-call AUC on clean labels
0.97
Recording-grouped CV, one-vs-rest
Call types CrowGPT can generate
4
Alarm, generic call, begging, song
Human-labeled call audio
8.1min
The number that actually limits the project
The murder in question
The crows I feed are five: Thrasher and Ripjaw the parents, Killer and Shredder
the fledglings, and Edgar, a yearling who stayed on to help guard the brood. The
notes I was already keeping on them turned out, by accident, to be proto-labels
of the call types further down this page, which is the whole reason a recording
and a witnessed situation arrive together here instead of being reconciled weeks
later.
Edgar on the line out front, watching the street.Killer on the office roof, the friendly one of the two.Killer earlier in the summer, still half down.Two of the family on the roofline, keeping watch.
Killer, Edgar and Shredder sharing a streetlight, which is most of what they
do.
Ripjaw on the wire, waiting for the peanuts to appear.Edgar, mid-call, from the top of the brick.Edgar deciding what I am. He usually gets there first.
Killer, Shredder and Edgar working through a handful of peanuts on the office
roof.
The family fighting over peanuts on a light post. No sound, no point. Just
crows.
Digging in the gravel out front. What they were after, I never worked out.
Killer calls back when I say “bye buddies.” That is not a result. It is
why I started.
Crow → human → crow
The end state is a translator that runs in both directions. CawDex,
the call-type classifier, already works. The missing ingredient is
semantics, what was actually happening when a call was made, and I
collect that every day. Pair the two and you get a
Crow Translate. Simple calls first: dog spotted,
human spotted, food.
The other half is CrowGPT, which generates crow calls instead
of reading them. Put the two together and it runs both directions: crow → human
→ crow. Cross-species conversation, and maybe friendship, with the help of AI.
Crow Translate · listening
What did that call mean?
A recording goes in; the corvids in it are found, named, and each call
labeled by type on a timeline. That part works. Turning a type into
meaning takes the second ingredient, what was happening at the
moment the bird called, which is what I am out collecting.
CrowGPT · speaking
Say it back
A model that generates crow calls rather than reading them, on demand and by
type. Early, and getting better. It is the half that makes this a
conversation instead of a very elaborate subtitle track.
Simple calls come first, because those are the ones a backyard produces often
enough to learn from: a dog walking past, a person arriving, food going down.
Nobody is decoding crow grammar here.
Two problems hide in one question
Inside the listening half there is a second split, and keeping it straight is
the main design decision in the project. These need different machinery, and
only one of them is solved.
Solved
What are the call types?
Are the categories acoustically separable at all? That is a plain
classification problem on audio, and the answer is yes, given labels good
enough to learn from. Everything below this line is engineering.
Open
What does each type mean?
Mapping a call to “I see a dog” needs that call paired with
witnessed context, written down at the moment it
happened. No quantity of
audio alone gets there, which is why the rest of this project is a
data-collection system. This is the frontier, and it is where I am.
What it can already tell apart
CawDex sorts every call it finds into one of five classes, the same five the
app offers a recordist. Finer field labels fold into them, so a mobbing crow
is filed as alarm and a rattle as call. The colours are the ones the app and
the dashboard use. Press a dot to hear one: the best example of each that the
project has, five seconds apiece.
Alarm call
Harsh, fast, repeated, with the whole bird in it.
Something is wrong, and the other crows are being told.
Also filed here: mobbing and scold, the directed,
persistent version aimed at a cat, an owl or a hawk on the ground.
Clip: recorded with the app in Salt Lake City.
Reads it at 0.97 AUC on clean labels.
Flight call
Spaced, carrying caws given on the wing.
Keeps the group in touch while it moves: here I am, this way.
The cleanest class acoustically, and the first one the pipeline learned.
Reads it at 0.91 AUC on clean labels, 0.56 on archive tags.
CrowGPT: saying it back
The speaking half. A small transformer, 18.7M parameters, trained from scratch on
call bouts cut out of archive recordings, with
no human labeling anywhere in the loop. Ask it for an alarm
call, a generic call, begging or song, and it produces one.
Hear it
Two generations of the model. Everything under a CrowGPT caption is
generated, not a recording of a bird. Next to them, the real thing: a
field-recorded American Crow alarm call, labeled by ear, so you can judge
the gap, and the progress, yourself.
CrowGPT1: generated, the early “crowbot” stageCrowGPT2: generated, the current model with a 6-second context windowA real crow: field recording
Please don't play these outdoors. A fake alarm call could
stress real crows, so I don't play them to birds either, and the model's access
terms carry the same rule.
Two things about it matter more than the audio. It does not copy its
training set: what it produces is generated, not retrieved. And on a crow
detector, an earlier 18M version scored well ahead of a 433M pretrained audio
model fine-tuned on the same corpus. That detector also chose the training data,
which flatters the lead by an unknown amount, so a listening test gets the final
word.
The app: collecting the other half
Building a labeled set by hand does not scale, so the collection is its own
system: record corvid audio in the field with GPS, get it analyzed, and say what
was heard and what was happening. Heavy model work never sits in the
request path.
Record, then keep. The ★ is the recordist saying this one is real.
Calls typed and placed on the waveform: flight call, begging, call.
The analyzer's guess is shown, then confirmed or overruled. Only the human
answer counts.
What was happening, from a closed vocabulary. Tapped per call, not per
recording.
The witnessed situation, written at the time. This is the scarce half.
Every capture carries where it was taken, so territory questions stay
answerable.
Guesses are kept apart from observations, so being wrong costs nothing.
The app
Android, local-first. Records unprocessed 48 kHz audio and video, shows
the waveform, and uploads only when asked, so an interrupted upload is
the normal case, and every step is resumable and safe to repeat.
The analysis
Detection and call-typing run in the background, off the recording
device. Findings come back attached to the waveform, and nothing waits
on a model to finish.
The labeling desk
Where the work happens. Calls get confirmed, corrected, or marked by
ear, and each one tagged with what was going on
at the time.
The whole field kit: a shotgun mic, a dead cat, and a bag that fits in one
hand. Crows notice equipment.
Three kinds of ground truth, deliberately kept apart
Labels are what the bird said: a span plus a call type from
a closed vocabulary. They are the only thing that feeds training.
Context is what a human saw while that call was made, and it
attaches to the call rather than the recording: tag the whole capture and you
credit every situation to every call in it, so a clip where a dog walks
through and the crow later sings would report that crows sing at dogs.
Observations are free text written afterwards, to explain a
confusing prediction and to find which situations recur often enough to earn a
tag of their own.
Try the app
The field app is Android, free, and in a closed preview: it records, labels
and syncs to the project server, so a recordist anywhere can add their crows to
the corpus. Ask below and Ben sends an invite with the app and the token it
needs to join. Say where you would record and which corvids are there; a
population the corpus does not have yet is the best reason to say yes.
CawScout: where to point the microphone
One backyard produces one family's calls. Finding the next family used to be
a random dawn walk, and the random walks kept coming home empty. CawScout
replaces them with a prior: a 250 m grid over the whole valley, scored on
where crows should be and ranked into a numbered target list on the app's
map.
The trees
Crows nest and roost high, so canopy is the loudest signal in the
score. A satellite canopy-height model reads every tree in the
valley at 1 m resolution; stands of 15 m and up, about
45,000 of them, go into the grid, joined by thousands of
individually-mapped street trees, each assigned its height from the
same model. The taller the stand, the harder it pulls on the cells
around it.
The greenery
Crows forage on open turf, so parks, cemeteries, golf courses and
grass come in from OpenStreetMap, nearly eight thousand features,
with water and human food (fast food, supermarkets, picnic sites)
as their own smaller signals. Each one radiates a few hundred
metres into the surrounding cells, because a crow near a park is a
crow the park explains.
The score
Every signal becomes a heat layer over the grid and they sum by
weight, canopy first, community crow sightings second, on purpose:
the satellite measures every block equally, while sightings pile up
wherever birders happen to walk. Predators then subtract, corrected
for how well-watched the ground is. Great Horned Owls, the top
predator of crow nests, cost the most. That penalty alone knocked
a raptor-patrolled university cluster out of the top five.
The map
Numbered targets over satellite imagery, alongside every capture's
pin and community sightings of crows and each predator species:
dated, photographed, toggleable per species. A target explains its
own score and offers street view and directions, and the header says
whether right now is even a good time to be out.
Target #1 open on the map: why it scored, how to get there, and the two
buttons a visit ends with. Owls and hawks drawn where others saw them.
CawDex: reading the calls
The listening half. CawDex is the call-type classifier: audio goes in, and
every corvid call in it comes out found, named by species, and labeled by
type on a timeline. It is the index the translator will be built on, hence
the name.
Raw audio is too high-dimensional to classify directly, so every clip becomes a
fixed-length fingerprint first.
Detect
BirdNET finds the corvids in any audio or video file and names the species,
with a waveform timeline of where the calls sit.
Embed
Each clip becomes a BirdNET embedding, a fingerprint where acoustically
similar calls land near each other. Classical MFCC features and a general
audio transformer both sat near chance; a bird-specific model was the
unlock.
Pool
Mean, standard deviation and max across the clip's three-second segments, so
the feature captures temporal spread, the signature of a repeated,
insistent alarm versus a single call.
Classify
A small logistic-regression head per species maps the fingerprint to a call
type. At a few hundred examples per species, a regularized linear model
beats a deep net.
The result that matters
Two call classes stayed stubbornly weak when trained on the archives' coarse
free-text tags. Trained instead on hand-reviewed labels (3,162 segments
somebody sat and listened to, one second at a time), the same pipeline,
unchanged, lifts them decisively. The bottleneck was never the algorithm.
One-vs-rest AUC by label qualityAmerican Crow · recording-grouped cross-validation, so nothing is graded
on data it trained on
Chance is 0.50. The two classes not shown, flight call and rattle, were
already strong on weak labels; these two were the failures.
So alarm and song were label-limited, not unlearnable. The path
to strong heads is clean contextual labels, not a fancier model, which is
exactly why the rest of this project is a data-collection system rather than
more machine learning.
Where it is today
The state of play, plainly. Every limit below is a piece of work waiting, and
most of them are more interesting than the parts that already run.
Where it still falls short
Human labeling is early. 77 labels across
21 captures, plus 168 context events. Measured
as call audio rather than label count, that is about 8.1 minutes,
the number that actually limits this project.
Detected calls are guesses. Only human labels are ground
truth, and until recently there were none at all.
No call has been translated. The classifier says
alarm call; it does not say dog. Nothing here has yet mapped
a call to a meaning, because that needs enough calls paired with a witnessed
situation, and I am still collecting the first few dozen.
Recording is invite-only for now. The code is open and the
server is hosted, but the field app is a closed preview, sent out one invite at
a time.
Scope is loud calls only. Crows also make quiet soft song
that likely carries more information, but it needs a close mic and drowns in
noise. Broadcast calls are stereotyped, repeated and high-SNR, which is what
weak supervision needs.
Open to collaborators
This is a one-person project that does not need to stay one. There is more work
here than time, and several parts are self-contained enough to hand to someone
who finds them interesting.
Birders and crow people. The scarce resource is not compute,
it is somebody who can say what a bird was doing when it called. If you watch
corvids closely, you already have the rare skill.
Audio ML. CrowGPT is small and there is obvious headroom in
the generation half, and a listening test to design that is not circular.
Android and dashboards. The capture app and the labeling
desk are where collection speed is won or lost.
Anyone with corvids nearby, any species, anywhere. This is
not an American crow project. Detection already routes to a per-species model,
and there are heads for hooded crow, carrion crow, rook and common raven as
well. So hooded crows in Germany or eastern Europe, carrion crows in Britain,
rooks in a field somewhere: all of it counts, and comparing what the same
call type means across populations is a better question than anything one
backyard can answer.
The lowest-effort way in is to record. The field app is in a closed preview,
and an invite takes one email:
request access. Every capture a tester syncs lands in the
same corpus and the same labeling desk as the backyard ones, and a species or a
place the corpus does not have yet is worth more than another hour of mine.
The code is open:
github.com/benfaerber/crowgpt.
Everything on this page is in it, the classifier, the generator, the app, the
field server and every numbered experiment, with a quick start that runs on a
fresh clone. The CrowGPT weights are on Hugging Face at
faerber/crowgpt, gated
behind an agreement not to play them to wild birds. Issues and pull requests
are welcome.
Tell me what you'd want to work on:
hello located at crowgpt period org.
What gave me the idea
Two projects convinced me this was worth attempting at all. Neither is mine, and
both are further along than anything here.
Vittorio Baglione and Daniela Canestrari at the University of León, with the
Earth Species Project, put tiny recorders on wild carrion crows in northern
Spain and mapped the repertoire of 24 family groups against what the birds
were actually doing. Those crows are cooperative breeders: the young stay
home for years and help raise their siblings, which is exactly what Edgar is
doing on the roof here. It is the same question as this project, asked with
instruments.
The preprint.
The Cetacean Translation Initiative is doing this for sperm whales: tags,
buoys and drones feeding machine learning, with the stated aim of a blueprint
for another species' language. They have proposed a phonetic alphabet for
whale codas. If it can be attempted at that scale in the open ocean, it can
be attempted in a backyard.