PokéSound

Overview

PokéSound builds a unique ambient soundscape for every Pokémon. Search for any creature and hear it, or take a short listening quiz to find your “Spirit Pokémon,” the one whose sound matches your ear.

The idea was simple: don’t hand a Pokémon a playlist, synthesize it a portrait from its own data. Charizard comes out as crackling fire, volcanic rumble, hot wind, and a distant roar. Gengar is eerie whispers, creaking floorboards, a low heartbeat, and wind howling through a cave. And the same Pokémon never sounds quite the same twice.

What’s live: soundscapes for every Pokémon, a 7-round Spirit Pokémon quiz, an audio-reactive visualizer, per-layer mixing, dynamic share images, and proper CC credit for every sound used.

Problem

Pokémon come with a mountain of attribute data (types, stats, habitat, legendary status) and nobody had turned any of it into something you could actually feel. Fan communities live and breathe this lore, but the apps built around them only ever show you numbers and pictures.

That’s the opening. These creatures already have vivid identities. The hard part is mapping their data to sound so it lands as intentional and evocative rather than random, all while staying inside the limits of free APIs and what a browser can do with audio.

Goals

Generate a distinct, coherent soundscape for any of the 900+ Pokémon with zero manual curation
Make the result easy to share
Do all the audio work in the browser (no server-side rendering)
Stay under Freesound’s 2,000 req/day limit through aggressive caching
Work beautifully on mobile, since that’s where things get shared

Tech Stack

Concern	Technology
Framework	Next.js (App Router) + TypeScript
Styling	Tailwind CSS
Audio	Tone.js + Web Audio API
Visualization	Canvas API
State	Zustand
Deployment	Vercel
OG Images	@vercel/og
Testing	Vitest (unit) + Playwright (E2E)

Architecture

The Mapping Engine (`lib/mapping.ts`)

This is the heart of it. It turns Pokémon data into Freesound search queries and a set of mix parameters.

Layer architecture: Every soundscape is five audio layers playing at once, each with its own job:

Layer	Role
Base	Foundational ambient bed (loops continuously)
Texture	Environmental detail (loops continuously)
Accent	Characteristic punctuation (triggers every 5 to 15s, driven by Speed stat)
Rhythm	Percussive element (loops or triggers periodically)
Atmosphere	High-frequency shimmer, wide stereo (loops continuously)
Epic (bonus)	Cinematic drone/choir, Legendary and Mythical Pokémon only

Type to tag mapping: All 18 Pokémon types map to a curated set of Freesound tags per layer. Fire’s base layer pulls from ["fire", "crackling", "lava", "furnace"]. Ghost pulls from ["dark ambient", "cave", "dungeon"]. These live in data/type-tags.json and feed the Freesound queries.

Dual-type blending: Roughly 60% of Pokémon have two types. The primary type drives base, texture, and atmosphere; the secondary type drives accent and rhythm. A Water/Psychic Pokémon gets ocean waves as its bed and meditation bells as its accent, a clean crossover with nobody hand-tuning it.

Habitat blending: When a habitat is known, the engine doesn’t overwrite the type, it blends. The first two type tags and the first two habitat tags get OR’d together. A fire-type that lives in a cave searches fire OR crackling OR "cave ambience" OR underground, so it keeps its type identity and picks up the room it lives in.

Stats to mix parameters: The six base stats control how the layers are mixed, never what sounds get chosen:

Stat	Controls
Speed	Playback rate (0.8× to 1.3×): urgent vs. lazy feel
Attack	Accent layer volume
Sp. Attack	Atmosphere layer intensity + stereo width
Defense	Base layer volume: heavier vs. lighter foundation
Sp. Defense	Low-pass filter cutoff: brighter vs. darker overall sound
HP	Layer density: high HP Pokémon have fuller, denser soundscapes

The Audio Engine (`lib/audio-engine.ts`)

Tone.js wires up the whole Web Audio graph:

Freesound Preview MP3s
        │
        ▼
Tone.Player (per layer, looped)
  ├── Base       → Volume → Panner → Filter → Reverb → Destination
  ├── Texture    → Volume → Panner → ┘
  ├── Accent     → Volume → Panner → ┘  (intermittent, Speed-driven)
  ├── Rhythm     → Volume → Panner → ┘
  ├── Atmosphere → Volume → Panner → ┘
  └── Epic       → Volume → Panner → ┘  (Legendary only)
                                  │
                          Waveform Analyser
                          FFT Analyser
                          (feeds canvas visualizer + particles)

The engine runs on a small state machine (idle → loading → ready → playing → paused → stopped), exposes per-layer mute and volume, hands a waveform and an FFT analyser to the visualizers, and tears everything down cleanly so audio nodes don’t leak when you navigate away.

Freesound Proxy (`app/api/freesound/route.ts`)

Every Freesound call goes through a Next.js API route so the API key stays on the server; the client never touches Freesound directly. Responses are cached in memory for 24 hours per query string, which keeps usage well under the 2,000 req/day ceiling.

The soundscape orchestrator (lib/soundscape.ts) falls back in tiers, per layer:

All tags OR’d together, plus a tag:loop filter
All tags OR’d together, no loop filter
Each tag queried one at a time

So even a weird tag combination comes back with something.

Spirit Pokémon Quiz (`lib/spirit-quiz.ts`)

The flow in reverse. Instead of picking a Pokémon to hear, you take a 7-round A/B listening quiz. Each round plays two short clips and asks which one hits harder. Four rounds test type affinity (fire vs. water, psychic vs. dark, and so on); three test stat preference (sparse vs. dense, fast vs. slow, bright vs. muffled).

Your picks build a listener profile, a vector of type affinities and stat weights. At the end that profile is scored against a pre-computed index of all 151 Gen 1 Pokémon, rewarding type and stat alignment, and the closest match is revealed as your Spirit Pokémon.

Key Technical Challenges

1. The autoplay policy

Browsers won’t create an audio context until you interact with the page. Every play action runs through Tone.start(), which has to fire inside a click or tap handler. The UI just leans into it: there’s always an explicit Play button, and nothing ever starts on its own.

2. Freesound tag query semantics

Early on, tags were joined with spaces ("shore lakeshore river bank"), which Freesound reads as AND, demanding every term match and usually returning nothing. The fix was to join everything with OR and quote multi-word phrases ("river bank"). A buildQuery() helper enforces that across all layers.

3. Levelling the clips

Freesound clips are mastered all over the place, which meant jarring volume jumps between tracks, worst of all in the Spirit quiz where you’re comparing two clips back to back. Every quiz player now runs through Tone.Player(-6 dB) → Tone.Compressor(threshold: -18, ratio: 4:1) → Tone.Limiter(-3 dB), closing the gap between loud and quiet sources without any pre-processing delay.

4. Non-looping clips in looping layers

A lot of Freesound clips aren’t tagged as loops and don’t loop cleanly. Setting loop: true on Tone.Player produces clicks when a clip has DC offset or a mismatched start and end. The engine spots those clips and crossfades across the loop boundary to smooth them out.

The search autocomplete kept rendering behind the Pokémon card grid because both parents sat in the same z-10 stacking context. Elevating the search section wrapper to z-20 gave the dropdown a higher effective stacking context than the grid below it.

Features Built

Soundscape generation for any Pokémon across all 18 types, with dual-type blending, habitat blending, and the epic layer for legendaries and mythicals
Audio-reactive visualizer: a circular waveform ring drawn from Tone.Waveform data, colored to the type and glowing
FFT-reactive background particles: type-themed shapes (flames, bubbles, leaves, bolts) whose size, speed, and glow ride the real-time FFT data
Per-layer mixer with mute toggles and volume sliders, plus animated equalizer bars while it plays
Spirit Pokémon quiz: 7 A/B rounds, a profile builder, and a matching pass against the 151-Pokémon index
Dynamic share images via @vercel/og: 1200×630 cards with the sprite, name, and type
Web Share API with a clipboard fallback
CC attribution: a collapsible credits panel listing every sound’s title, author, license, and link (required for CC-BY/CC-BY-NC)
Autocomplete search with keyboard navigation and ARIA combobox semantics
Type-themed pages: background gradient, particle color, and waveform ring all driven by the Pokémon’s primary type

Testing

206 tests across 9 files:

Suite	Tests	Coverage
`mapping.test.ts`	52	All 18 types, dual-type blending, habitat blending, legendary/mythical, stat edge cases
`audio-engine.test.ts`	42	State machine, playback, layer control, waveform/FFT analysers, dispose
`spirit-quiz.test.ts`	28	Round structure, profile building, matching algorithm, edge cases
`freesound.test.ts`	18	Cache hit/miss/clear, quality filters, error handling, missing API key
`soundscape.test.ts`	10	Full pipeline for Charizard/Pikachu/Gengar/Mewtwo, partial failure resilience, attribution
Playwright E2E (4 files)	56	Landing page, soundscape player, Spirit quiz flow, responsive layout (Desktop + Mobile Chrome)

Design Decisions

Dark and atmospheric: The whole app is built for immersion. Each Pokémon page uses a type-derived gradient (fire → orange-950/red-950, ghost → purple-950/violet-950, and so on) with floating type-themed particles behind the sprite. It should feel like a stage, not a dashboard.

Type-colored card borders: The popular grid uses ring-2 borders in each Pokémon’s primary type color (orange for fire, yellow for electric, cyan for ice), so the grid reads as alive and tells you something before you’ve even clicked.

Deliberate randomness: The generator pulls 5 candidates per layer from Freesound and picks one at random. That’s why the same Pokémon shifts slightly every visit: a choice made on purpose, to reward hitting regenerate and coming back.

Mobile first: Sharing happens on phones, so the sprite, controls, and mixer are all sized and spaced for thumbs, with safe-area padding for notched devices.

What I’d Do Differently

Pre-compute the original 151 at build time and serve them as static JSON. That drops the Freesound calls entirely for the most common searches and makes the app viable at scale without paying for an API tier.
Fill the curated sound bank. A bank for 8 priority types (data/curated-sounds.json) is scaffolded and wired up in soundscape.ts, but the first population run hit Freesound’s rate limit. Finishing it would sharpen the soundscapes and lean less on the live API.
Add evolution-chain crossfades. Hearing Charmander’s soundscape slowly morph into Charizard’s, with the Tone.js players fading between them, would be a genuinely great moment.

Outcome

PokéSound shows that structured Pokémon data can work as a compositional system: the Mapping Engine is a deterministic score, and Tone.js performs it fresh in the browser on every visit. The result feels both algorithmic and alive, with enough variety to keep pulling you back.