The "mud" taste isn't actually from mud, but from geosmin, which is a metabolic byproduct of cyanobacteria (blue-green algae). It has an earthy or "dirt" flavor that people can detect at extremely small concentrations, as small as 5 parts per trillion.
Since cyanobacteria thrive in very warm, fertile (muddy) water, it does tend to be associated with muddy water, but it's not the mud itself that you're tasting.
There are ways to cure the geosmin flavor even if you don't have a clear water source where you can purge your fish.
Geosmin breaks down to an odorless, flavorless compound under acid conditions, so you can get rid of the geosmin flavor with acid-based marinades that include lemon, vinegar, or other acidic elements. There are some fish that I soak in orange juice overnight.
In addition, geosmin (and the associated methylisoborneol compound) are primarily stored in the fish's skin and dark muscle tissue, so you can get rid of a lot of the problem by carefully removing all of the skin and any fatty darker meat close to the skin (especially on catfish).
bd