It could be geosmin.
We encountered geosmin once before in our butter but most people notice it, about this time of year, in their tap water. Geosmin is a volatile compound that's made by soil bacteria like Streptomyces, as well as some plants like sugar beets, and cyanobacteria. It gets blamed quite often for making water taste like dirt. Only a small amount is required (10 nanograms per liter) for detection by the human palate (1) and it's been blamed for adding a "dirt-like" taste to many things like water, wine, apple-juice, and catfish. So, late in the summer when everyone is wondering why the water tastes like dirt, you can surprise them. Just sigh dramatically, and say, "It must be the geosmin."
References:
1.F. Pollack and R. Berger. 1996. "Geosmin and Related Volatiles in Bioreactor-Cultured Steptomyces citreus CBX 109.60." Applied and Environmental Microbiology 4:1295-1299.
2. D. Cane and R. Watt. 2003. "Expression and mechanistic analysis of a germacradienol synthase from Streptomyces coelicolor implicated in geosmin biosynthesis." PNAS 100:1547-1551.