The closet thing I've discovered is a bond on the Australian stock exchange, titled ASX:AEBD
It's invested in the Australian federal government, several provinces of Canada (Ontario, BC, and Quebec), and a few other places I don't recognize:
The fund manager (Betashares) describes the fund as follows:
AEBD aims to track the Bloomberg Australian Enhanced Yield Ethically Screened Composite Bond Index (Index), before fees and expenses. Bonds in the Index are screened to exclude exposure to activities associated with significant negative environmental, social and governance (ESG) factors
Though not municipal, this same company also manages another fund (GBND), whose description says
GBND aims to track the performance of an index (before fees and expenses) that comprises a portfolio of global green bonds (using the definition applied by the Climate Bonds Initiative), issued specifically to finance environmentally friendly projects
The GBND fund appears to only include the EU and European countries (namely France, Italy, and Germany), but the mention of Climate Bonds Initiative might be key to discovering similar funds that invest in actual green cities' bonds