Do go! I'd consider it a bucket list item. I visited the inner market a few times in the late 1990's and early 2000's, which was before the tourist restrictions. Sounds like @pkjames has the current access info. In addition to the football field of frozen giant tuna (no wonder bluefin stocks are threatened), you'll see stalls filled with all manner of ocean creatures, both frozen and alive in tanks. Was fortunate enough to watch a couple of workers carve a bluefin with one of their giant tuna swords.
After about a decade hiatus from visiting Tokyo, I went again in 2013. We weren't able to get into the inner market (got there too late), so we consoled ourselves with a fresh sushi breakfast. Heard rumors that the fish market may be relocated. The surrounding area has become prime real estate with new highrise office buildings cropping up. We stayed at the Conrad at Shiodome which is literally a stones throw away. Staying close by would be helpful as the subways aren't running yet at 2:30am.