Japan’s temples and shrines, to me, are far more special alone. New Year’s Day crowds can be interesting too, but visiting a temple or shrine with loads of tourists isn’t usually my thing.
Here is a tip if you are on Koyasan. Some temples, like Kongobuji, close at 5. Dinner at most of the lodgings is at 5:30 or 6. Most people head back to their lodgings before 4 to relax before dinner. You are better off relaxing during the middle of the day when the places are crowded and experiencing the temples between 4 and 5 (or early in the morning) when they are empty.
Such was the case when I walked up through this gate to Kongobuji after 4 p.m. (passing many tourists scurrying back to their temple lodgings). I was the only one there!