The general rule in surf travel is this; The cheaper you pay, the more crowded it is.
Unfortunately, crowds are part of surfing but there are certainly ways to avoid them and there are still areas you can go or times of year that are less busy than others.
Getting off the beaten path:
Generally, the closer one stays to an international airport, the busier it will be. For example, The Male Atolls in the Maldives has seen a spike in popularity for surfers around the world over the last 5 years. More flights, cheaper boats and land camps popping up mean the crowd factor is higher than it used to be and a lot higher than other parts of the Maldives. If however, you pay a little bit more for a boat that is willing to travel a little further afield OR you jump on a short domestic flight, you could be surfing perfect empty waves for your entire trip. These areas generally have less accommodation options for surfers, therefore, limiting the crowd factor.
Parts of Indonesia also requiring a flight, stopover hotel, ferry and a bus quite often mean fewer people will be in the water. These places still exist, you’ve just got to earn the right by taking on those harder yards.
Avoiding busy periods in the year:
Avoiding busy times in the year is a no-brainer for a surfer who wants to escape crowds. Booking your surf charter right in the middle of school holidays is perhaps the dumbest thing you can do unless it's to an area where the accommodation supply is thin, like the Outer Atolls of the Maldives.
School holidays also means the airlines jack their prices up. Flights can be 20-50% dearer in school holidays!
If you want to take advantage of public holidays, be aware that others will also have the same idea and therefore these dates can book out very quickly. And flights are often 20-50% dearer than normal.
If you have the flexibility to avoid holidays altogether, then do so.