Actually, to engage before the velvet is gone to risk injury to the antler. Once it is shed, the antler is hard and they begin light sparring immediately. Actual hard fighting does not occur until quite a bit later. However, antler angagement of some form may continue right up until the antlers are shed.
Antler engagement is not all about fighting or dominance or even territory. It can be and is also a friendly act between two or more bucks. I have seen as many as eight bucks taking turns sparring. So many hunters do not understand antler engagement and therefore do not have positive reults.
Population makeup is an important factor. But during certain periods, buck/doe ratio is of no consideration at all. It is far to lengthy and complicated to explain here but equate it to boys on a school playground. At some point, the playfull shoving may result in an actual fight. This is heightened when a female enters the equation.