I had this situation two years ago. Except he was in a climber about 20 yards from my stand. I let him know I wasn't happy and walked him off the property. He gave me is name and apologized. He did lie saying it was a one time thing. I pointed out how chewed up the bark on the tree was.
I put a charge on him for hunting without permission. When we got to court I told him I would agree to under advisement for 6 months, stay off my property and pay court costs, 10 hours of community service and it can be dismissed. He was agreeable to it and the Judge accepted it. Haven't seen him again.
If you let a trespasser go it is an open invitation to trespass on your property.
I have prosecuted several over the years, some of whom were very bad people and not trespassing to hunt but to do far worse things.