They eat the snot out of them.
In addition, another fruit deer eat during the summer that most hunters never think about is wild cherries. I've noticed that many of the cherry trees are really loaded this year. They will ripen and fall in July.
Deer also eat wild plums and muscadine grapes in September. Those can be early bow-season hot-spots.