Vegetable rinse/soap products aren't particularly necessary. Vegetables won't have the oily coating skin does/can.
You do, however, need to scrub it. It's honestly a bit of a Catch-22 because if you don't also clean the vegetable brush every time, guess what's now a bacterial buffet?