Puppy eating everything that fits in his mouth?

I have a 12 week old Shiba. When I take him outside I have to scan the puppy safe backyard for any fallen leaves, bugs, etc. otherwise he will be eating it in a second. He is like a vacuum cleaner out there. I don't have toxic plants, etc. around. So is this OK and will he grow out of being a vacuum cleaner? PS. any paper, pebble, worm, etc. will get swallowed if I don't catch it.