I have basically no experience with any machine vision, I’ve done some edge/color detection once, but my intuition says that sounds like a plan. Because you have a pretty unique feature for each brick, its number of studs and size of cavity on the bottom, you might not need as full a picture of each brick as you might think. That’s pure speculation though. Have fun with the project.