In other words, make it an architecture that accurately reflects your probabilities of the world.
Similarly, make it an architecture that can be flexible to update with your priors.