Can you have too many visual references?   The simple answer is yes.  The idea is to focus on the the message and present it as clearly and precisely as possible.   You need to be concise with your point and ensure it can be understood as quickly as possible without a lot of thought. If people are taking to much time to understand what you diagram is or you have to many you have lost your audience.