Something I'll mention is that association of colors with numbers and letters and such is a relatively common form of synesthesia, which is, as best as my ****ing stupid ass can explain, is basically a mixing of senses and/or mixing of different concepts (shapes and colors in this instance).
I have no idea where all these number associations come from in my mind but I swear I’ve had them since I was a child