Normally, when we read, we do so by recognizing words through the shapes made by letter combinations. When you reduce readability, you make readers work harder. They can't read as fast because they have to look at each letter. It can get quite frustrating to attempt to read long passages of text through busy backgrounds or low contrast. Just try reading dark green type on a red background or yellow type on a light blue background. Or any words set against a background image that has words in it.