Should we end affirmative action?

Many people, predominately whites disagree considerably with affirmative action, and believe that we should end it, however the conflict is, if we are to end it then we have to be able to talk openly about the race issues we face here in America which most Caucasians are usually not willing to do. Racism in the most obvious overt forms, that existed in the sixties usually don't exist to day. Its a general belief that, that kind of behaviour is unacceptable. However latent racist practices are still at work, and we have yet to address them, such as a white man with a felony conviction is more likely to get a job call back than a black man without one, a white male is 4x more likely to be in possession of an illegal substance than a black however a black is 6x more likely to get to get stopped and search(department of justice statistics), whites with a college degree make on average 20 to 60k more a year than blacks with the same degree and same level of experience... The list goes on, and this is not to put a guilt trip on anybody but I would like to know that if Caucasians are so determined to end affirmative action because they think its unfair then how should we correct the current discrepancies in the system?