Willie Joe Patmon comes from a lineage of farmers dating back to slavery. Born into slavery, Willie's great grandfather Henry Patmon acquired land in Georgia and continued farming following emancipation. Willie's grandfather, Joseph Patmon, eventually moved to Oklahoma in the 1920's and bought 200 acres of farmland. It was on this family farm where Willie was born to Bose and Mary Patmon in 1934.

As the eldest of 17, Willie had significant responsibilities on the family farm growing up. Despite not being able to attend school until the last crops were in each year, Willie still excelled in his coursework graduating at the top of his class from Douglass High School in Crescent, Oklahoma. Following high school, Willie attended Langston University where he majored in Industrial Arts and Machine Design.