Allen is the son of Allen and Janie Conner, born on July 5, 1915. He married Annie Mae (born December 2, 1918).

Allen and Annie Mae had eight children: Daisy Conner Davis (born July 31, 1937), Rebecca Conner McFaddin (born October 10, 1939), Annie L. Conner Peterson (born January 31, 1941), Mary Conner Johnson (born August 5, 1943), Shirley Conner McCain (born December 31, 1946), Carrie Conner (born August 2, 1950), Hanna Conner Stynler (born February 27, 1952), and Allen Conner, III (born November 8, 1954).