James M. Goss was born in Owen County, Indiana. He enlisted in the Third Iowa Regiment, Infantry Volunteers, during the Civil War. He was captured at the Battle of Shiloh and held as a prisoner of war. He was discharged by reason of disability in 1863 and returned to Gosport, IN. Early in 1865 he enlisted in the 149th Indiana Volunteer Infantry and was appointed hospital steward. In August 1865, he was promoted to Assistant Surgeon of his regiment. He graduated from the University of Louisville Department of Medicine in 1872.