Born in Gandersheim, Gille began his military career as a first lieutenant in the artillery branch during the First World War and won the Iron Cross First and Second Classes. He left the army in 1919 and remained a civilian working in agriculture and as a car dealer until 1931 when he joined the Nazi Party and the SS. He married Sophie Charlotte Mennecke on 4 January 1935 and his only child, a daughter, was born on 9 October 1935.

In 1934 he was re-activated by the SS combat support forces. He became a Company Commander in Ellwangen, then a Battalion Commander of the SS regiment Germania in Arolsen. He later served as the commander of an artillery unit in Jueterbog. As the commander of the 1st Battalion of the SS-V Artillery Regiment Gille participated in the invasion of Poland and in the western campaign. In 1940 he took over the artillery regiment of the 5th SS Panzer Division, led by SS Obergruppenführer Felix Steiner.

After the assault on the Soviet Union, Gille, as a leader of an advance guard, reached the Kuban and received the Knight's Cross on 8 October 1942. Shortly thereafter he took command of the Wiking Division (5th SS Panzer Division) on the Eastern Front. Early in 1944, Gille was instrumental in the withdrawal of his command and others of the encircled Group Stemmermann through "Hells Gate" during the Korsun cauldron encirclement, also known as the Cherkassy Pocket. The Soviets greatly outnumbered the German forces but they failed to cut off their retreat, though they were able to inflict serious damage on the German formations. Gille received the diamonds addition to his Knight's Cross with Oak Leaves and Swords on 19 April 1944. Shortly after his escape from the Cherkassy Pocket, Gille and members of his staff were flown into the encircled town of Kovel to organize its defense. Under Gille's steady leadership the mixed army and Waffen SS units maintained a vigorous defense until the siege was raised by approaching German units from the West.

 

 

His troops stood strong on the East Prussia border with the 3rd SS Panzer Division and prevented the planned Soviet breakthrough to Berlin in the autumn of 1944 destroying large numbers of Soviet tanks. In January 1945 Gille, as leader of the IV SS Panzer Corps comprising the 3rd and 5th SS Panzer Divisions, was sent to Hungary to attempt to relieve the encircled city of Budapest. However, his troops were unable to break through to the city. In March 1945 he led the IV SS Panzer Corps in the failed Lake Balaton Offensive and following the Soviet counter-offensive his corps was forced to retreat into Austria. When the end of war was clear, he marched towards the U.S. troops in order to avoid surrendering to Soviet forces. He was held by the U.S. for three years, and released in May 1948. Despite being an early Nazi Party member, Gille was known for his apolitical views.

The author Heinz Höhne in The Order under the Death Head characterized Gille as an enigma and "Nur-Soldat" (soldier - nothing else) who once threatened a newly assigned Weltanschauungsoffizier (political indoctrination officer) with a clean-out squad to gather his uniforms and other possessions and throw them and the officer out of the unit.