After World War II,
most Germans were expelled to
Allied-occupied Germany, including Marquardt who had to leave in July. Frauenburg's
cathedral chapter then elected the aged Canon Johannes (Jan) Hanowski, a German of Polish ethnicity and long-term archpriest of
Allenstein (today's Olsztyn), as
capitular vicar, i.e. provisional head of the see, on 28 July 1945. Kaller, who had been stranded by the end of the war in
Halle upon Saale, made his -long way back to his see and arrived on one of the first nights of August 1945 in Allenstein/Olsztyn, taking on the jurisdiction from Hanowski. He started to develop new plans for his diocese especially aiming at overcoming the nationalist antagonism between Catholics of the German and Polish languages, reshaping the diocese in the spirit of German-Polish reconciliation. Kaller further appointed an ethnic Pole as new
cathedral provost, since his predecessor provost, Franz Xaver Sander (also
official), and five more fellow cathedral canons had been killed by the invading Soviets. (The other killed canons were Andreas Hinzmann, Dr. Franz Heyduschka, Dr. Wladislaus Switalski, Anton Krause and Dr. Bruno Gross.) Addressing the Polish authorities in the annexed area of his diocese, Kaller declared that he wanted to continue his episcopate within
Poland, but officials said it was for neither him nor them, but Warsaw to decide that. Afterwards, in a private conversation, Hlond urged Kaller to resign which he did for the jurisdiction in the Polish-occupied diocesan area, but retained the office of Bishop of Ermland, which rather turned quite void, especially since in the Soviet-occupied diocesan area no Catholic ecclesiastical activity whatsoever was tolerated. Later in Poznań, Hlond praised Kaller for how he had complied with the demanded resignation from jurisdiction. On his way back, accompanied by Borowiec, Kaller cried and told him that the jurisdiction in the Polish-occupied diocesan area would be passed on to
Teodor Bensch, a German-born naturalised Pole, who would arrive within days officiating as apostolic administrator. They returned home in the evening on 16 August. Kaller could not appoint the four new canons for the chapter any more but was expelled the next day, transferred by lorry to Warsaw, accompanied by Borowiec, who also joined him on the train to Poznań on 18 August. Then Borowiec, who had not been expelled, returned to the diocese, while Kaller had to leave via
Stettin for Allied-occupied Germany. ==Kaller's last years==