In the 15th century, the Rum-Seljuks had landed on the Italian peninsula and conquered most of Naples in the Great Napolitan War. During 1459-61, Naples was completely subjugated by the Seljuks. Europeans feared that the Papal States would be attacked next. The helpless pope fled to Avignon, as did king Charles / Carlo V.
In 1466, the infamous Sacco di Roma happened. The Seljuks invaded and smashed the outnumbered Papal troops.
In the following years, the church lost a lot of power to the various state governments (whether monarchical, republican or something in between). Castille-Portugal sent troops to Italy to fight for the pope.
1472: After lots of fighting, the Seljuks kept Latium, calling themselves from now on "rulers of both Romes" despite the fact that the sultan was disappointed how insignificant Rome had become. The eastern parts of the Papal states became the Duchy/Protectorate of the Marches, theoretically still under the pope, de facto under the duke of Alba (Castille). Although none said it aloud, the pope and the other church leaders were quite content in Avignon and didn't care that much about Rome any more.