The scope of this question runs from 1866 with the defeat of the Austrians in the Austro-Prussian war of the same year through to the split of Germany in 1945 after Hitler’s defeat. As a side note, the ‘little’ German nations, such as Hanover, Saxony, and Bavaria were not inconsiderable nations in their own rights. That Prussia, through the will of Bismark, was able to unify them under one flag was due in part to a nationalist zeal ignited in the post-Napoleonic period, and also in part to the declining influence of Austria/Austria-Hungary in the corresponding period. There was a desire by the rulers of the smaller constituent parts of the Germanies to avoid outright conquest by Prussia, as well as a desire to enact social reforms without a revolutionary bloodbath that had swept up France in 1792. The 1830 and 1848 revolutions stoked German liberalism and nationalism, and it took all of Bismark’s political acumen to convince the disparate German nation states that the Prussian constitution and military might were worth binding under, vis-a-vie Austrian hegemony (which is why the Prussians fought the Austrian-Prussian war).
Prussian desire for hegemony arose after the 1815 Vienna settlement in part because Prussia feared Austrian, French, and Russian encirclement. The treaty had two outcomes that would shape the rest of 19th century German history: 1) The establishment of the German Confederation, made up of 39 independent states, and 2) the treaty had ceded territory on the Rhine to Prussia. This had put Prussia under considerable internal strain, as geographically, culturally, and politically the west of the new nation was far removed from the Berlin Prussian monarchy.
Napoleon’s sparking of nascent German nationalism during his conquest of the Rhineland was fanned by the liberalization of Europe during the period 1820-1848, which the Austrians under Metternich attempted to suppress. Prussia was kept in check by Austrian military and political supremacy during this period, but as Austria lost political prestige Prussia began to reassert herself, especially when Otto von Bismark became Prussian Chancellor in 1862. Prussia, with the support and aid of Austria, fought the 1864 Second Schleswig war against Denmark, which gave them control Schleswig-Holstein. Prussia then went to war with the German Confederation, led by Austria, in 1866, which led to the defeat of the Confederation and then to it dissolving as a political entity. In 1867 the Prussians unified the northern German states into the North German Confederation, which had a federal constitution with Prussian monarch as the head of state.
The German empire was formally declared at the Palace of Versailles on 18th January 1871, at which point the independent principalities and kingdoms of the North German Confederation, led by Prussia, merged with the south German kingdoms and principalities. At this point the various independent entities that had formally been part of the Germanies became states within the new German Reich, with their own laws, customs, and state legislatures. Central authority in the empire rested with the Emperor and the Reichtag, though the democracy that Bismark, the Prussian prime minister, envisaged was of a limited form, with ultimate power resting in the hands of the German Kaiser, who was also the King of Prussia.
Initially the 27 constituent states that formed the newly unified Germany were left to manage their own internal affairs without too much interference from Berlin, as Bismark did not wish to empower the Reichtag with too much power. Over the course of the period between 1871 and 1900 the German state centralized federal control in the hands of the civil service through the State Secretaries, with the effect that even though there was near-universal suffrage in Germany for most adult males, the power of their elected representatives was muted when compared to the bureaucrats who controlled the affairs of state at the behest of the Kaiser.
The effect for the 27 state rulers was that their pre-1871 powers were constrained by the 1871 German constitution. All states sent deputies to the the upper house called the Bundesrat, and although all states were nominally equal in the eyes of the law, as Prussia was the largest state in terms of both population and size they had 17 of the 58 seats, which meant they had little difficulty in obtaining the majority they needed to pass favorable legislation. One key area the 27 states maintained their own control was the armed forces, so that the larger states, such as Prussia and Saxony, had their own armies, which in turn was modeled on the Prussian army. This had the consequence that during WW1 many regiments were led, or had a titular head, by members of the royal families from the various states.
In terms of the pre-1871 monarchs, Saxony’s monarch Fredrick Augustus III abdicated in 1918, turning Saxony into a free state in the Wiemar republic; the same happened with Bavaria, and by the declaration of the armistice on 11th November 1918 all the monarchs who remained in power after 1871 had been removed from power in favor of republican ideals. The reasons for the loss of power are varied, ranging from a general disdain for support of Prussian policies during WWI (in the case of Bavaria and Saxony), to a ground-swell of desire to reform the country after the catastrophic defeat caused by the allies. The monarchical principals of power were discredited in the eyes of the German public by the actions of the ruling elite during the war, even to the point where the Bavarian’s sought a separate peace with the allies.
Due to this, many royal titles were abrogated in 1918, with all temporal power being transferred into the hands the people. The Wiemar government did not remove smaller titles, though, and those landowners, including the Prussian Junker class, were still classed as nobility through the Wiemar and Nazi governments. The fall of Germany in 1945 resulted in a complete overhauling of the governance of Germany in the East and West, with the nobility loosing all their privileges in the process. It is also worth remembering that many of the Prussian noble houses were absorbed into present day Poland and the Baltic states, were their estates were seized by the Communists and turned over to state control.
So, in summation: Most of the German monarchs, Dukes, and princelings of the pre-1871 era were allowed to remain as heads of state of their respective German state (much like an US governor). Due to the actions of the various monarchies during WW1 when the German Republic was declared in 1918 the states transitioned from monarchies and hereditary rulers to directly elected state legislatures, and the former monarchs either went into exile or were settled into civilian life.
2. The Army of the German Empire 1870-1888 by Albert Seaton.
3. Contesting the German Empire 1871 – 1918 by Matthew Jefferies.