The title was created after Aegon I Targaryen unified the Seven Kingdoms in the War of Conquest. While kings of the Targaryen dynasty ruled from the Iron Throne in King's Landing, the heir to the throne would traditionally rule over the ancestral Targaryen lands: the volcanic island of Dragonstone, located on the far side of Blackwater Bay from King's Landing itself.
Dragonstone was usually ruled by the Crown Prince. In practice, if the heir apparent had not yet reached the age of legal majority, a regent (usually a blood relative) would administer Dragonstone in the interim - but the under-aged heir would still nominally be Prince of Dragonstone. If a king left behind multiple sons when he died, and the eldest had no children when he succeeded his father as king, the next oldest son would become Prince of Dragonstone, as his brother's legal heir. He would yield the title as soon as his older brother produced an heir.
The Targaryen dynasty maintained the title of Prince of Dragonstone for the heir apparent because the island was their home territory. After Robert's Rebellion, the new Baratheon dynasty did not apparently continue use of the title. Robert Baratheon did give rule of the island to his younger brother Stannis, but there has been no mention that he ever used the title, nor is there mention that the title was ever used for Robert's (alleged) first son Joffrey.
Technically, the title became "Princess of Dragonstone" if the heir apparent was ever female, however this hardly ever occurred during three centuries of Targaryen rule. Rhaenyra Targaryen was the one major exception, but her succession was controversial and sparked a civil war with her younger brother. Generations later, after the exile and near-extinction of House Targaryen, Viserys Targaryen technically became Prince of Dragonstone as the legal heir to his nephew Jon Snow, and after Viserys' death, his sister Daenerys theoretically became Princess of Dragonstone as the legal heir to her nephew Jon. Technically, Daenerys still holds the title, but she and also most of the people in The Known World, including Jon Snow himself, are still unaware of Jon's real heritage and his legal right to the Iron Throne, therefore, after his brother's death Daenerys allegedly inherited his claim as the Targaryen monarch. Daenerys had before her Invasion of Westeros only briefly been on Dragonstone, when she was born there. She was secreted off the island mere weeks later and has no memory of it.
In the books
In the A Song of Ice and Fire novels, the title dates back to the two sons of Aegon the Conqueror, Aenys and Maegor. Because Aegon had no other sons nor any daughters, for many years Maegor was the heir of his older brother Aenys. Because King's Landing was still under construction, Aegon spent the next four decades of his reign on royal processions across the realm, and he took Aenys with him. In contrast, Maegor stayed with his mother Visenya at the old Targaryen seat on Dragonstone - and as a result, over time people took to informally calling Maegor "the Prince on Dragonstone". Years later, Aenys succeeded his father upon his death and Maegor became his Hand of the King. Maegor later outraged the Faith of the Seven by taking a second wife in a polygamous marriage, however, and in punishment Aenys exiled him to Pentos for five years. As a further punishment against Maegor, Aenys declared that his eldest son Aegon was the new "Prince of Dragonstone" - officially making it the title of the designated heir to the throne, and stressing that his own children would inherit the throne ahead of his brother (while Andal custom is that a lord's children inherit ahead of his younger brother, Valyrian custom was more complicated, and some argued that Maegor should be Aenys's direct heir anyway). Therefore Aenys's son Aegon was the first official "Prince of Dragonstone", though the title unofficially originated with Maegor. Eventually Maegor died and Aenys's youngest son Jaehaerys I succeeded him, but it isn't clear if Maegor ever named Jaehaerys I as his heir (apparently, Jaehaerys I fled into hiding after Maegor killed his two older brothers).
Most kings were usually Prince of Dragonstone before ascending the throne, but a few unexpected deaths upset the line of succession. Probably the most prominent example of this is among the heirs of King Daeron II Targaryen. His eldest son Baelor died in a tournament accident, so he was succeeded by his second son Aerys I. Aerys I never had children so Daeron II's third son Rhaegal was heir, but Rhaegal was medically insane, so Aerys I was succeeded by Daeron II's fourth son Maekar I. Maekar didn't particularly like Dragonstone, however, so he chose Summerhall for his holding as his brother's heir, and simply never claimed the title "Prince of Dragonstone". Maekar himself had four sons, yet his eldest son Daeron later died of a pox, then his second son Aerion Brightflame died drinking Wildfire. A Great Council was held to determine who the new king would be. Maekar's third son Aemon had already joined the Order of Maesters and considered himself ineligible to succeed - though there were those who offered him the throne anyway, because as king he could simply release himself from the vows, but he did not wish to be used as a political pawn against his younger brother, so he refused the throne (and then joined the Night's Watch to further discourage any who wished to use him as a rival heir). Maekar was thus succeeded by his fourth and youngest son, Aegon V. As he was the fourth son of Maekar, who was himself a fourth son, he was called "Aegon the Unlikely". Aegon V was therefore never Prince of Dragonstone before he ascended the throne.
It isn't clear exactly what happened to the title after the Baratheons overthrew the Targaryens. Stannis was given Dragonstone to rule, though he would have preferred Storm's End, because it was a hotbed of Targaryen loyalists that needed a firm hand to rein in. A few years later, however, Robert's marriage (apparently) produced a son, Joffrey. Neither Stannis nor Joffrey, however, are ever referred to as the "Prince of Dragonstone", despite either ruling the island or being the Crown Prince. For that matter, after the War of the Five Kings breaks out, Joffrey's younger brother Tommen becomes his legal heir, and Stannis's daughter Shireen become his heir, yet neither of them is then referred to as the Prince or Princess of Dragonstone. Apparently it was considered specific to the Targaryens, because the island was their ancestral holding, and it seems to have been dropped after they were deposed.
Technically the title became "Princess of Dragonstone" if the heir apparent was ever female, however this hardly ever occurred during three centuries of Targaryen rule.
The one prominent example of a Princess of Dragonstone was Rhaenyra Targaryen, who as her father's eldest surviving child was officially the Princess of Dragonstone for many years. After her father died, however, her younger brother Aegon II tried to seize the throne from Rhaenyra. Both died in the resulting civil war, after which the royal inheritance laws were altered to put female heirs behind all possible male ones, i.e. allowing an uncle to succeed ahead of his older brother's daughter.
Daenerys is referred to as the "Princess of Dragonstone" several times in the first book (most often in her own mental narration) - up until the point when Viserys dies, and she inherits the position of Targaryen monarch and Queen. For example, when she is frightened during her wedding to Khal Drogo (or when she is frightened by Viserys in general), she tries to reassure herself in her inner POV thought monologue by repeating that she is a descendant of Aegon the Conqueror, that she is Princess of Dragonstone, and that "the dragon's daughter" should not be afraid.
The use of a specific princely title for the heir to the throne parallels real-world monarchies: the heir to the ruling monarch of the United Kingdom (and before that, the ruling monarch of England), is granted the title "Prince of Wales", while the heir to the throne of Spain is traditionally the "Prince of Asturias".