The Fingers is a region on the east coast of the Vale of Arryn, located along the shores of the Narrow Sea. It consists of five long, slender peninsulas extending into the sea, divided by bays, inlets and channels. House Baelish rules over the smallest of the Fingers.
Petyr Baelish, who holds a keep in the smallest of the Fingers, is popularly called 'Littlefinger' for his small size during his childhood and the location of his family's lands.
In the books
In the A Song of Ice and Fire novels, the Fingers are lonely, cold and windswept. The Andals landed on the Fingers during their invasion of Westeros and weathered stones bearing the symbol of the Faith of the Seven can be found dotted along the shore in some areas.
Following Sansa's escape from King's Landing, Littlefinger takes her first to the Fingers. Sansa is stunned to see what a miserable, forsaken place it is.
In "House Baelish (Histories & Lore)", Baelish says that the Fingers consist of four peninsulas, but the World of Ice and Fire sourcebook (2014) confirms that there are actually five fingers (the title "Lord of the Five Fingers" was used by the rulers of an Andal proto-kingdom). "Littlefinger" Baelish is from the smallest of these, the northernmost, nearest to the Three Sisters islands. The mistake is easy to make: the peninsula is so "little" that it isn't clear just by looking at a map if it is actually considered a "peninsula" or just a variation of the shape of the coastline. It also was never stated before the World book exactly how many peninsulas make up "the Fingers".
Mountains of the Moon · The Fingers