The historic core of Taipei is just a tiny corner of the modern metropolis, in a section known as Wanhua, southwest of Taipei Main Station, near the Danshui River. The city spreads out from here, mostly to the east and north.
Central contemporary Taipei is constructed on a grid, with major streets running east-west and north-south. Taipei breaks these major streets up by direction (eg east and west) and then into sections numbered according to the distance from the central axis (Sec 1, Sec 2 etc). Zhongshan Rd forms the central Y-axis, and a combination of Civil Blvd and Bade Rd makes the central X-axis.
Taipei also has numbered 'lanes', which generally run perpendicular to the main streets. Major sights, hotels and restaurants are located along the main streets, but for more intimate places you'll probably find yourself in the lanes. A typical address is 5 Lane 260, Guangfu S Rd. Lane 260 is where 260 would be if it were a building, and 5 indicates the building number within the lane.
Alleys are to lanes as lanes are to streets: it sounds complicated, but it's actually quite logical.
Taipei proper is divided into 12 districts (Zhongshan, Xinyi, Beitou etc). Occasionally the names also correspond to districts of interest to visitors, but usually they're most useful as parts of postal addresses. More than likely, however, visitors will concentrate on neighbourhoods within these districts, notably Dinghao (east of the city centre), Shida (the university district to the south), Shilin and Tianmu (to the north) and the mountain Yangmingshan, which dominates the northern city limits.