It's a while since i went but the Checkpoint Charlie museum is a must. Full of all the contraptions that people used to escape from East to West. Hot air balloons made out of old raincoats, hollowed out Trabants, etc. Amazing place.
You're surrounded by remnants of WWII in the centre of the city. Hitler's concreted-over bunker. SS barracks. Bullet holes in the walls of old buildings around the centre. Red Army grafitti on the walls of the Reichstag.
The old TV tower in East Berlin is worth a visit. It's a bit like the BT tower. You can go right up to the top and get a 360 degree panoramic view of the whole of Berlin. Think it might be free as well.
Not far from there is Orianburgstrasse which is good for bars. You can also see old rail lines that used to transport people to Dachau in this district.