Pearl is a gigantic store (an entire building, in fact) that offers everything from origami paper to portfolios; they sell hundreds of different types of paint, drawing utensils, and other art supplies. If you are overwhelmed by the bounty of goodies, their helpful staff will show you where to find whatever you are looking for. There is an elevator and restroom by the back of the store, and the frame shop, with a fantastic selection of posters, is right across the street behind the building.
308 Canal Street
Between Church Street and Broadway
A,C,E, J, N, R, 6 to Canal Street (M, Z sometimes)