If you plan to travel between cities in Japan, the Japan Rail Pass is almost certainly worth it. The pass gives unlimited travel on all JR trains — including most shinkansen (bullet train) lines, except the Nozomi and Mizuho services — for seven, fourteen, or twenty-one days.
You must purchase the pass before arriving in Japan. Buy a voucher through an authorized agent in your home country, then exchange it at a JR Travel Service Center once you arrive. The pass is only available to visitors on tourist visas. You choose the start date when you activate it, so there's no rush.
Once activated, the pass covers JR intercity trains and JR-operated local lines like the Yamanote and Chuo lines in Tokyo. It does not cover private railways such as Tokyo Metro, Seibu, Keio, or Nagoya's Meitetsu line. For those you'll need separate tickets or an IC card like Suica or Pasmo.
Even without the rail pass, Japan's train system is straightforward. Ticket machines at stations have English instructions, and station staff are generally helpful. Most larger stations have tourist information counters with maps and English-speaking staff.