This is what I found on Japanese knife laws on Wikipedia:
With the exception of any type of switchblade, any knife with an overall length of 15 cm (about 5.9 in), with a blade length of no more than 6 cm, is legal to carry. Knives must be concealed and should not be easily available. The Japanese Guns and Knives Control Law forbids knives being used as defensive weapons, but is relatively tolerant toward knives which can be opened using a single hand. For example, any type of butterfly knife is legal.
With the exception of Japanese Katana, any other swords are regarded as offensive weapons. However, if they have artistic value, possession is legal as long as the sword is registered.
Violations of the law come with a sentence of up to one year in prison and a fine of ¥300,000 or about U.S. $2,600 or about U.K. £1,300.
I carried a SAK too while in Japan and never had a problem, but I'd advise you keep the SAK in your travel bag or camera bag when you're out and about and you should be fine. And be sure to put it in your checked luggage when you leave. I've lost a few good knives at the airport security check point.
