In my readinng about this over the last few days
here is a link to a Japanese programm ( translated ) which examines how much the Japanese knew about being bombed back into the stone age.
Quite a lot and all done with call-signs ( sig-int I think )
http://ex-skf.blogspot.jp/2013/08/special-post-for-august-15-part-1.html
[ I am not sure if the Japanese broke any American codes -
there was no counter intelligence by the Japanese on whether their own codes had been broken [yes, extensively] because they assumed it would be too difficult ]