If we're really digging in to the meaning of "Not Really" here, then I would interpret it to mean that he felt the same way I felt when a person I knew and used to work with got sent up for something very similar: There were some behaviours that at the time I thought were a bit strange, but didn't really dig too much in to at the time, since everyone is a little bit weird in their own way. In retrospect, it all looks perfectly clear about what was going on, blanks get filled in, dots are connected and all that, but at the time, really, there was nothing to it.
For the first little while after I found out, I felt guilty; I thought that I really should've inquired more and done something, been better at figuring out something was happening, or said something to someone, or done something. But back then, it really didn't cross my mind that something of the sort could've been going on. Should I be evaluating everyone I meet as a potential abuser?
Bah, now I'm thinking about it, and it's making me angry/sad.
Anyway, if you'd asked me, soon after the news came out, if I had had any ideas that it was going on, I might have said "Not Really", because there was (still is) a part of me that thought at the time that yes, I should have, and that I should have done something about it and helped kids to not go through whatever it was that was done.