I've noticed this occasionally with peach stones - AFTER eating the peach, of course. I don't know anything about the process by which the original seed is formed by the previous plant, or what could get inside it at that stage to lay eggs, but once the seed is formed and starts to grow, it's very tightly sealed watertight and airtight all round, allowing nothing into the flesh, and nothing from the flesh into the stone. I'd have thought that as long as the flesh itself tasted good, and clearly wasn't foul or rotten, you won't come to any harm.