This can damage the bearing but you could use a pair of needle-nose pliers. Im not sure if i spelled that right but you get the idea. So basically just get a pair of pliers and tug at it, it should pop out rather easily with pliers. If that doesnt work, well im all out of ideas. Good luck!
Actually, that doesnt damage the bearing, it only damages if you somehow scratch it. That is the correct thing to do. YYJ's are made so the bearing doesnt come off easily, so just do that to get it off. Thats what I do.