So the rubber eye piece on the SLCs is glued to the eye cup assembly, which you can purchase separately. I just assumed they were removable for cleaning purposes. I ended up losing both of mine.
I called Swarovski and told them what I did. Which is when I found out they weren't designed to be removable. The Swaro rep was super nice and shipped me two new eye piece assemblies free of charge. Great company, give them a call.
He did tell me for future reference, if I ever had one come off again, I could just glue it back on (remove the eye piece assembly first!) Now that I know the rubber eye cup isn't supposed to come off by itself, I don't anticipate it happening again.
Editted to try and be more clear:
Replacement Eyecup (Don't purchase until you've talked to Swaro.)

The entire eyecup assembly is meant to be threaded off for cleaning. The rubber end cap is glued to this assembly.