Scared? Not really. Kinda depends on some stuff.
At home, my family knows I'm into games. Big time. Been playing them all my life, I went to school and got a degree in game design, and for a time I worked for Electronic Arts. (Currently no longer working in the game field, but I'm hoping to transition back.) It's no surprise to them that, given my love for games, I'm into some pretty niche stuff. Imports kinda weird them out since I don't speak Japanese. My recent rush over the last few months to import a bunch of iM@S games (seven of them, as well as a few other JP games like Toradora Portable and Saki Portable) had them asking all the sarcastic questions like "Have you learned Japanese yet?" or "Do you know what they're saying yet?", or the occassional comments like "How do you even know what you're doing?" or "How did you even find out about this?", but otherwise they don't care much.
At work I'm using a generic iM@S wallpaper (black Project-iM@S logo on a white background, very basic and low-key) and so far no one's asked about it. If anyone does, I'll just say it's from a game, but I really hope they don't dig further than that. It would be really awkward to explain. Would I be scared? No. Would I be embarrassed? Probably. But that's only if it happens and I'm happy having the background in the mean time.