I think it would be a bad idea. I haven't done it. I had a coworker give me his number once and we texted and just THAT caused problems and everyone gossiping and then just making up stuff. And then if things didn't go how the other wanted them to go, you'd still have to see them every day and put up with working with them. My brother was messing with this crazy ass chic we work with and he ended things, now she's trying to start shit and get him fired and is going around showing everyone pictures of his dick on her phone and telling people he's saying things about them and all kinds of childishness and drama and crazy stuff. She would just stare at him all day like a retarded lunatic until they finally moved her to a different area. Nope, I wouldn't do it. I wouldn't even let my man start working somewhere if I was working there. People don't want to see other people happy in a relationship, it seems. At least where I work.
I had a fling with my boss a few years ago... and despite having the conversation at the time of "If this ends, we won't let it ruin our working relationship"... it ended and it completely ruined our working relationship. Just lucky for me I had transferred to another team by that stage. I wouldn't make that mistake again!
I wouldn't recommend it. I just saw a newly appointed supervisor get terminated because she was cheating on her newlywed husband with another co worker (whom is also married) that's 20 years older than her. Once the Supervisor's husband found out she was doing the coworker, he flipped and caused a scene at work, ultimately getting her fired and him suspended temporarily.
Never understood shit like that. Why shit where you eat, the dick/pussy is never worth losing your livelihood not to mention betraying your spouse.