Mate cornered me at a wedding in Galway. Will not let it go

Mate cornered me at a wedding in Galway last Saturday. Two pints in. Outside smoking area, drizzle. He goes lads I know what ye are at. I just laughed but he was not joking. Background. Eddie I have known since secondary, twenty years. He got divorced two years ago. Knows my wife knows my house knows my kids. Now single forty one and has somehow figured out we are in the lifestyle, says he saw something on my wife's phone last summer when she left it on the table at a barbecue. Did not tell anyone he said, fair play. But now he wants in. Single male wants to join us specifically because he trusts us and we trust him. I said no immediately. Wife agrees no. But he keeps. Texts every few days. Hey lads how are ye. Pints soon. Then drops a line like I have been reading up actually. He is being respectful and friendly but he is not letting it go. Stoneybatter house, kids 7 and 10, herself works HSE I am self employed. We have a small group of three other couples we trust completely after eighteen months of building it. Single male in is a no for our setup not because of him personally but because of group dynamics. How do you say no to a friend without losing the friend. Because right now Eddie is being weird and I cannot tell if he is hurt or just not getting the message. Pints with him Thursday in Smithfield.