cali sober couple in sydney scene, fitting in or not

gday from sydney, were a couple in our 40s who dont drink at all but do enjoy a bit of weed (legal grey area but whatever). been to a handful of meet n greets in the inner west and the vibe is VERY winery-and-cocktails over here, like every social is at a wine bar. we feel a bit alien, sipping our soda waters while everyone else has had three glasses by 8pm. weve also noticed people get a bit suss when you decline a drink twice, like youre not "fun". any other california-sober or just plain sober aussies in the lifestyle out there? does it get easier or are we trying to plug into the wrong scene entirely...