Yes, that's one of the things that helps narrow the list down, although most of the people engaging in annoyed gossip are being polite enough to not specifically name names. He can't be completely sure that all of the people being complained about are actually involved in the cult, as opposed to just rude guests choosing a bad time to vacation, but those 35 are the people being complained about.

It's certainly possible for them to follow this up by investigating all those people in the context of a custom mission, either with just the two of them or with additional characters brought in to help. If you prefer to get the list of names to be ICly shared, that's also an option, as is handwaving checking out all of those people to narrow down the list to those who appear the most suspicious.