John Chrysostom, on 1Cor 14:23

And to show thee that he introduced the term "sign" here, not as a thing which of course did some good, he added that which resulted from it. And what was this? "They will say," saith he, "that ye are mad" This however not from the nature of the sign, but from their folly. But when thou hearer of unbelievers, do not suppose that the same persons are in every case intended, but at one time they which are incurably diseased and abide uncorrected, and at another they which may be changed; such as were they who in the times of the Apostles admire the mighty things of God which they hear of; such as in the case of Cornelius. His meaning accordingly is this; that prophecy avails both among the unbelieving and among them that believe: as to the tongue, when heard by the unbelieving and inconsiderate, instead of profiting by it, they rather deride the utterers as madmen. For, in fact, it is to them but for a sign, i.e., in order to astonish them merely; whereas they who had understanding used also to profit by it: with a view to which the sign was given. Even as then there were not only certain who accused them of drunkenness, but many also admired them as relating the wonderful works of God. It appears then that the mockers were those without understanding. Wherefore also Paul did not simply say, "they will say that ye are mad," but added, "unlearned and unbelievers."

Having said then, that a tongue profited not, and having again qualified this statement by turning the charge upon the Jews, he proceeds to signify that it even doth injury. "And wherefore was it given?" That it might go forth with interpretation: since without this, it hath even the contrary effect among them that are without understanding. "For if," saith he, "all speak with tongues, and there come in unbelievers or unlearned, they will say that ye are mad;" as indeed even the Apostles incurred the suspicion of being drunken: for "these men," it saith, "are filled with new wine;" but it is not the fault of the sign, but of their unskilfulness; therefore he added, "unlearned and unbelievers," to show that the notion belongs to their ignorance and want of faith; for, as I before said, his object is to rank that gift not among things that are disparaged, but among those which do not greatly profit, and this, in order to repress them, and bring them to a necessity of seeking for an interpreter.