In 1980, Elder Neal A. Maxwell summarized the critics:
you may even see a few leave the Church who cannot then leave the Church alone.
Let these few departees take their brief bows in the secular spotlight; someday they will bow deeply before the throne of the Almighty, confessing that Jesus is the Christ and that this is his work.
Meanwhile, be unsurprised if, as the little stone seen by Daniel rolls relentlessly forth, some seek to chip away at it (see Dan. 2).
In 1996, he discussed the topic again.
Church members will live in this wheat-and-tares situation until the Millennium. Some real tares even masquerade as wheat, including the few eager individuals who lecture the rest of us about Church doctrines in which they no longer believe. They criticize the use of Church resources to which they no longer contribute. They condescendingly seek to counsel the Brethren whom they no longer sustain. Confrontive, except of themselves, of course, they leave the Church, but they cannot leave the Church alone (see Conference Report, Oct. 1980, 17; or Ensign, Nov. 1980, 14).
Like the throng on the ramparts of the “great and spacious building,” they are intensely and busily preoccupied, pointing fingers of scorn at the steadfast iron-rodders (see 1 Ne. 8:26–28, 33). Considering their ceaseless preoccupation, one wonders, Is there no diversionary activity available to them, especially in such a large building—like a bowling alley? Perhaps in their mockings and beneath the stir are repressed doubts of their doubts. In any case, given the perils of popularity, Brigham Young advised that this “people must be kept where the finger of scorn can be pointed at them” (Discourses of Brigham Young, sel. John A. Widtsoe [1941], 434).
Therefore, brothers and sisters, quiet goodness must persevere, even when, as prophesied, a few actually rage in their anger against that which is good (see 2 Ne. 28:20). Likewise, the arrogance of critics must be met by the meekness and articulateness of believers.
https://www.churchofjesuschrist.org/study/general-conference/1996/04/becometh-as-a-child?lang=eng
