Chaff is the dry, inedible sheathe of cereal grain that must be removed before the latter can be used to make bread and other foodstuffs. Thus, the process of “separating the wheat from the chaff” has been a common metaphor for dividing things (or, often enough, people) that are worthwhile from those of no value.
The expression turns up several times in the Bible -- in Psalm 1, for example, which refers to “the wicked,” who “are like chaff that the wind blows away”; or in Matthew 3:12, where the Lord’s “winnowing fork” will “clear his threshing floor and gather his wheat into the barn, but the chaff he will burn with unquenchable fire.”
This illustration depicts men threshing with hand flails, in a Ransomes, Sims & Jefferies Ltd. (a British agricultural machinery firm) lithographed advertising poster, circa 1875.
Here, in the Unbearable Lightness, the editor and Tomas's son visit Tomas after his firing from the hospital, when he is making ends meet by being a window washer, and ask him to sign a petition demanding the regime release all political prisoners. His son calls it "separating the wheat from the chaff," by showing who is brave enough to sign the petition and defy the regime.
Tomas realizes his signature, though somewhat prestigious as being that of a person who has already defied the regime, would not be as effective as the signature of someone who has not yet stood up to the authorities. But such a person would of course be less likely to sign because it would only call down trouble on his own head. Tomas also recognizes that bringing further attention to political prisoners by demanding their release would, by calling attention to them, only make it less likely that they would be released.
So he is on the fence about it, not because he is afraid of the regime (he already has almost nothing to lose), but because the gesture would be at best futile, and at worst, counter-productive. Separating the wheat from the chaff seems rather beside the point.