This is a reference to the Old Testament story of David and Goliath, from the First Book of Samuel. Saul, the king of Israel, should have battled the giant, but he was afraid and offered a reward to any man who defeats Goliath. Young David volunteers, and Saul offers him his armor, but the boy goes off to battle with just his sling and five stones. The Bible story is intended to show how Saul is not a fit leader for his people, while David might be.
Since Major Major has been suddenly promoted by an IBM machine, the sergeant can no longer “beat hell out of any man in his outfit,” because he would be assaulting a superior officer, so he broods like Saul.