To have your cap in your hand and therefore to have your head uncovered is a mark of respect and also of subordination. The idea of a cap as a begging bowl into which coins can be dropped may also be present. A North American version of this expression is hat in hand.
Related Idioms :
if the cap fits wear it
used as a way of suggesting that someone should accept a generalized remark or criticism as applying to themselves.
Early examples of this saying show that the cap in question was originally a fool's cap. The variant if the shoe fits, wear it is also found, mainly in North America.