Heat shock proteins (HSPs) were originally identified as stress-responsive proteins required to deal with proteotoxic stresses. Besides being stress-protective and possible targets for delaying progression of protein folding diseases, mutations in chaperones also have been shown to cause disease (chaperonopathies). The mechanism of action of the “classical”, stress-inducible HSPs in serving as molecular chaperones preventing the irreversible aggregation of stress-unfolded or disease-related misfolded proteins is beginning to emerge. However, the human genome encodes several members for each of the various HSP families that are not stress-related but contain conserved domains. Here, we have reviewed the existing literature on the various members of the human HSPB (HSP27), HSPH (HSP110), HSPA (HSP70), and DNAJ (HSP40) families. Apart from structural and functional homologies, several diversities between members and families can be found that not only point to differences in client specificity but also seem to serve differential client handling and processing. How substrate specificity and client processing is determined is far from being understood.