However, for all IRAs, IRS Publication 590 states that custodians or trustees must be an entity permitted because of the IRS, a financial institution, a personal savings and loan Affiliation, or an insured credit rating union. Storing gold together with other precious metals in the account