It is usually the first place people look. You already have an IRA or 401(k) at Fidelity, Vanguard, or Charles Schwab, gold is making headlines at record prices, and you wonder whether you can simply buy coins or bars inside the account you already own. The short answer is mostly no, with one limited exception, and the reasons say a lot about how the retirement industry is built.
The Short Answer for Each Brokerage
At Vanguard, you cannot hold physical gold in an IRA at all. Vanguard does not custody precious metals and does not even offer its own gold ETF. Gold exposure there means third-party ETFs, gold mutual funds, or mining-stock funds.
At Charles Schwab, the answer is the same. Schwab IRAs can hold gold ETFs, gold mining stocks, and gold futures for approved accounts, but no physical bullion or coins.
Fidelity is the partial exception. Through a program run with FideliTrade, an independent Delaware metals firm, a Fidelity IRA can hold a short list of metals, specifically Gold American Eagles, Gold American Buffalos, Silver American Eagles, Platinum American Eagles, and bullion-quality bars, per Fidelity’s own precious metals page. It is real physical gold, but it is not what the industry calls a Gold IRA. You cannot choose your custodian or depository, the coin selection is narrow (no Canadian Maple Leafs, for example), phone orders only, and fees include a purchase charge of 0.99 to 2.90 percent depending on size, a selling charge of 0.75 to 2.00 percent, and storage billed quarterly at 0.125 percent of value.
Why Mainstream Brokers Do Not Custody Physical Metals
This is not an oversight. Federal law treats precious metals in retirement accounts as a special case. Under Internal Revenue Code section 408(m), collectibles are prohibited in IRAs, with a carve-out for certain coins and for bullion meeting minimum fineness standards (99.5 percent for gold), and only if the metal is held by the IRA trustee, not the owner. The IRS IRA FAQ covers the collectibles rule.
Meeting that trustee requirement means vaults, insurance, armored logistics, audits, and per-item recordkeeping. Mainstream brokerages are built around electronic securities that settle in two days and cost fractions of a basis point to hold. A warehouse full of coins is a low-margin, high-liability sideline for them, which is why Vanguard and Schwab skip it entirely and Fidelity outsources it to a third party with a restricted menu.
What this means for you: the brokerage saying no is not saying gold is a bad idea. It is saying physical custody is not their business model.
Paper Gold vs a Physical Gold IRA
What the big three offer instead is paper gold, most commonly ETFs backed by vaulted bullion. The differences matter more than marketing on either side admits.
With a gold ETF you own shares in a trust that owns gold, not gold itself. You cannot take delivery of metal, and you accept a layer of structure between you and the asset. In exchange you get instant liquidity, no dealer premiums, and low carrying costs, typically 0.25 to 0.40 percent per year in expense ratios. With a physical Gold IRA at a self-directed custodian you own specific coins or bars in a depository, can take in-kind distributions of actual metal in retirement, and face no fund structure at all. In exchange you pay dealer premiums of roughly 3 to 8 percent when buying, plus custodial and storage fees that commonly total $175 to $600 per year.
Neither is objectively better. The ETF is cheaper and simpler. The physical account delivers direct ownership, which is the entire point for investors who distrust financial intermediaries in a crisis.
How to Move Brokerage IRA Funds Into a Physical Gold IRA Tax-Free
If you decide you want the physical version, you do not have to leave Fidelity, Vanguard, or Schwab entirely. The standard route has four steps. First, open a self-directed IRA with an IRS-approved custodian that handles alternative assets. Second, request a direct trustee-to-trustee transfer from your existing brokerage IRA. The money moves between institutions, never touches your hands, is not taxable, is not reported as a distribution, and has no 60-day deadline or once-per-year limit. Third, choose IRA-eligible metals through a dealer once the funds arrive. Fourth, the custodian ships the metal to an approved depository in the account’s name.
The common and expensive mistake is taking a distribution instead, cashing out with the intent to “buy gold yourself.” A distribution from a traditional IRA is ordinary taxable income, can push you into a higher bracket, and adds a 10 percent penalty if you are under 59 and a half. Gold you then buy personally cannot be added back into an IRA, since IRS rules require IRA metals to be purchased with funds already inside the account. If anyone advises you to take the check, walk away.
When a Gold ETF at Your Broker Is the Smarter Option
Honesty requires saying this clearly, sometimes the dead end at your brokerage is a useful stop sign. A gold ETF inside your existing IRA is likely the better tool if your intended allocation is small (under roughly $25,000, where flat fees bite hardest), if you value one-click rebalancing inside an existing portfolio, if you are within a few years of RMDs and want effortless partial liquidation, or if you simply want price exposure rather than ownership of metal.
The self-directed physical route earns its costs when the position is large enough to dilute the fees, when direct ownership and the option of in-kind delivery genuinely matter to you, and when you want your gold outside the securities system rather than inside it.
The Bottom Line
Vanguard and Schwab do not offer physical gold in IRAs at all, and Fidelity offers only a narrow, phone-order program through a third party. That is a structural feature of the brokerage business, not a verdict on gold. If paper exposure is enough, your current broker already has you covered at lower cost. If you want actual coins and bars with tax-advantaged status, the path is a self-directed IRA funded by a direct transfer, done custodian to custodian so the IRS never sees a taxable event.
This article is for educational purposes only and is not financial or tax advice. Consult a qualified financial or tax professional before moving retirement funds.
