Is there a physical Ultimate Edition?

No. Ultimate is digital only. If you prefer a box, buy the Standard Edition (a download code, no disc) and add the upgrade from the store later.

Checked by the GTA 6 Guide against Rockstar, Take-Two, official stores and the press. Sources for each fact are on the credits page.