HP's Z2 Mini is a tiny workstation

They are typically sold barebones; but back in the day, all the cool LAN Party kids had Shuttle PCs. Looks like they still make them much as they used to.

Not an absolute joy to carry(~33cm long, 22cm wide, 20 high, you’ll want a bag); but pretty much the smallest thing that will take your choice of CPU and GPU(Including full height double width cards; constrained mostly by the PSU). If you need a bit more storage, the SZ170R8 V2 somehow crams 4 3.5in HDD bays into a similarly sized package.

There are some gamer-oriented design choices(not so much with Xeon and ECC support; more use of Realtek NICs than my snobbery prefers); but the design is unobtrusive enough, and it will work with all but the most power hungry CPU and GPU options.

Worth a look if you need some mobile punch.