Some designs I thought about for data storage systems involve a dead man's switch combined with a perimeter alarm. If the perimeter is violated, or the machine is attempted to be moved or manipulated with without proper authorization, a timeout starts. If the timeout expires without valid code being entered, or if a duress code is entered, the data inside is destroyed. It may be a big thermite charge that melts disks, a small one that just burns a flash chip, or a non-pyro solution entirely where a crypto key for the disks gets overwritten in the SRAM of the key storage chip.
Difficult to reach for a destroy switch when a bunch of testosterone-poisoned guys aim guns at you. Less dfficult to just do nothing and let the machine do its job. The system also works if the location gets compromised when you aren't present.
The lightweight style with key forgetting can be augmented by storing the key backup somewhere securely. Possibly split to parts, and stored in custody of people in different jurisdictions. Possibly without them even knowing what they hold. Hairy Potter's "horcrux" is a similar concept.
The machine itself must be mechanically designed in a way that slows down entry/dismantling enough for the timeout to expire. Tack welds and epoxy potting can do a good job here. The chassis can also serve as TEMPEST-grade shielding so the key won't be leaked (easily) via EMI.
The rest is a common computer security.
The machine itself can be two machines, where one is exposed to the outside and handles the I/O and is considered untrusted, and only the inner one handles the red data, and is connected to the outer one via some interface that exposes bare minimum of the kernel and libraries (and therefore the potential bugs). Proper balance has to be striken between the security and the ease to use (which aren't necessarily mutually exclusive, but require some creativity to not be).
It's easy to be a tough guy for a few seconds. The system should not require more from the operator. Then, once the operator cracks under pressure, and few don't, there's nothing he can do anymore, especially if the holders of the key fragments are instructed to not cooperate even with him under such scenarios. A m-of-n key split scheme provides a degree of safety against both modes of key storage node failure (loss, improper divulging).