Originally published at: https://boingboing.net/2018/10/05/anonymous-sources-bold-claims.html
…
3 Likes
“Spy tech?! Nooooo. One of our workers was eating lunch on the job and dropped some rice onto the motherboards he was working on. And then accidentally programmed some unfortunate coding onto the rice. And accidentally dripped some solder on the rice. These things happen.”
5 Likes
Here’s my pick for best “anonymous source”.
Rudy knows cyber. /s
3 Likes
Fitz gives some good reasons to be skeptical, but I’d like to add one point about hardware-vs-software attacks that I haven’t seen anywhere else.
If you were pull a running-but-suspicious server out of service and discover that it had a malicious BIOS or malicious BMC application in flash, you may never be able to tell how that malware got there. It could have been included at manufacturing time, during maintenance updates, by accidental exposure to malicious media, injected remotely through some other exploit, etc. Even if you can gather evidence that the malware was included at manufacturing time (for example), you won’t be able to say if the manufacturer was malicious or if they were exploited by the people providing the firmware. Maybe a server where the firmware was stored was hacked by a third party. Everything is fog-of-war, there’s deniability at every level.
If, on the other hand, you were to pull a running-but-suspicious server out of service and discover that it had a non-BOM micro-controller in a position to cause mischief, that’s like a giant red neon arrow pointing directly at the manufacturer. The maintenance guy didn’t put that in. You didn’t accidentally download a chip over TCP/IP. There will be details you can’t prove based on this alone, but it’s much less ambiguous, much less deniable situation.
I think that’s a reason to be a little skeptical of this story. It’s not that hardware hacks are infeasible. It’s that software/firmware approaches are (sadly) plenty feasible, historically fruitful, and a better match for for the goals intelligence agencies have.
18 Likes
I’ve never seen an out of band system that didn’t rely on its own RJ45, optical, or serial connection. As such, while BMC and IPMI are handy, the obvious flaws make them very risky to use. One may avoid such exposure by employing enough techs and/or redundant hardware to not require the use of a management interface. IP based KVM systems provide “up” access and if your hardware isn’t up, connect remotely to your PDU and power cycle it. If that doesn’t get it back online send a tech down to the server room to get it back up.
The problem with this approach is cost. You’ll have to trade convenience and cost savings for security and higher personnel costs. That’s a pretty universal truth in the IT world.
1 Like
But does he know 400 lbs of cyber?
2 Likes
All of the most detailed articles I’ve read since yesterday from hardware hacking experts about the Bloomberg story (which I have read twice now), seem to end with folks saying it’s definitely possible, people have been concerned about these issues for years, the specificity of how Bloomberg described the hack is plausible, and so yes, this story may be true.
Imagine for a moment if it was true. Apple and Amazon go to the Feds. They’ve studied what’s going on, and they are concerned. The Feds instantly are very concerned, but they want to learn more about the situation. It can be contained, but monitored, with the help of the firms involved. But they will have to deny the situation vehemently, if it comes to light, or the operation to trace this back is potentially foiled. Classic cop thinking, delay busting lower-level folks so you can work your way up to the boss.
The firms are happy to go along with this, because after all, they are very in bed with the Chinese, and this allows them to not piss them off so they can keep making money from the Chinese market, can continue to get access to Chinese manufacturing. The firms’ denials, the Feds tell them, will be completely backed up by the Feds, as the investigation is ongoing.
It’s not only theoretically possible – I think if you don’t think this kind of thing is happening, when we have all our high-tech products manufactured by a country with nuclear weapons pointed at us, is insane. I am personally very open to the possibility this story is largely true. I think if SuperMicro, Apple and Amazon fail to sue Bloomberg for libel, it will be telling.
7 Likes
Fail to sue Apple? Typo?
1 Like
I think the Register’s article on this issue hits the nail on the head: Bloomberg’s reporters put two and two together and got five. This is what can happen if a journalist is too unwilling to trust what a company’s spokespeople have to tell them about an issue. Amazon and Apple did have separate, unrelated problems with malware infected drivers and untrustworthy firmware. China is trying to have backdoors and spyware installed on devices made in China, especially generic no-name phones. That’s the smoke. But the journalists chased that down the wrong direction and ended up reporting on a fire that doesn’t exist.
(Thanks to @DasKleineTeilchen for the link)
Also, bigger scale, the story makes no sense: a spying chip on the motherboard is hard to insert without being detectable. Especially one big enough to hold enough code to insert a backdoor. A patient engineer would be able to identify any chip that didn’t belong after a few hours of comparing blueprints to production circuit boards. On the other hand, inserting a backdoor into one of the several firmware chips that are supposed to be there would be easy as pie to do if you had the resources of a government spying agency and access to the factory where the chips are made.
10 Likes
The problem is that the firmware is signed so it’s not as easy to modify as you might think.
The twitter chain posted is pretty damning IMHO. If this is a hoax they had someone extremely technical on the team to make it believable, since the device is in the one place on that board where it would work as described.
3 Likes
If the firmware belongs to the company that makes the chips and not to the company that’s commissioning the construction of the device (which is the case with multiple chips even in extremely customized devices), then all the spy agency has to do is suborn a few workers at the company making the chips and they get access to the signing keys as well as to the workstation that contains the firmware source code.
For companies incorporated in and operating in a totalitarian dictatorship, like China, if the state wants to put a compromised firmware onto a chip, all the security precautions in the world cannot stop them.
8 Likes
The other issue is that your implant is in danger of being wiped next time someone flashes the firmware. You have to make the system pretend to update, but then the victim may notice that his new firmware didn’t activate the feature/fix the problem he was expecting. You also have to deal with returning fake results when someone tries to checksum the firmware or verify the signing keys. In several ways the hardware implant is less problematic, assuming the communication protocol with the chip doesn’t change.
The denials from the three companies are not a surprise. If the FBI were investigating this they would instruct the companies to keep quiet about it to avoid tipping off the perpetrators. On the other hand, these investigations are three years old, so if it’s an active case they are seriously slow rolling it.
1 Like
The two things I’ve seen repeatedly pointed out would seem to bear on this.
If this were real. Even if Bloomberg’s article was it being discovered. Apple and the other big companies wouldn’t really be able to put out the sort of detail, direct denials they have. There’d be an active intelligence investigation involved, and in that situation these companies would be restricted to giving pretty vague rote denials and “no comment” sort of responses.
And that if this was known or in some way suspected. And the MO was to study it. Then broad military and government procurement wouldnt have continued. But it did.
Its all sorts of little things like that, that don’t seem to wash. I think the big one is that this doesn’t seem to have gone through the usual pathway for publishing security vulnerabilities.
The companies don’t appesr to have been notified. There doesn’t seem to have been an embargo while they and researchers examines it. There doesnt appesr to be a fix or mitigation in place. There no white paper announcing the risks and laying them out so people and companies can avoid these things if there isn’t a mitigation.
It looks an awful lot like that “worse than meltdown” amd vulnerability that was published this way a while back. A little known security firm with public ties to an investment grouo that had a short position on the stock. Press based release without proper white paper, notofication or embargo. Justified by it being impossible to fix. Technical details thst were both vague. And while feasible, not in the way described or as bad as claimed.
And that vulnerability turned out to be trivial to fix. Despite the researchers claim that it could never be fixed. AMD apparently patched it in days.
Pretty clearly stock manipulation. And it seems like exagerating or faking security breaches in becoming a common avenue to do that.
And the pictured chips and descriptions of size and how its installed don’t appear to wash with that level of capability. I haven’t read the articles linked here yet. But the quick technical ones I ran across yesterday. It sounds like China would have to have secretly figured out how to make smaller chips, with more packed into them than anyone else out there. And they’re just using it for spying. While they license American chip designs and manufacturing tech to build a domestic processor market…
Like I said the circumstancial stuff seems awful weird.
1 Like
It doesn’t need to be a big chip if you’re talking about watching the bus lines and patching certain results. It would be pretty expensive to fab out a one-off at 15nm but not impossible for a nation-state actor.
1 Like
As long as the firmware is closed source and you have control over the company’s source code repository, inserting your malware into each new update should not be an issue. Also, making a firmware controller that has a partition between the legit firmware and the hitchiking spyware (which never gets its signature checked, does not get wiped by an update, etc) would solve all such problems quite handily.
Again, if China’s spy agency wants to cause a rootkit to be inserted into every motherboard assembled by a certain Chinese logic board maker, making a poisoned version of a chip that’s supposed to be there would work a lot better and be a lot less detectable than putting in an extra chip that’s not supposed to be there.
Keep in mind that the instant you use the backdoor you’ve had put in, your victim will see anomalous behaviour in their network logs and know that something weird is happening. If they suspect something fishy, they will examine the motherboard to determine if it is within spec, and at that point they will find any chips that don’t belong. On the other hand, they might never figure out what’s going on if the board has a chip that belongs but is poisoned.
2 Likes
That does not accurately characterize third-party expert perspectives many have posted. In fact, they go to the other extreme – the vector of attack as described int he Bloomberg article is specifically the MOST LIKELY vector for this type of hardware attack on a server, and, many security experts have been worried about this for ages.
I also don’t think companies vehemently denying anything means anything, especially in this era. Apple is the biggest company on the planet. 100% squeaky clean to get there, you think? Yeeeaaah… no.
I don’t know if this story is true, but I do know it could be. Good server admins put lights-out management interfaces on their own subnets, behind VPNs, really because these interfaces have been known to represent major security holes for a while now.
One thing we do know: A few years ago, the US Government admitted that someone, probably China, stole ALL OF THE SF-86 FORMS FOR ALL CLEARED GOVERNMENT EMPLOYEES AND CONTRACTORS. This was one of the biggest intelligence coups of all time. How did they pull that off? Had to be something really shifty, probably. And if you don’t know what an SF-86 is, go and find it online, read it over, imagine it being filled out in super-duper-detail by every single person with a security clearance, and now China having ALL of those forms. It’s a spymaster’s wet dream, and nobody is denying that this occurred (the affected even got letters in the mail from Uncle Sam, free credit monitoring, yada yada).
2 Likes
Yeah but like I said it seems like they’re claiming its doing a lot more than that. Feasible but not neccisarily exacly as claimed. And did China even have Fabs for that size at the time? These aren’t new boards. Their push for domestic CPUs is a partnership with AMD, they’re building Ryzen derivitives on licensed 14nm nodes. Pretty sure any fabs were recently built/converted. Just seems very “wait a minute…”.
1 Like
exactly this, very well put.
That’s how it read to me as well, using terms and ideas they didn’t fully understand and getting basics wrong even in non crucial parts of the story reducing its overall credibility.
same with the kernal and much of the system software, and in the bloomberg article they say they can simply alter the code that compares the login password string. lol. obviously doesn’t know how that code works or how modern OS are built and didn’t run it past anyone like a security researcher first. very bad reporting.
if they can make chips like described, they can make replacement for chips that are supposed to be there with altered instructions much more easily that are fully signed because of where they have supposedly inserted themselves in the manufacturing chain.
It isn’t that it is impossible to plant malicious firmware or hardware, just the bloomberg article gets enough wrong to require more credible accurate information from another source before believing a shred of the rest of what they are saying.
2 Likes
It’s fun to wonder what a compromised hardware manufacturer could do, but given that very little of it has ever proven difficult to accomplish through software alone, it’s hard to imagine that China would take the enormously greater risk and cost of doing it that way.
If you taint a motherboard’s firmware at the factory, you can achieve pretty much whatever you want, and it’s cheap, and you can update the compromised firmware if/when you find faults in it, and if the political situation changes you can even remove the backdoor if you need to. A hardware implant is a physical evidence trail that you can’t control once it’s out the door – that motherboard you sold to Facebook could end up in an NSA lab – and if you later find out it has a flaw that might advertise its presence, you’re out of luck.
I guess a hardware implant could do stuff like communicating via a secret radio signal, or physically disabling a server, but if all it’s doing is providing a backdoor to modify the board’s firmware, that seems like an insanely dangerous and expensive way to do that.
Anyway, if you are worried about what China can do with hardware implants, you are probably underestimating what China can do regardless (and not just on Chinese-built systems). And ultimately, there’s not much point worrying about what states can do, because it’s certainly more than bad enough. Just focus on how the US and China have managed not to have a war so far, and consider (for example, when you go to vote) what it takes to sustain that.
2 Likes
hardware attacks are possible. how bloomberg describes is impossible. how security researchers replying to bloomberg’s absurdity speculate are feasibly possible with massive state level manufacturing resources, but they don’t match the bloomberg story and are just one of many many holes in that sieve of a story.
bloomberg gets even the non-trivial details wrong. doesn’t mean there isn’t a real story underneath, just means we need credible facts and more information before we can know if a real thing happened that bloomberg just completely botched in describing, or if this a fabrication. at least that is my understanding. the register pieces are much better.
should have been a lot more skepticism and fact checking prior to any reporting on this, imho. but integrity and news are no longer overlapping vim, lol, news these days is just as exploited and exploitative! 
