Hah, good comments, you know your stuff. I have not done much hardware hacking myself but worked on a team where this was done and did not forget the experience. The obfuscation at the time meant to me “easy pickings”, post-mortem… I ended up taking a very dim view of pretty much all hardware systems out there (and still have a few annoyances, for instance, with router “security”).
And yes, it was easy pickings for exactly that reason, the obfuscation of it, the obscurity of it. It was clear quite often this was code which was rushed to market with little to no qa whatsoever, and also clear though that it was in that near perfect area of places to find vulnerabilities because it was possible to be ubiquitous or powerful of an attack and at the same time complicated enough to get to that likely others would not find it and duplicate it or protect it. Which leaves hardware companies and the buyers and “idk wtf is under this hood but this puppy is golden” marketing.
IMO, I strongly suspect some of these systems have had direct or indirect pressures from some intel agencies to keep the floor this dirty.
(I have seen as a software security supplier employee and working at telcos how the government can get what they want done because of the contracts they hold.)
Not sure, however if the situation will change much to becoming more transparent out of security concerns, though this trend is likely inevitable either because of some future yet to happen major security event… or because of the ever growing expansion of DIY and opensource type software/hardware models.
A consequence of Moores Law and the idiocratic security state. In which every computer needs little computers to ride on it like a fleas on a dog, and the weenies in charge have never heard that obscurity ≠ security.
Now there’s this (ipmi):
The thing I really REALLY hate about the current cyberwar is that the participants have no intention of cleaning up after themselves. They mindless disseminate attacks and give no thought to defense.
We have been trained by thousands of years of conventional conflict to regard attack as the key to success. But, there is no possibility of long term success in internet attack. It is easy and trivial to create attack. We can easily create attack much faster than we can deploy effective defenses. You can’t use an attack without giving it away to your enemies. You can’t maintain effective attack capability if you don’t practice it on your potential enemies. The end-game of internet attack is to destroy the internet and all interconnected computing devices. There is no other outcome.
If we wish to survive as a technological species, we have to systematically defund and discourage internet attack. We have to treat it like biological warfare. We have to erect diplomatic and economic sanctions against attack. We have to give up on the idea of retaining attack capability against our enemies. You can’t create an effective internet defense without giving it to everybody.
If we don’t stop our own people from attacking, they will destroy us. It doesn’t matter who they attack. The result is destruction for everybody.
IPMI is hardly a recent development, but it has been obscure enough to fly under most peoples RADAR. Check it out, folks! Secondary firmware and CPUs which have made most computers vulnerable to remote “administration” or even lockout without the OS or user being aware of it.
Is there a synopsis, please? Or a transcript? 45-minute talk is fairly long to sacrifice uninterrupted focused-to-listening time to…
My infrastructure is littered with homebrew fragments of this technology. Computers that can reset or power off/on other computers, supervise their activity by harddrive lights and/or serial consoles, switch from primary to secondary internet by the means of a relay on the ethernet cable, some of the operations in-band via SSH, others out-of-band via text messages. Dirty hacks partially based on a Raspberry Pi, a bunch of shift registers, and a handful of relays and optocouplers.
I wouldn’t worry about IPMI per se; it is a powerful technology that can give you a lot of peace of mind. As long as it is you who has the keys and control.
And of course my new motherboard could’ve had IPMI if I knew I could buy one (but then it’d be about two to three times more expensive, and take a day more to source, so I’d have to turn that down). So back to the poor man’s hacks…
Todo: some hack to coax a raspi to sample VGA output and via a serial line and a microcontroller emulate a keyboard and maybe a mouse, and allow something like VNC for screen access down to the level of BIOS boot. No more having to dispatch somebody to “press F1 to continue”…
Remember that time the CIA was looking for Osama and gave half the polio vaccine and then never came back to finish the job and then everyone in Pakistan pretty much said “this polio vaccine thing is a total scam”?
Yeah, this just happened to the hard drive market. Sorry Thailand.
It’s mostly a “thing” in the server room. But it’s a significant thing.
The tl;dr is that most of the vendors of server-class hardware have these little raspberry-pi-sized pricessors attached to their motherboards, and most of the little processors are running Linux, and are rarely to never patched. And in many cases, they are connected to the public IPV4 internet with little to no firewalling, authentication, or security.
The purpose and intent of the little processor is that it allows the admin to power off, power on, reboot, and update the server without physical access to the system. So for really big ISPs, it saves the effort of locating a system in a campus/building/room/rack, and trundling back there and plugging in a laptop to look at its console log, patch, reboot, etc. The little processor has a separate battery and a separate network connector so it doesn’t go down when the server goes down. The ethernet connection on the IPMI is intended to be on the ISP’s internal LAN and firewalled (or not connected) from the internet but not every ISP has done that. And some hardware vendors have designed their IPMI cards to fall back and use the server’s main internet connection if their ethernet isn’t plugged in.
For a while, Dell servers were all using the same SSL cert for their IPMI cards, and the private key was in the firmware, which you could download from Dell. That’s the sort of security problem … the speaker in the video shows some other really dumb exploits that he has discovered for the things. Ways to either root the server or kill it remotely.
He said the vendors have patched these exploits, but of course not all admins at all ISPs are keeping the firmware in their IPMI cards up to date, as it doesn’t seem important.
Ahhh. So pretty much a bit more than but mostly what I cobbled together from a Raspberry Pi and optocouplers and chewing gum. With the same update issues, plus some dumb ones (hardcoded passwords, dumbheaded certs,…). Aggravated by it not having the form factor of a computer, so even the admins tend to forget about its presence as a computer.
The trouble with IPMI is that it is often implemented with the same…rigor and attention to detail…that characterizes $20 routers, but with a hell of a lot less 3rd party support and knowledge and much, much, deeper access to expensive hardware doing things better left un-snooped.
The concept isn’t so bad; but if you are going to implant something in a server’s brainstem ‘don’t screw up’ isn’t just a suggestion.
I suspect that Thailand would be a bit more worried if there were some sort of alternative…
This ‘equation group’ has exploits for basically all HDD vendors(possibly not all models; but no way to check, and I suspect that similarities across product lines make porting comparatively practical); SSDs aren’t mentioned explicitly; but tend to have even more onboard intelligence(and frequently a fair bit more cache RAM and a bunch of handy ‘reserve’ NAND capacity for hiding things in) so they are, or soon will be, even more dangerous(plus, while there are a lot more SSD ‘brands’, because slapping some NAND and a controller on a board is easy compared to building an HDD, there isn’t actually nearly as much variety in controllers and firmware. Not quite as buttoned up as HDDs; but much of the apparent diversity is just rebadge jobs or minimally altered reference designs.); and both USB and MMC/eMMC/SDIO devices also have integrated controllers powerful enough for proof-of-concept persistent exploits to have been demonstrated.
You’d have to go at least as far down the food chain as fairly dumb NAND/NOR flash packages(eg. no onboard wear leveling or bad block handling) to get something that doesn’t have enough intelligence to potentially bite you.
By contrast, skipping polio vaccines is relatively easy, if you don’t mind a few crippled children.
Sure I’m concerned about my government (and piles of US companies) spying on me, but does anyone worry about Kaspersky?
“But Kaspersky’s rise is particularly notable - and to some, downright troubling - given his KGB-sponsored training, his tenure as a Soviet intelligence officer, his alliance with Vladimir Putin’s regime, and his deep and ongoing relationship with Russia’s Federal Security Service, or FSB,”
It’s not really an either/or: Just as the honorable Mr. Snowden is currently in Russia(which is a terrible place to be a leaker of state secrets, or even an opposition journalist; but beats the ‘free world’ if you’ve just pissed off Uncle Sam a lot), I wouldn’t necessarily trust Kaspersky to not be in bed with the FSB(especially if installed in some really cool location, tipping your hand by owning ma and pa’s Win 8.1 with Bing cheapo box would be a waste of good secrecy); but they do have an unusually good track record, compared to western ‘security’ firms of actually looking into western fed malware, adding it to virus detection signature lists, etc.
There are probably some pure White Knights(I’m guessing that Richard Stallman isn’t secretly running the FSF in order to sneak an OSS backdoor onto your system); but it’s not clear there are nearly enough of them(or enough time away from paying work for them) to rely purely on the better nature of people who are very good at security.
I would very much suspect Kaspersky of potentially having their own entanglements, would not choose them to head up an ‘Is Vladimir Putin bugging my computer?’ investigation; but that doesn’t change their value when it comes to homegrown dangers, about which more local companies can be pretty feckless(even on much more minor matters, with no feds involved, remember how long it took the Sony rootkit to be deservedly detected and blacklisted by some A/V vendors. There wasn’t even a TLA leaning on them, the mere notion that a ‘respectable’ company was doing some nice, honest, DRM, was enough.
‘Trust’ just isn’t a unitary metric for these purposes. If I’m worried about the NSA, Kaspersky is probably my best buddy, since they get brownie points for rubbing egg in the face of the American spooks. If I’m worried about ruskies, or Chinese, I’ll probably have to go elsewhere.
The other thing that makes IPMI cards(or their embedded equivalents) nasty is that, in addition to being an untrusted computer on the LAN, usually with approximately the power of your basic embedded-linux-plastic-box, they often have some impressive(but scary) integration with the computer you care about. Any cheapy embedded box can serve as a backdoor for remote exploits; but a rooted IPMI card can probably reflash your BIOS, emulate HID events, and monitor your video output directly without any additional exploit against the target system.
Very well said, and that is one issue which irks me about this. It escalates global cyberwar. It gives carte blanche to any nation or group standing in the sidelines.
It is also very true that these attacks can not be performed without being caught. Which for one means that even if they get any useful, good intelligence from it – how can they trust it is true? It is across the world, and likely that nation’s counterintelligence caught it years before it was publicly released. Like in WWII, they inject truly damaging false information, and the other side does the very same thing. The information is useless, so… what is the point?
Which can get to another factor: passive intelligence gathering is used for governmental actions. Nations go to war on such intelligence, and when they get it they do not usually tell anyone where it came from but the heads in charge of the decision. Which means no one can really know what decisions are being based on. On what they say it is based on?
So passive intelligence has limited value… what happens next? Well, they have sent systems designed to destroy or definitely muck up nuclear sites. Why stop there? Grab foreign commercial intelligence and give it to your own corporations for an advantage. Control politicians like Hoover and the KGB did by extortion data. Subtly change important commercial and governmental documents to sabotage major projects. Your country’s auto industry is faring poorly? Cut the competition’s corporations down.
Stock market manipulation. Have a field day. Besides Unlimited Insider Information which is completely illegal but extremely powerful… there are all sorts of options to play with live financial trading data.
Where would any of this stop? Easy money, easy power… no reason it would stop.
Right now? Governments are attacking other country’s corporations. And they go for the money, too, the jugular. Why not? Countries need money too. Why would this trend abate in the least? Way I see it, at this rate, it would only grow. It is an espionage boom unlike anything ever before.
Besides all of that? Historically, countries that can, do, surveil their own people. FBI surveilled Martin Luther King Jr and ran very nasty operations against him. They wired the politicians and literally extorted presidents. Why not do this, when it means unlimited funding and control of the nation? And this kind of crime is exceedingly difficult to detect and route out.
For instance, who polices the NSA? The NSA. After Snowden revelations, a politician asked the NSA for documentation on their own policing. If I recall they returned something ludicrously small, like twelve cases.
Who policed the FBI for those many decades while they were illegaly surveilling and running illegal operations? Really? The FBI. The Justice Department above them and politicians at times banned actions they became aware of, and the FBI just kept on doing it anyway.
Not real sure how that slide towards totalitarianism could be stopped.