If Snowden returned to US for trial, would any NSA leak evidence be admissable?

[nonfer]

this predates the western millennium by twenty plus years.

[Rindan]

@nonfer

that you are paid by the government to collect for a private interest that then sells that data? i hope not for your sake. not attractive. good luck, i don’t envy you.

Dude, there are many more types of massive datasets than mass warrantless spying. Like I said, the dataset of all phone calls in the US isn’t actually that big. It is pretty boring sized compared to many datasets. If you are dying to know, I actually work with a number of very large data sets. The biggest ones I work with in terms of data size is a pattern recognition system that finds defects on computer chips. A camera looks at each and every single die, finds anything that looks amiss, and records some data and a picture in a database. It isn’t as a big as a NSA database, in terms of entries, but it is probably a couple orders of magnitude bigger than the NSA’s mass warrantless domestic spying database in terms of size due to the number of high quality pictures of it. The database of the electrical testing of these die is even bigger in terms of entries, but smaller in terms of size. I would guess it has roughly the same numbers of entries per days as the NSA mass warrantless domestic spying phone database.

Like I said, this sort of data is boring sized in terms of consumer databases. Google and Amazon easily blow the NSA’s database out of the water in terms of size and complexity with their databases that just try and rout web traffic. There is no special secret sauce the NSA mass domestic spying other than the fact that they are doing something everyone was pretty well convinced was illegal.

[nonfer]

looking at their activities… it kinda all looks like special sauce… with extra idiot. glad to hear things are good.

[Rindan]

@bwv812

The NSA didn’t keep the fact that it had surveillance operations authorized under the Patriot Act secret. They did keep exactly what they were doing secret, just like Google keeps its precise algorithm secret and Coke keeps its recipe secret.

Literally without exception, every single person who thought that the NSA wasn’t conducting mass warrantless domestic spying thought that they were not doing it because it was illegal. There is absolutely no exception to this. As I said, the only “secret sauce” is the fact that they were breaking what everyone thought was a law. Everyone realizes that the NSA can handle a boringly mid level database given to them by the phone companies. The phone companies can handle that database, we assume the NSA can too. The NSA kept this secret because it is fucking embarrassing and/or illegal. In fact, the first real judge to have a courtroom hearing in a real court agreed that it was in fact utterly illegal, just as everyone thought it was.

[quote=“bwv812, post:27, topic:17433”]
There are legitimate differences of opinion as to where data collection becomes surveillance. [/quote]

Hilariously, those euphemistically called “legitimate differences of opinion” have only been hashed out in real court once, and the real court, the kind with two fucking sides, agreed that the people collecting a massive database of all American phone call activities, to say absolutely nothing of the mass domestic internet spying, is illegal and clearly violates the shit out of the fourth amendment if it wasn’t obviousness enough on its face.

@bwv812

You’re right: ex parte proceedings where only one side is represented are far from ideal under the US system (although they do happen on occasion in the US legal system, and are more prevalent in some foreign legal systems), but it’s also true that despite the ex parte nature of the proceedings the FISC court has found that the NSA has failed to carry out the law correctly (most notoriously in failing to have RAS for their database queries), so they are clearly capable of pushing back against the NSA.

Let me translate this: True, this isn’t a real court by any definition of the word, but one time after the “court” found that the NSA was doing something so completely illegal that even the Soviet style kangaroo court, that doesn’t have two fucking sides, felt squeamish about it and gave the NSA a stern finger wagging.

@bwv812

FISA Courts do not deal with National Security Letters at all. FISC deal only with foreign intelligence matters. Because NSLs are domestic, they are typically requested by the FBI, not the NSA.

Semantics is the last argument of those who are completely wrong. True, it isn’t “national security letters”. I pulled it from memory. Let me correct it and expand it with a direct quote from the first Google hit I got:

But the FISC has declined just 11 of the more than 33,900 surveillance requests made by the government in 33 years, the Wall Street Journal reported Sunday. FISA Court Surveillance Rejections Extremely Rare

Yup, the FISA “court” is still clearly a flaming piece of shit Soviet style non-court that doesn’t know the definition of the word no.

[wrecksdart]

@bwv812

The NSA didn’t keep the fact that it had surveillance operations authorized under the Patriot Act secret.

Semantics at best. That the public knows the NSA is authorized to do stuff means nothing if the public also has absolutely no idea of what that stuff is. Paraphrasing Rindan’s thought as well as the public outcry in relation to the NSA’s surveillance, the public and even our elected lawmakers chosen to oversee these things had no clue as to the depth or breadth of the NSA’s reach into our private lives.

@bwv812

It’s also not at all straightforward that the NSA is conducting massive domestic spying.

Holy fucking shit that smacks of being willfully obtuse. It’s spying when they’ve cataloged each and every connection in a person’s address book having already stolen the address book. The most recent ruling on the data gathering led the Judge (U.S. District Judge Richard J. Leon, ala-President G.W. Bush), to pass on this tidbit (<a href-http://www.washingtonpost.com/national/judge-nsas-collecting-of-phone-records-is-likely-unconstitutional/2013/12/16/6e098eda-6688-11e3-a0b9-249bbb34602c_story.html>WashPo):

“I cannot imagine a more ‘indiscriminate’ and ‘arbitrary invasion’ than this systematic and high tech collection and retention of personal data on virtually every single citizen for purposes of querying and analyzing it without prior judicial approval.”

Judge Leon seems to feel pretty clear on that.

@bwv812

Note that under the law the NSA has to have reasonable articulable suspicion that the query will return information relevant to a foreign intelligence suspect in order to query the database.

I see that as being equal to the tried and true police method of bringing the drug dog to a traffic stop such that the dog can alert on the car and game on. I think the NSA’s version of reasonable would be something like, “American citizen X traveled to Greece. There are terrorists in Greece. Let’s look at every internet connection that person has ever made.” And how much do we know about NSA process in terms of who is making that decision?

@bwv812

Because NSLs are domestic, they are typically requested by the FBI, not the NSA.

And how many of those requests have been turned down? I feel safer already.

@bwv812

Don’t forget the third forum about how comments are written.

Meh.

[bwv812]

@Rindan

The NSA kept this secret because it is fucking embarrassing and/or illegal. In fact, the first real judge to have a courtroom hearing in a real court agreed that it was in fact utterly illegal, just as everyone thought it was.

This is ridiculous. The FISC was created in 1978 to deal with secret programs. These programs are not secret because they are illegal, they are secret because elected officials believe they would be less effective or compromise security if they were public. Ask Congress why the create secret programs, or why they gave FISC oversight of the NSA and its section 215 activities. I’m pretty sure they won’t say it’s because they knew the program is illegal.

And whether you like it or not, the FISC judges are real judges. They are members of the regular Article III Federal Courts who have also been appointed to a limited rotation on the FISC. “Real courts” also deal with ex parte proceedings from time to time, such as your everyday search warrants. Does this make them less real? Are all ex parte proceedings shams?

And what is your solution to the “kangaroo courts”? Presumably to have everything done through regular courts, and not through FISC. OK, lets get our warrants through regular federal courts. Oh wait! It turns out that regular search warrants are also granted through one-sided ex parte proceedings where only the government is represented! And it will be the very same federal judges who grant the warrants (in fact, it may be even worse because you may have magistrate judges—and not life-tenured Article III judges—making the decisions). Does this mean all search warrants are shams? Keep in mind that while regular search warrants are rejected at a higher rate than NSA warrants, I would still guess that as few as 1% of regular search warrants are rejected.

Finally, I don’t think it’s true that no one thought that the NSA was doing this sort of thing. Indeed, shortly after the Snowden revelations, some of his defenders were saying that they wouldn’t have a real effect on security because al Qaeda already knew they were doing these sorts of things. Others suggest that the real security implications of the Snowden leaks were not about the details they revealed but the publicity: the NSA only would have caught stupid terrorists with the various programs, and now even stupid terrorists would have been reminded not to be so stupid. But regardless of what you think of the merits of the entire security and secrecy arguments, the Snowden leaks were heavily analyzed by the press in terms of their impact on security: clearly there was also belief that the secrecy of the program had benefits.

@Rindan

Semantics is the last argument of those who are completely wrong. True, it isn’t “national security letters”. I pulled it from memory. Let me correct it and expand it with a direct quote from the first Google hit I got:
“But the FISC has declined just 11 of the more than 33,900 surveillance requests made by the government in 33 years, the Wall Street Journal reported Sunday. FISA Court Surveillance Rejections Extremely Rare”

Yes, I’m sure that facts and accuracy are the last refuge of the hopelessly wrong. Nevermind that being completely wrong who issues NSLs and who oversees them that is actually kind of important to the extent it demonstrates your grasp on the subject matter.

And if you want to talk about how only 11 of 33,900 requests have been denied, then it might be important to note that FISC judges have required the modification of at least 504 FISA warrants. This certainly suggests some oversight by a judiciary that isn’t simply rubber-stamping everything without any questioning. Yes, the numbers are still very low. But in the real world almost all normal search warrants—which, again, are granted in proceedings where only one side is present—are also granted. Although there don’t seem to be statistics on this, as I said above I wouldn’t be surprised if as few as 1% of regular search warrants are ever denied.

@wrecksdart

“It’s also not at all straightforward that the NSA is conducting massive domestic spying.”

Holy fucking shit that smacks of being willfully obtuse. It’s spying when they’ve cataloged each and every connection in a person’s address book having already stolen the address book. The most recent ruling on the data gathering led the Judge (U.S. District Judge Richard J. Leon, ala-President G.W. Bush), to pass on this tidbit (WashPo):

Why not actually address what I wrote? At what stage does it become “spying”? As soon as the data is collected? Are telcos also spying, then? I understand Judge Leon’s position, but it’s not the only one.

And I’ll be sure to let my old law professors know they’re being willfully obtuse when they bring up arguments like the one I presented.

@wrecksdart

I see that as being equal to the tried and true police method of bringing the drug dog to a traffic stop such that the dog can alert on the car and game on. I think the NSA’s version of reasonable would be something like, “American citizen X traveled to Greece. There are terrorists in Greece. Let’s look at every internet connection that person has ever made.” And how much do we know about NSA process in terms of who is making that decision?

The Reasonable Articulable Suspicion test certainly has its problems, but it’s far from unique to the NSA. It’s the basis for Terry stops (i.e., stop and frisk), and it’s something that every beat police officer does every day. They are given pretty broad discretion, and obviously make stuff up after the fact if they are challenged on it. Often, the RAS they give runs along the very subjective lines of “Based on my x years of experience as a police officer…” So is this a problematic standard? Yes. But it’s also very entrenched in the legal system, and has well-defined contours.

2: http://www.politico.com/story/2013/06/national-security-leaks-fallout-92554.html#ixzz2Vvjsc3ig[quote=“Rindan, post:44, topic:17433”]
Literally without exception, every single person who thought that the NSA wasn’t conducting mass warrantless domestic spying thought that they were not doing it because it was illegal. There is absolutely no exception to this.
[/quote]
Really? Everyone who thought they weren’t conducting this surveillance thought they weren’t doing it because it was illegal? So what about all the people who think that the program is at least constitutional under the 4th Amendment, if not completely legal? They all thought the NSA was collecting metadata the entire time? OK.

[anon34812172]

I wonder how much all that call data actually amounts to. I routinely work with about 1Tb at a time, sometimes multiple years of data for about 5Tb total. But usually, I’m stepping through the master dataset to get down to a few Gigs of stuff I actually need. I’m not actually “working with 1Tb” back-and-forth a bunch of times. It’s usually 1 trip through every few weeks. Chewing through that 1Tb takes about 15 minutes, since it’s billions of rows and a heavily used system with other users doing the same thing simultaneously.

I imagine people who work with 10Tb on lightly used systems can step through tens of billions of rows in about 10 minutes or less. And people who work with 100Tb would set up smaller test jobs to work out the bugs, then run full queries against the big data, taking about an hour or so. Or use a distributed system to get multiple processors searching in parallel.

And I’m sure geneticists, astronomers, physicists and other big data crunchers routinely work in the range of thousands or tens of thousands of Tb, i.e. in the Pb range, possibly bigger. Probably using their local cluster, which is about a room full of servers, costing a few million dollars.

A data center the size of the NSA’s in Utah would handle many MANY times the amount of data we’re talking about here. Not hundreds of Tb. Probably in the Exabyte or Zettabyte range. IOW, either thousands of Tb or millions of Tb. Just phone call metadata records per day would be several orders of magnitude less than that.

Let’s do a little math.

It’s estimated that there are about 2 Trillion call detail records in the NSA database.
(wiki-fu: http://en.wikipedia.org/wiki/NSA_call_database)

Here is a typical final call detail record:
event=B:Rel src_cgpn=32 cause=02_80_90 src_cdpn=7300961321 dst_cdpn=7300961321 dst_cgpn=32
(wiki-fu: http://wiki.innovaphone.com/index.php?title=Reference:Call_Detail_Record_CDR)

It’s 90 bytes.

90 * 2,000,000,000 = 180,000,000,000 bytes

or, about 167 Gigabytes. My laptop could handle a database this size.

So let’s assume they are collecting more than 90 bytes for each call. Let’s say they throw in a bunch of other stuff, not even compressed. Let’s say each CDR+stuff is about 1k, or 1024 bytes.

1024 * 2,000,000,000 = 1.9 Tb

Still miniscule. Nothing you’d need a whole datacenter for.

So, makes you wonder what they are actually storing there in Utah. And makes you wonder how long they’ll keep lying to us about it. Any idiot, including me, can do the math and see that the reality of what they are doing is not matching up to what they are saying.

[bwv812]

@awjt

A data center the size of the NSA’s in Utah would handle many MANY times the amount of data we’re talking about here. Not hundreds of Tb. Probably in the Exabyte or Zettabyte range. IOW, either thousands of Tb or millions of Tb. Just phone call metadata records per day would be several orders of magnitude less than that.
[. . .]
So, makes you wonder what they are actually storing there in Utah.

Well, they’re obviously not storing only a single copy of anything, but are using massively redundant RAID systems for speed and redundancy.

But there’s a good Wired article from last year about what they’re doing in Utah and their planned storage capacity:

As a result of this “expanding array of theater airborne and other sensor networks,” as a 2007 Department of Defense report puts it, the Pentagon is attempting to expand its worldwide communications network, known as the Global Information Grid, to handle yottabytes (1024 bytes) of data. (A yottabyte is a septillion bytes—so large that no one has yet coined a term for the next higher magnitude.)
It needs that capacity because, according to a recent report by Cisco, global Internet traffic will quadruple from 2010 to 2015, reaching 966 exabytes per year. (A million exabytes equal a yottabyte.)

[anon34812172]

Yep, I’ve read that. But they haven’t admitted anything. They and Obama have only so far said they are collecting “metadata.”

[technogeekagain]

@awjt

So, makes you wonder what they are actually storing there in Utah

I don’t think anyone ever claimed that this is all the data center is doing.

Much more than raw telephone connection records, certainly. Even if it was primarily phone info, don’t forget that they’re going to be massaging it, correlating it with itself and with data from other sources, maintaining the higher-level data needed to actually extract patterns… which, given that simply deriving the conversational proximity info they’re looking for is something of an N-squared problem, is likely to involve a lot of duplicating of data into other structures optimized for that purpose. Admittedly that only ups things by an order of magnitude or so, but I suspect we’re still under-spec’ing the problem.

[anon34812172]

True. I think a huge part of it really is an encryption cracking facility. But the original point is that while they admitted to storing metadata, they have a facility that is sized millions of times larger. It’s a laughable absurdity which feels like an outright lie:

Q. Why did you build the strongest vault in the world?
A. To store a few pennies I picked up on the sidewalk.

[bytefyre]

I hope you recognize the irony/hypocrisy in your post.

[bytefyre]

This reminds me of what happened to Tim DeChristopher. He wasn’t allowed to use anything with which he could defend his actions, the difference of course being that he didn’t face nearly the kind of punishment that Snowden would. http://en.wikipedia.org/wiki/Tim_DeChristopher
Perhaps the government should just have an announcement “We’ll do whatever we want and anyone who thinks otherwise should probably give the freedom dick a suck until they undestand how freedom works, God Bless America”

[technogeekagain]

My impression was that this is more of

Q: Where do you store the pennies you pick up on the sidewalk?
A: In the strongest vault in the world.
Q: Why, for gosh sakes?
A: I already have that vault and it has plenty of space.
Q: Why did you build that vault in the first place?
A: I’m not going to answer that.

They haven’t lied; people just asked the wrong questions and assumed.

[anon34812172]

@technogeekagain

They haven’t lied; people just asked the wrong questions and assumed.

No, they’ve lied all right. They don’t talk substantively about the datacenter. When pressed by external factors, the President was forced to admit they are collecting metadata. So add up the things they’ve admitted, the things leakers have leaked and the actions the NSA are taking, and there is a huge disconnect, which is deception, which is lying.

[technogeekagain]

If you call not volunteering information lying, point granted. I use a different definition.

[anon34812172]

True. As a personal, simple omission, that’s not necessarily lying. It could be, but not always. On a higher order, a more institutionalized form of deception, it is.

[bwv812]

@awjt

So add up the things they’ve admitted, the things leakers have leaked and the actions the NSA are taking, and there is a huge disconnect, which is deception, which is lying.

Do you think that there’s ever any reason that justifies having secret programs or classified materials?

[anon34812172]

Of course. It’s just totally way wayyyyyy disproportionately out of control now, on every level possible.

[Rindan]

@bwv812

This is ridiculous. The FISC was created in 1978 to deal with secret programs. These programs are not secret because they are illegal, they are secret because elected officials believe they would be less effective or compromise security if they were public. Ask Congress why the create secret programs, or why they gave FISC oversight of the NSA and its section 215 activities. I’m pretty sure they won’t say it’s because they knew the program is illegal.

There is a difference between an operational secret, and keeping something secret because it is embarrassing and/or illegal. We didn’t keep the fact that we were out to kill Bin Laden secret. We did keep the operation secret. After the operation was complete, we revealed the secret and fair amount of the detail around what happened. Why? Because we were not embarrassed and happy to stand behind it. We knew that if the public learned what had happened, the US public would be a-okay with it. The mass warrantless collection of every phone record on the other hand was not kept secret for operational reasons. It was kept secret because Americans would be pissed, there would be a political price to pay (as we are seeing right now), and it would face real judges in real courts with two sides. I am all for operational secret, but not secrets that are there simply to avoid the legal system or negative political consequences. This is a fucking democracy, negative political consequences means that you are doing it wrong.

The FISA court isn’t there to protect spy agencies from being embarrassed or to protect them from the constitution. It is there to protect legal operational secrets.

@bwv812

“Real courts” also deal with ex parte proceedings from time to time, such as your everyday search warrants. Does this make them less real? Are all ex parte proceedings shams?

Yes, all ex parte proceedings are in fact a shame. This is why all such proceedings in real courts can be challenged after the fact. You can get a search warrant and keep it a secret for operational reasons, but if the warrant was improperly given the results of the search are trashed. Every time a warrant is used, the target of the warrant in the US will at some point get to challenge it.

@bwv812

And what is your solution to the “kangaroo courts”? Presumably to have everything done through regular courts, and not through FISC.

Presumably, it would be to have two sides. At an absolute bare minimum, there should be someone appointed through a counter balancing method who is there to defend the liberty and constitutional rights of private citizens. If there is an obvious legal grey area like “can I collect every single phone record on every single American”, there should be a side arguing “no”. In an ideal world that defense, we be able force legal question to go through real courts where America’s wonderful civil liberty protecting organizations like the EFF and ACLU would be able to join the challenge. I know the constitution and organization that defend liberty are really inconvenient for the NSA, but tough shit.

@bwv812

And if you want to talk about how only 11 of 33,900 requests have been denied… [insert more redirecting here]

Cool, we agree that the FISA court doesn’t have no in its vocabulary and is a rubber stamp. I mean, they approved the collecting of every single fucking American’s phone record. I am pretty sure that I am not a terrorist, so I know at least one of those requests was issued erroneously. That is kind of what makes this so hilarious. The number of requests is a few tens of thousands, but at least one of those requests asked for a couple hundred million people’s records, assuming they bothered to ask at all.

@bwv812

Really? Everyone who thought they weren’t conducting this surveillance thought they weren’t doing it because it was illegal? So what about all the people who think that the program is at least constitutional under the 4th Amendment, if not completely legal? They all thought the NSA was collecting metadata the entire time? OK.

Yes, really. We know any boring mid sized IT department could handle the data, and we know that the NSA would want the data. Literally every single person who thought that the NSA wasn’t doing this thought that the NSA wasn’t doing it for purely legal reasons. The only people on this planet, literally without exception, who believed that the NSA wasn’t conducting mass warrantless spying on Americans thought that they refrained because it was illegal. Everyone who believed that it was legal or that the NSA doesn’t give a shit about legality believed that the NSA was spying. There are literally no exception to this. Everyone before Snowden, without exception, agreed that the NSA had the capacity to conduct mass warrantless domestic spying and that they would do it if they thought it was legal. The only thing Snowden revealed is that NSA thought it was legal. The only people surprised by the Snowden leak are people who thought that the NSA was held in check by law. Everyone else was unsurprised. In other words, the only operational secret here is that NSA was violating the shit out of the constitution or that the constitution doesn’t protect you from mass warrantless domestic spying by military spy agencies.

Even all of this is red herring bullshit. The mass warrantless spying on phone records is small potatoes compared to the fact that the NSA spies on all e-mail traffic, broke into American companies to spy on their internal traffic (without a warrant), paid off businesses to intentional poisoned public encryption, and whenever this stuff saw any sort of court challenge (which it often didn’t until well after the fact or simply not at all), it was a worthless secret court with one side that says no about as often as a crack whore jonesing for the next hit. Further, this is just the stuff we know about. Now that we know the NSA is basically unchecked by law, your most paranoid imagination is likely not all that paranoid anymore.