> when it comes to large-scale webapps (e.g. Obamacare, UK NHS systems, UK taxation systems.)
There's absolutely no reason to believe the spy tech isn't as poorly implemented and terrible value for money as the examples you mention. Given their secretive nature it's even more likely to be the case I'd suspect.
This is my thinking as well. A lot of what spy tech is, is the ability to throw a warrant or NS letter into someone's face and force them to open the gate. The mythical uber-competent security services seem to be a pretty questionable premise. The few criminals caught via these tools are almost always the low hanging fruit of unconnected crazies, who seem to be harmless cranks and not real terrorists. Five eyes didn't stop or predict Snowden, Assange/Manning, Boston bombers, the rise of ISIS, Putin's advance on Crimea and Eastern Ukraine, the massive cyberwar Russia and China have declared on us, etc. In fact, all these things were major surprises to these security services, at least from what we can tell.
There's a lot of federal tax money spent on what is cargo cult science and cargo cult intelligence. The NSA has the ability to buy a lot of PhD's but they are stuck with the same batch of incompetent and profiteer federal contractors the rest of the government is stuck with.
It can't help that the Secret Service is losing top talent to private security firms like Academi (aka Blackwater), which draws a parallel with parts of the tech industry.
It makes me wonder exactly how much worse it is in the Intelligence industry. I agree with others in this thread, given the secrecy involved I expect it to be orders of magnitude worse than what we see in the private sector.
It also makes me wonder about the private Intelligence industry, about how much better they must be than their government counterparts, and about how they work together with government to "fight terror" and other demons. Combine that with the idea that government IT is losing talent to private firms in the same way... Maybe we should be more worried about private Intelligence firms than we currently are.
These days too many of them are a bit intimidated by the mere presence and aura of people in uniform, intel people, cops etc...Years ago, many of them had served in the military or a had a very good understanding of what they can really do (not what they claim they can do) how they worked/the types of people behind them(JFK, Churchill etc) so were rightly sceptical of their claims. I mean, has anyone ever met a cop/spook/politician who didn't want more power to do X, more money to do Y, more people, more secrecy etc?
One of the main problems for intel agencies is that they are merely one conduit to policy-makers, amongst many these days. Ultimately it is those policy-makers who need to (and increasingly are incapable of) using this information for strategic decision-making...Tactically it is easy to say "arrest/shoot that guy," because they get kudos and less blame if some thing goes wrong. Strategically it is harder to say "don't shoot that guy, keep him alive and in 5 years we can negotiate with him." It took the British a long time to learn how to do this in Northern Ireland, and it was only when they did that they managed to start to move towards peace with the IRA. I'm not sure would that be possibly in the bloggy, politically polarised, short-term, tabloid, Sky/Fox News type of world we live in these days.
After all, even if they did predict the rise of ISIS, Boston, it is the policy-maker who must decide what action to take. Difficult when:
a) There are sooo many warnings coming across their desk
b) Limited resources
C) Not many real policy instruments available. For example, the US public would not have been up for attacking for acting on intel available about ISIS 18 months ago, thus that policy option was thus not available.
>The few criminals caught via these tools are almost always the low hanging fruit of unconnected crazies
This is true of the criminals caught via FBI counterterrorism's more traditional police work that ends in civilian courts.
It remains entirely possible that the intelligence establishment enjoys some success in penetrating more sophisticated groups, but doesn't write press releases about it (since it will be acted upon with further covert operations to get further into those organizations, or with secret assassinations.)
Why does every post on this subject always have someone come in to slam home the notion that the government is always inept in everything they do? Its really good propaganda for keeping the public pacified. But it doesn't reflect reality. The government as well as its component organizations are not monolithic, and are made up of both bafoons and geniuses. Based on conversations I've had with people that have been part of the apparatus, the CIA is known to be somewhat incompetent when it comes to computer tech, but the NSA is phenomenal.
Stop apologizing for the government, whether conscious or not.
Because they keep proving it over and over again? (at least in certain problem domains they do.)
Government activities are generally important to a lot of people, if not because said people are direct consumers of the alleged benefits then because they paid for them. We keep paying and they keep ballsing it up, and everything we know about the theory of government (in an economic sense) tells us this is a design flaw and, ergo, not something that can be expected to self-resolve.
And it's not that they're totally inept all the time (although ineptitude is a recurring theme) - sometimes they do get good stuff done. The much harder question to answer in these cases is how efficient was it? How much tax money was shat away in admin/stupid compliance BS, and is all that wasteful pomp really a necessary price to pay for democracy? Could there be another way?
It's all very contentious, high-stakes stuff and if you disagree strongly enough with it all to wilfully not comply you'll soon find the whole system is propped up on various threats: they'll take your money, your home, your pride - they may even take you. And they'll do it all under a banner of moral righteousness that turns it into you vs. "society". How could that ever not be a talking point?
I talk about it most days in one way or another, and not because I'm some horrible right-wing fanatic who wants to wipe my ass with money and glee in the suffering of others, but because it's an incredibly fragile, wasteful system that perpetually sits on a knife edge and I suspect does just as much harm as good. It's misguided in so many ways and so, so hard to change that all I can do is moan about it and hope that my moans contribute in some way to getting the volume of the debate loud enough to think that something could change.
> And it's not that they're totally inept all the time (although ineptitude is a recurring theme) - sometimes they do get good stuff done. The much harder question to answer in these cases is how efficient was it? How much tax money was shat away in admin/stupid compliance BS, and is all that wasteful pomp really a necessary price to pay for democracy? Could there be another way?
I think this largely misses the mark, for several reasons.
1) The problem with government isn't ineptitude. At least the federal government is probably average to above-average in terms of competency. But what's the competency level of the IT folks in your average Big Corp where IT isn't a front office department? It's not very good. Take those not very good IT folks and then make them do projects 10x the size and complexity of what your typical Big Corp handles, in areas where the tolerance for error is nil (health care, benefits), and political concerns make things like iteration and building MVP's intractable.
2) It's easy to talk about "admin/compliance BS" because you don't have to point to any actual procedure that you might have to defend as being BS. It's not clear to me that the federal government has any more internal red tape than your typical Big Corp.
I spent a short time at the FCC, after working as an engineer in the wireless sector, and it was pretty eye-opening. On one hand, it was quite lean and efficient for what it is tasked with dealing with. At the top it's five commissioners, a few very experienced policy advisors, and a rotating staff of interns doing grunt work. On the other hand, the necessary complexity is staggering. For every decision, there are dozens of stakeholders. Things that were no-brainers from a purely engineering point of view became multi-facted problems combining social justice issues and economic issues.
I think engineers, software engineers in particular, confronted with complexity, seek to eliminate it. Something is hard? Do less. That's usually just not an option with government.
>Because they keep proving it over and over again?
The incredibly complex and expensive purchasing/hiring system that seems to fail every single time exists because we don't trust government employees to spend our money. Perhaps we shouldn't. But you cannot deny that the system designed to prevent improper spending is the cause of a hell of a lot of improper spending.
Government agencies are often legally bound to purchase from the lowest bidder even when they know the product will be "technically correct" but extremely low-quality. Because the regulations and processes are so complex, it is incredibly expensive and difficult to sell to the government, meaning the few companies that actually manage to do it have higher costs and less competition. Of course prices are high.
Sure, the system is necessary to stop incompetent/malicious actors from wasting money or throwing contracts to their cronies. But it also stops a lot of smart and diligent people from getting the resources they need to complete their projects.
If you want government agencies to be able to purchase high-quality goods and services at reasonable prices, then you're going to need to loosen the chains. And that probably means some fraud. And that probably means some agencies buying slightly nicer stuff than the absolute cheapest thing that would work on paper. But the overall effectiveness of government might end up higher, and its cost much lower, if you were to eliminate the "government services" industry and open it to the wider market.
Honestly, government is pretty much the same as a big company. The wild card is that you have a strata of extra executive management that may be brilliant and fighting the professional workforce or incompetent but capable of screwing things up.
I work for a large state government. Plenty of Dilbert moments, but important stuff gets done. There are people all over the place that have dedicated their life to what they do and are masters of their craft or policy area. In IT, we have plenty of crap, but a few things that are world class and make a real difference in people's lives.
I've worked for small companies where the "friends & family plan" ruined key aspects of the business/workplace. I've also seen large companies with all manner of incompetence and bad behavior -- one bank leader I know actually built a business unit with no useful purpose... specifically for the purpose of having chum for a layoff.
Lowest bid policies are not just stupid, they're obviously stupid. So not only do you have an obviously stupid policy, you've got the obviously stupid system that settled on such an obviously stupid policy in the first place. Given this mountain of stupidity, it is safe to conclude that the government is not being run nearly as inefficiently as it could be in the absence of such multifaceted idiocy.
Did I say the people are dumb? No, I said the rules are dumb and the system that formulated those rules is dumb. Extending this to the people running the system was an inference you made, not something I actually said.
As it turns out, the current rules are actually very effective if what you want is a system tailor made to produce large and profitable overages after any danger of losing a contract to a competing contractor has been mitigated.
HN used to have a libertarian lean to it back when it was a smaller community of programmers who were starting companies like Dropbox and AirBnB, people discussing SICP/Lisp, and willing to fill the homepage with Erlang articles to scare aware newbs. Now that it has been fully transitioned into mainstream site and much more like reddit, it is much less libertarian and more /r/politics.
So that generalization doesn't apply so cleanly here anymore.
There's no need to merely suspect. You can be (depressingly) sure thanks to Thomas Drake, who blew the whistle on the NSA for not only not doing things the right way, but for spending gargantuan sums of money to do things in a way so wrong that the only thing done at all was something that flatly violated the Constitution.
Well said. And even more of a reason to insist they respect your privacy. Even if you think "you don't have anything to hide," you still don't want your personal information held on a system designed by anyone with that government contractor level of QA/QC.
There's absolutely no reason to believe the spy tech isn't as poorly implemented and terrible value for money as the examples you mention. Given their secretive nature it's even more likely to be the case I'd suspect.